The Tsinghua team has developed a base model that can read heart rates, giving machines "eyes" that can read people.

A team of doctoral students from Tsinghua University decided to give robots a pair of eyes that can “read people.”

Around June 10, Beijing FacePhys Technology completed multi-million-dollar financing, led by Shunwei Capital. In one sentence, what this company does is: enable an ordinary camera to detect your heart rate, breathing rate, heart rate variability in real time, then combine facial expressions and eye movements to judge whether your smile is genuine or fake, or whether you’ve reached physical exhaustion—all without physical contact, with local on-device inference, and latency below 10 milliseconds.

They have packaged this capability into a foundation model called FacePhys, claiming it to be “the world’s first foundation model capable of real-time understanding of physiology and emotion.” This claim is not an exaggeration—academia has been working on rPPG (remote photoplethysmography) for more than a decade, but achieving medical-grade heart rate accuracy, compressing the model down to 0.2M parameters to run on ordinary phones, and training it according to the foundation-model paradigm is indeed rare.

The invisible ceiling of AI interaction is “not being able to see you”

Let’s first discuss the underlying logic of why this direction is suddenly being pursued.

Over the past two to three years, large models have evolved with a clear bias: language capabilities have skyrocketed, visual understanding is catching up, but perception of “the person” is almost non-existent. When you chat with ChatGPT, it cannot tell whether you’re angry or anxious at that moment; embodied robots can recognize your actions but cannot read your thoughts. A commonly cited research figure says that 55% of human expression is non-verbal information. To current AI, this part is essentially a black box.

The result is that all AI interactions rely heavily on the user’s “explicit input.” If you don’t type, speak, or make obvious movements, the model can only wait passively. This paradigm of passive response fundamentally limits the “active empathy” capability of agents or robots.

FacePhys believes that to break this ceiling, large models need to be supplemented with a “physiological-emotional data input.” In other words, first let the machine understand you as a person before discussing interaction.

rPPG is not new—the hard part is eliminating noise

The principle of remote photoplethysmography (rPPG) is quite clever. Each time the heart beats, blood flows through skin capillaries, causing the skin’s light reflectance to change periodically. The change is tiny and invisible to the naked eye, but a camera can capture it. Extracting this signal allows reverse calculation of heart rate, breathing rate, and blood oxygen.

It sounds simple, but implementing it is full of pitfalls. The signal strength accounts for only 0.1%–1% of overall background light, and even minor interference can drown it out. FacePhys founder Tang Jiankai stated in interviews: “It’s perfect in the lab, but once you’re in real-world conditions, even slight movement or changes in lighting cause accuracy to drop sharply.”

They solved this problem along two paths:

First is data accumulation. The team created a clinically annotated dataset of tens of thousands of people, with hundreds of millions of sample points, covering different skin tones, lighting conditions, and movement states, and developed with Anzhen Hospital a clinical cohort validation involving thousands of participants. Such a scale is rare in rPPG academia—most papers still use public datasets of a few hundred people to improve scores.

Second is switching the modeling paradigm. This is more interesting. They introduced “State Space Models” into physiological signal modeling.

Tang Jiankai explained the analogy clearly: large language models predict the next token; FacePhys predicts “the human physiological behavioral state at the next moment.” Heartbeat is no longer stitched together from discrete video frames but modeled as a continuous physical process. This modeling method captures time-dynamic features more accurately and aligns better with the physical laws of heartbeat itself.

The final results of this approach:

• Heart rate detection error < 0.7 BPM (latest disclosed data), meeting medical-grade standards
• On-device inference latency ≤ 10ms
• On-device model parameters 0.2M, runnable on ordinary phones and cameras
• Real-time output per frame of 120+ indicators, covering heart rate, HRV, breathing rate, facial action units (AU), eye movement features, emotion dimensions, and voice features

The parameter scale of 0.2M is worth noting. At a time when visual models often have billions of parameters, compressing physiological perception ability to this size, while still running in real-time on-device, means it has the engineering foundation to become a “standardized perception component”—rather than yet another research project that can only run demos.

Key weapon: HRV physiological-emotional barometer

Measuring heart rate is nothing special—smartwatches on the market can do it. What FacePhys really aims to do is “read emotion,” and its killer feature is the HRV physiological-emotional barometer.

HRV (heart rate variability) refers to tiny variations in the time interval between consecutive heartbeats. This metric has been thoroughly validated in medicine—it is a non-invasive indicator of autonomic nervous system function, showing significant negative correlation with depression and chronic stress. Simply put: people chronically under high pressure or emotionally suppressed usually have lower HRV, and this is not something you can fake.

FacePhys ties HRV to acute emotion, then combines it with facial action units and eye movement data to accomplish something previous emotion recognition systems could not—distinguish fake smiles from real smiles and identify suppressed emotions.

The value of this capability lies in its “unforgeability.” Traditional facial-expression-based emotion recognition essentially reads the performance on your face; HRV reads your body’s true reaction. When AI receives “objective physiological truth” instead of “subjective performance,” the credibility of interaction changes entirely.

The real target customers: robots

FacePhys’s commercialization path is already quite diversified: infant monitoring (infrared-assisted nighttime heart and breathing monitoring), smart cockpits (fatigue-driving alerts + in-car infant monitoring, already integrated into next-generation “health cars” in some automakers), clinical hospital applications (the “health mirror” in cooperation with Anzhen Hospital), and personal apps and browser-based real-time monitoring systems.

But these are vertical scenarios. Tang Jiankai’s real bet is on the robotics track.

His logic: service robots, elder care companion robots, and humanoid robots urgently need the ability to “understand users’ physiological state and emotional intent.” A robot without this ability is merely a moving tool. FacePhys aims to become the standardized perception module for robot manufacturers—just like LiDAR’s role in autonomous driving.

This reasoning makes sense at this point in time. Mass production of humanoid robots is ramping up towards 2026, but the industry is generally stuck at “robots can work but cannot read facial expressions.” If an rPPG module can let a companion robot detect abnormal blood oxygen in an elderly person, or let a service robot sense a user’s impatience—this kind of “soft perception ability” cannot be achieved by piling up hardware specs.

FacePhys currently offers the industry three integration solutions:

• Cloud API/SDK: zero hardware modifications, two-week integration, SaaS subscription
• Complete module: delivering full camera modules, with algorithms hardened into chips, low latency and low power consumption, suitable for flagship mass production
• Plugin mode: linking to existing cameras, without altering original architecture

These three tiers essentially cover scenarios from lightweight trials to deep mass production.

A reality check

Enough hype—let’s get into less optimistic points.

The rPPG track has been studied in academia for many years, and industrial implementation has been lukewarm—not just due to technology, but more to do with business models. FacePhys also took a wrong turn in 2024, creating an online teleconsultation commission platform modeled on overseas examples, but ran into the reality that China’s medical system is completely different from overseas: domestic offline consultations are efficient, and demand for light consultations is far lower than in the US. They lost that battle.

The later pivot (focusing on non-contact physiological-emotional perception + building perception modules for robots) is correct, but essentially a B2B hardware accessory business. The ceiling of such a business depends on downstream demand—if humanoid robots fail to boom as expected, rPPG module penetration will stall.

Another potential risk is privacy. A camera continuously reading your heart rate and judging your emotions—even without uploading data—may face questionable user acceptance. FacePhys emphasizes local on-device deployment and privacy-leak avoidance at the foundational level, which is necessary but not enough—how regulators will define the compliance boundaries of “non-contact physiological data” is currently unclear.

Lessons for large models

Putting aside FacePhys as a company, the FacePhys paradigm offers insights for the large-model industry.

Over the past two years, multimodal large models have focused on vision-language, but the vision side mostly stays at “object recognition + scene understanding.” FacePhys offers a new perspective: there is another layer of ‘physiological signal’ within the visual modality, which is orthogonal to language and images. If future foundation models can incorporate this layer, AI’s understanding of humans will rise to a new dimension.

Imagine: while collaborating with you on a task, an agent can sense that you’re obviously anxious at a certain step, and proactively pause to explain; a companion robot notices an elderly person’s HRV is persistently low and emotional suppression, and proactively suggests contacting family—this “active empathy” loop cannot be achieved solely by language models.

FacePhys’s approach is to make this capability into a “data input,” then embed it into the large-model ecosystem. Whether it’s worth watching depends on two factors: first, in what form integration with mainstream large models happens (API? multimodal alignment?); second, whether robot manufacturers are willing to make it standard.

As a side note, OpenAI Hub has been tracking multimodal model integration, and if foundation models like FacePhys that provide physiological-emotional input open interfaces, they could be valuable supplements—allowing a key to trigger not only models that can talk but also models that can read people.

This direction is far more interesting than just scaling parameters.

References

External public reports and research materials in this article come from multiple tech media exclusive reports and interview resources, including interviews with FacePhys’s founding team, disclosure of product technical parameters, and financing information. Due to restricted source links, readers are advised to search for “FacePhys Technology,” “Tsinghua rPPG foundation model” for more first-hand information.