💓Heartian: Physiology-Aware Relightable Gaussian Head Avatar

People

Xiaoyue Fan1	Jose Echevarria2	Akshay Paruchuri3	Kaan Akşit1

¹University College London, ²Adobe Research ³Stanford University

SIGGRAPH Asia 2026 Technical Communications

Resources

Manuscript Supplementary arXiv Code

Bibtex

@inproceedings{fan2026heartian,
      author = {Fan, Xiaoyue  and Echevarria, Jose  and Paruchuri, Akshay  and Ak{\c{s}}it, Kaan },
      title = {{💓Heartian: Physiology-Aware Relightable Gaussian Head Avatar}},
      booktitle = {SIGGRAPH Asia 2026 Technical Communications (SA Technical Communications '26)},
      year = {2026},
      month = {December 01--04},
      publisher = {Association for Computing Machinery},
      location = {Kuala Lumpur, Malaysia}, 
      pages = {4},
      isbn = {979-8-4007-2841-9/2026/12},
      doi = {10.1145/3829339.3847838},
      url = {https://arxiv.org/abs/2609.28539}
      }

Video

Abstract

Gaussian head avatars typically model intrinsic facial appearance as temporally static, omitting subtle cardiac-induced skin-color variation. We propose 💓Heartian, a physiology-aware modulation framework that learns cardiac-cycle-dependent per-frame albedo modulation of facial skin-region Gaussians within a relightable head avatar to encode remote photoplethysmography (rPPG) signals. Using synchronized contact PPG supervision, 💓Heartian models the prescribed cardiac waveform as the sum of two Gaussian functions and learns per-frame spatial residuals via a lightweight MLP. Across 152 stationary recordings from UBFC-rPPG, PURE, and MMPD, attribute-space recovery of the supplied signal achieves a pooled recording-level heart-rate MAE of 0.29 bpm and MAPE of 0.38%. The signals remain detectable after rendering by benchmark rPPG methods, with the best tested configuration - a motion-augmented TS-CAN decoder pretrained on UBFC-rPPG - recovering heart rate from the rendered MMPD avatars at 0.97 bpm MAE and 1.21% MAPE. Meanwhile, 💓Heartian maintains reconstruction quality comparable to the baseline, with negligible average PSNR degradation of 0.005 dB. Overall, our work embeds recoverable rPPG signals as controllable material attributes to subject-specific Gaussian head avatars, while retaining reconstruction quality.

Proposed Method

We propose 💓Heartian, a physiology-aware modulation framework that embeds prescribed rPPG signals into relightable Gaussian head avatars. Building on HRAvatar reconstruction, we optimize per-frame spatial modulation factors acting on the albedo of facial skin-region Gaussians.

For each facial skin-region Gaussian as illustrated in red in the above figure, a spatially varied transient offset is applied to the green channel albedo \(c\) on a per-frame basis, while all other attributes remain unchanged.

\[ \mathbf{c}_i^{\text{m}}(t) = \mathbf{c}_i^{\text{base}}(t) + A \cdot m_i(t), \]

The PPG waveform within each cardiac cycle characteristically exhibits two distinct peaks, a systolic and a diastolic wave. Their structure can be embedded in phase space by mapping the temporal signal to a unit circle parameterized by the cumulative cardiac phase \(\theta(t)\).

A typical PPG waveform with systolic (blue) and diastolic waves (red) in the time (a) and phase domains (b).

Therefore, the modulation comprises two components: a fundamental waveform modeled as the sum of two Gaussian functions and a lightweight MLP that learns per-frame spatial residuals.

\[ m_i(t) = \underset{\text{fundamental}}{\sum_{k=1}^{2} A_k \exp\left(-\frac{d(\theta(t), \mu_k)^2}{2\sigma_k^2}\right)} + B \cdot \underset{\text{residual}}{f_{\text{MLP}}(\theta(t), b(t), p_i)\vphantom{\frac{d}{2\sigma_k^2}}} \]

where \(d(\theta, \mu) = \arctan2(\sin(\theta - \mu), \cos(\theta - \mu))\) denotes the shortest angular distance on the unit circle. The fundamental component models the characteristic PPG waveform morphology as a sum of two Gaussian functions in phase space, with learnable centers \(\mu_k\), widths \(\sigma_k\), and amplitudes \(A_k\). \(\theta(t)\) is modeled as a learnable initial phase \(\phi_0\) accumulated with per-frame increments \(\{\delta_k\}\), which are passed through a softplus activation to ensure positivity, and projected onto the unit circle as:

\[ \begin{aligned} \phi(t) &= \phi_0 + \sum_{k=0}^{t} \delta_k,\\ \theta(t) &= \arctan2(\sin(\phi(t)), \cos(\phi(t))). \end{aligned} \]

The residual term is a lightweight \(f_{\text{MLP}}\) conditioned on \(\theta(t)\), a beat index embedding \(b(t)\), and the normalized spatial position \(p_i\) of each skin-region Gaussian.

The rPPG signal is extracted as the mean green channel intensity after modulation across the skin-region Gaussians, supervised against the ground truth PPG waveform.

Conclusion

Our method enables the recovery of heart rate from embedded signals with a mean MAE of 0.29 bpm and MAPE of 0.38% against ground truth PPG measurements while preserving comparable reconstruction quality after modulation, with a marginal cost of 0.005 dB, 0.00003, and 0.0003 in average PSNR, SSIM, and LPIPS, respectively. Meanwhile, embedded rPPG signals are vastly preserved in the rendered videos, evaluated with the benchmark methods by rPPG-Toolbox, exhibiting particularly strong performance on the MMPD dataset, as shown in the Table 1 below.

Table 1. Benchmark Evaluation Results. Performance of benchmark methods on our rendered 💓Heartian videos, evaluated across datasets. Green highlights indicate metrics where embedded signals remain recoverable on par with the original benchmark results on the selected subsets.

Method	Train Set	UBFC-rPPG		PURE		MMPD
		MAE↓	MAPE↓	MAE↓	MAPE↓	MAE↓	MAPE↓
POS	-	1.23 ± 0.77	1.01 ± 0.62	14.23 ± 6.78	29.34 ± 13.98	1.53 ± 0.63	2.66 ± 1.20
TS-CAN	UBFC-rPPG	-	-	10.01 ± 4.52	17.89 ± 8.68	2.59 ± 0.52	3.89 ± 0.94
	PURE	9.22 ± 7.47	7.41 ± 5.56	-	-	2.51 ± 0.57	3.87 ± 1.03
TS-CAN (MA)	UBFC-rPPG	-	-	4.74 ± 4.22	5.23 ± 4.67	0.97 ± 0.27	1.21 ± 0.31
PhysFormer	UBFC-rPPG	-	-	17.40 ± 6.55	34.29 ± 13.78	8.69 ± 1.31	13.96 ± 2.21
	PURE	5.53 ± 4.44	4.37 ± 3.32	-	-	3.12 ± 1.08	5.27 ± 1.95
FactorizePhys	UBFC-rPPG	-	-	14.58 ± 6.82	29.95 ± 14.13	2.88 ± 0.98	4.83 ± 1.70
	PURE	6.41 ± 5.09	5.05 ± 3.80	-	-	1.38 ± 0.50	2.40 ± 0.99
Ours vs Baseline
POS	-	− 21.97	− 18.88	− 10.72	− 13.78	− 12.72	− 17.78
TS-CAN	PURE	− 26.98	− 24.72	-	-	− 12.87	− 16.28
TS-CAN (MA)	UBFC-rPPG	-	-	− 15.73	− 25.01	− 13.31	− 17.28
MAE = Mean Absolute Error in HR estimation (Beats/Min), MAPE = Mean Percentage Error (%).

Besides, our approach of embedding rPPG signals directly into the Gaussian albedo offers attribute-level signal preservation. As Table 2 shows, 💓Heartian Heartian recovers the heart rate information missing from the static HRAvatar baseline. Compared with results derived from the rendered videos in Table 1, the extracted signals show stronger fidelity, suggesting that rPPG information is numerically preserved within the avatar representation when it attenuates through rendering or post-processing.

Table 2. Attribute-level Evaluation. rPPG signal metrics extracted from the Gaussian albedo of baseline and 💓Heartian.

Dataset	Baseline			Ours
	MAE	MAPE	SNR	MAE↓	MAPE↓	SNR↑
UBFC-rPPG	59.00	53.81	−21.52	0.00	0.00	1.45
PURE	21.53	25.37	−7.13	0.17	0.27	3.47
MMPD	39.11	34.81	−11.47	0.33	0.42	4.04

Lighting condition and skin tone have been challenging factors for rPPG signal estimation. However, benefiting from our attribute-level embedding strategy, well-learned signals remain largely embedded and recoverable across predefined external conditions. Our pipeline also enables rPPG avatar synthesis under novel illumination environments without compromising recoverable signal quality.

Furthermore, our strategy enables prescribed waveform and heart-rate control within physiology-aware Gaussian representations, potentially supporting controlled physiological training augmentation and informing future physiology-aware real-time avatar models for applications such as telemedicine.

Photo gallery

Relevant research works

Here are relevant research works from the authors:

• Editing Physiological Signals in Videos Using Latent Representations
• Heart rate monitoring via remote photoplethysmography with motion artifacts reduction

Outreach

We host a Slack group with more than 250 members. This Slack group focuses on the topics of rendering, perception, displays and cameras. The group is open to public and you can become a member by following this link.

Contact Us

Warning

Please reach us through email to provide your feedback and comments.