Physics-Aware Deepfake Detection via Distance–Speech Consistency

Kyeongrae Kim1✉ Kim Sung-Bin2✉ Oh Hyun-Bin2✉ Tae-Hyun Oh1,†✉

1 KAIST   2 POSTECH

INTERSPEECH 2026

† Corresponding author

TL;DR We detect deepfakes through distance–speech physical consistency, complementing lip-sync cues.

Authentic and deepfake video examples: lip-sync cues can fail when lips are obscured or well synchronized, while our method checks distance–speech consistency.
Physics-aware detection complements lip-sync. Our method checks whether speech acoustics are consistent with the speaker’s distance, providing a complementary cue when lip details are unreliable.

Abstract

Recent progress in generative models has made talking-head deepfakes highly realistic and easy to create, jeopardizing public trust. However, most audio-visual deepfake detectors rely on lip–speech synchronization and are designed for static, frontal videos, limiting their reliability in dynamic, in-the-wild recordings where the speaker moves and lip cues are degraded or missing. As an alternative, we propose a physics-aware detector for dynamic speaking videos that leverages an acoustic constraint: in real recordings, speech energy varies predictably with speaker-to-camera distance, whereas deepfake generation can break this coupling. Our method combines distance estimates from video with distance-dependent acoustic measures from speech to detect deepfakes. Experiments on in-the-wild benchmarks show that the proposed physical cue is effective in dynamic scenarios, and that it complements lip-sync-based detectors, with a simple ensemble achieving consistent gains across datasets.

Real follows physics, fake breaks physics

Poster comparison of real and deepfake videos: the real example shows an inverse relationship between speaker distance and acoustic measures, while the deepfake example shows inconsistent trends.

Distance–Speech Physical Consistency

As a speaker moves, distance and speech acoustics change together.
We use this physical relationship as a cue for detecting deepfakes in dynamic speaking videos.

Framework: face landmarks and monocular depth estimate speaker distance; sound separation yields acoustic measures; distance-wise binning and statistical modeling produce features for an MLP detector.

01 Estimate distance

RetinaFace locates the speaker’s facial region. MiDaS depth estimates provide a proxy for relative speaker-to-camera distance.

02 Measure acoustics

SAM Audio separates speech from background noise. Signal-to-noise ratio (SNR) and clarity (C50) capture distance-dependent acoustic behavior.

03 Detect inconsistency

Distance-wise bins aggregate acoustic variance and entropy. Five statistical features feed a lightweight MLP to predict a deepfake score.

Key Results

Scroll to explore

All Physical Features Improve Deepfake Detection

Poster ablation charts comparing the full model with removal of SNR, C50, variance ratio, entropy, and variance features on DF24 and RW.

Sound Separation Improves Detection

Poster chart comparing detection with and without SAM Audio sound separation on DF24 and RW.

Lip-Sync And Our Physics-Aware Method Are Complementary

Poster chart comparing SpeechForensics and AuViRe, our physics-aware model, and their ensembles on DF24 and RW.

Our physics-aware method complements lip-sync.

BibTeX

@inproceedings{kim2026physics,
  title     = {Physics-Aware Deepfake Detection via Distance--Speech Consistency},
  author    = {Kim, Kyeongrae and Kim, Sung-Bin and Oh, Hyun-Bin and Oh, Tae-Hyun},
  booktitle = {Proc. Interspeech 2026},
  year      = {2026},
  pages     = {2424--2428},
  doi       = {10.21437/Interspeech.2026-1541}
}