Beyond Spatial Frequency: Pixel‑wise Temporal Frequency‑based Deepfake Video Detection

Interactive Demos

Abstract

We introduce a novel method for deepfake video detection that utilizes pixel‑wise temporal frequency spectra. Unlike previous approaches that stack 2D frame‑wise spatial frequency spectra, we extract pixel‑wise temporal frequency by performing a 1D Fourier transform on the time axis per pixel, effectively identifying temporal artifacts. We also propose an Attention Proposal Module (APM) to extract regions of interest for detecting these artifacts. Our method demonstrates outstanding generalizability and robustness in various challenging deepfake video detection scenarios.

Method & Architecture

• Pixel‑wise Temporal Frequency: 1D FFT on the time axis per pixel to capture subtle artifacts.
• Attention Proposal Module (APM): Weakly‑supervised focus on temporal‑artifact regions.
• Joint Transformer: Fuses global/part‑based frequency signals and spatio‑temporal context.

Experiments & Results

• SOTA performance on FF++, CDF, DFDC, KoDF, etc.
• Strong cross‑deepfakes and cross‑synthesis generalization.
• APM effectively identifies temporally inconsistent regions.

Limitations & Future Work

Known Limitation

• Heavy compression (H.264/JPEG/WebP) merges neighboring pixels, weakening pixel‑level motion → temporal‑frequency shift.
• Low‑frequency components align with raw signals, but high‑frequency components diverge after compression.

Future Work

We will investigate temporal‑frequency regularization to mitigate compression‑induced degradation.

📚 Citation

@misc{kim2025spatialfrequencypixelwisetemporal,
  title        = {Beyond Spatial Frequency: Pixel-wise Temporal Frequency-based Deepfake Video Detection},
  author       = {Taehoon Kim and Jongwook Choi and Yonghyun Jeong and Haeun Noh and Jaejun Yoo and Seungryul Baek and Jongwon Choi},
  year         = {2025},
  eprint       = {2507.02398},
  archivePrefix= {arXiv},
  primaryClass = {cs.CV},
  url          = {https://arxiv.org/abs/2507.02398}
}

🙏 Acknowledgement