May 30 2025 – Apple Inc. has recently unveiled a groundbreaking study titled “Foundation Model Hidden Representations for Heart Rate Estimation from Auscultation,” which could pave the way for future AirPods to monitor users’ heart rates if commercialized.
The research delves into leveraging foundation models initially designed for speech recognition to estimate heart rates by processing phonocardiogram (PCG) data. PCGs are graphical representations of the heart’s mechanical vibrations converted into electrical signals.
In their experiments, Apple researchers tested six mainstream models, including Whisper, wav2vec2, and wavLM, alongside their in-house CLAP (Contrastive Language-Audio Pretraining) model for comparative analysis.
The study utilized the publicly available CirCor DigiScope Phonocardiogram dataset, comprising approximately 20 hours of heart sound recordings. Each recording varied in length from 5.1 to 64.5 seconds. The research team divided the audio files into 5-second segments, moving one second at a time, resulting in 23,388 heart sound segments for heart rate conversion measurements.
The findings revealed that Apple’s CLAP model achieved the lowest mean absolute error (MAE) across different data segmentations, outperforming benchmark models trained on traditional acoustic features. The researchers attributed CLAP’s superior performance to its training data, which encompassed a broader range of non-speech content, enabling it to better capture heart sound-related features.
According to Apple’s researchers, foundation models tailored for speech processing can be effectively adapted for auscultation and vital sign estimation tasks, serving as a robust alternative to traditional methods. They also observed that larger model sizes do not necessarily correlate with better heart rate estimation accuracy, but further fine-tuning could potentially enhance precision.
This technology holds promise for pathological analysis of heart and lung sounds, aiding in more accurate detection of abnormalities such as arrhythmias and murmurs.
