Treacherous vibrations: Eavesdropping on cell phone calls with radar and AI

A team of computer scientists at Penn State University has succeeded in eavesdropping on cell phone conversations using millimeter wave radar. The radar detects the vibrations on the surface of the cell phone that occur when speaking. The vibration patterns are decoded by artificial intelligence (AI).

The researchers had already used a similar method in 2022 to eavesdrop on conversations made on a cell phone. At that time, however, the eavesdropping performance was lower. The system achieved an accuracy of 83% with only ten predefined keywords.

The method now developed can decode more words. A millimeter-wave radar sensor is used for this purpose. The technology is used in autonomous vehicles alongside lidar, for example, to estimate distances. In eavesdropping technology, small vibrations caused by speech on the surface of a cell phone are recorded. However, the quality of the data is rather inferior, as the researchers write in their study "Wireless-Tap: Automatic Transcription of Phone Calls Using Millimeter-Wave Radar Sensing", which was published in the proceedings of the ACM WiSec 2025 security conference. It must therefore be possible to interpret the very noisy data.

Transcription by AI

The scientists use the open source AI speech recognition model Whisper for this purpose. Whisper is actually used to transcribe unambiguous audio data. The researchers used a low-rank adaptation technique from machine learning to train Whisper to interpret the radar data. This meant that the scientists did not have to create the speech recognition model from scratch.

The specially tuned AI model was able to create transcriptions for a vocabulary of up to 10,000 words from the data. The accuracy was 60 percent. Overall, this is a significant improvement on the method used in 2022, which could only recognize ten words. However, even with the new system, what is intercepted must be put into context, interpreted and corrected if necessary. This is also the case with lip-reading interceptions, where only between 30 percent and 40 percent of spoken words are recognized. However, when put into context, this results in a high level of speech comprehension.

The eavesdropping system using radar and AI currently only works at a distance of up to 6 meters. With their research results, the scientists want to draw attention to how easily vulnerabilities can be exploited by attackers and sensitive information can be intercepted. They therefore wish to focus their attention on possible defensive measures in the future.

(olb)