Vision Pro: eye tracking allows passwords to be reconstructed – theoretically
Security researchers have shown a way to reconstruct sensitive data based on the eye movements of users of Apple's headset. The company reacted.
Vision Pro with user.
(Image: Apple)
There is a saying that the eyes are the window to the soul. At the very least, you can tell from them what mood someone is in and how he or she is feeling at the moment. US IT security researchers have now shown that you can also find out what someone is typing through their eyes and thus obtain sensitive data such as passwords, PINs or the content of a private message. The attack, which is currently only theoretical, was carried out on Apple's Vision Pro mixed reality headset.
The attack, published under the name GAZEploit, uses Apple's Persona function. They did not need to access the headset to do this. Instead, they analyzed the eye movements of the virtual 3D avatars captured by the Vision Pro's internal cameras. The persona avatars can be used in Zoom, Microsoft Teams, Reddit, Slack, Tinder, X, Skype or in FaceTime calls, for example, replacing the missing video image of the Vision Pro user.
Avatar revealed keyboard strokes
It was possible to reconstruct the keystrokes because eye tracking can be used on the Vision Pro as an alternative to typing directly with your hands in the air. If you have the right letter in view, it can be selected with a finger gesture (tap with thumb and index finger), which is more comfortable for some users.
Videos by heise
The knowledge of what the user is doing is actually reserved for the wearer of the headset; however, anyone who taps with their eyes while participating in a Zoom call, livestream or FaceTime call with their avatar is obviously revealing more than expected. The users' virtual avatars mirror their facial and eye movements.
Biometric data as an attack vector
The attack is based on two types of biometric data that the researchers extracted from the video recordings of an avatar: firstly, the aspect ratio of the eyes and secondly, the estimated direction of gaze. First, the researchers used a neural network to identify the patterns that can be used to recognize whether someone is typing. To do this, they fed the deep learning model with video recordings of the avatars of 30 different people typing with their eyes on a QWERTY keyboard in the Vision Pro.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes YouTube-Video (Google Ireland Limited) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (Google Ireland Limited) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
They then used geometric calculations to find out where and in what size the users had positioned their keyboards in the air in front of them. If the video material was sufficient to reconstruct the position and size of the keyboard, it was possible to recognize the subsequent keystrokes, one of the researchers involved told the US technology magazine Wired.
Gap closed, avatar is hidden
Apple has now fixed the vulnerability, which was assigned the CVE ID 2024-40865, and released the fix to users with the visionOS 1.3 software update. However, the company is taking a rather brutal approach: If you now use the virtual keyboard while participating in a video call or livestream, the glasses simply interrupt the sharing of the avatar, making it invisible in the meantime. Apparently, Apple couldn't think of any other solution at first.
Although the attack was developed in the laboratory and no Vision Pro user was hacked as a result, malicious attackers could theoretically have misused the biometric data leak in the glasses to develop a similar method. The researchers' work only shows a specific risk in gaze-based typing. Cheng Zhang, an assistant professor at Cornell University who studies the development of wearables to interpret human behavior, told Wired that this exploit is just one possible example. Users of wearables often overlook how much information the devices can collect and disclose about them and the data protection risks this entails.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externer Preisvergleich (heise Preisvergleich) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (heise Preisvergleich) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
(kst)
