Table as keyboard and touchpad: What's Meta's futuristic input method worth?
Meta Quest can now project keyboards and touchpads onto any surface. How well does this new control concept work?
Meta's new surface keyboard and touchpad.
(Image: Tomislav Bezmalinović / heise medien)
The surface keyboard is a new feature introduced with Horizon OS 85. It is currently being rolled out gradually to users. According to the release notes, only Meta Quest 3 is supported. Quest 3S and older headsets remain excluded for now.
As this is an experimental feature, you first need to enable it in the settings under "Experimental". Afterward, it can be set up under "Devices", but only with the English localization of the operating system.
To use the surface keyboard, you need to place your hands flat on the desired surface. After a short scanning process, the flat outlines of the keyboard and touchpad appear on the table (see video below). Practical: The position of the keyboard can be adjusted afterwards via a digital grip bar. You can also place the touchpad to the left of the keyboard, but it remains attached to it and cannot be freely moved around the room.
A different form of keyboard
The surface keyboard supports multi-finger input as well as simultaneous pressing of the Shift key for capitalization. Activated keys light up briefly, providing visual hit feedback. The system captures the outlines of the hands and cuts them out of the digital projection in real-time, so that the hands remain visible and realistically obscure the keyboard.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes Video (TargetVideo GmbH) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (TargetVideo GmbH) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
According to Meta, the speed and accuracy of the input method are around 37 words per minute with an error rate of 3 percent. This puts it on par with text input on a smartphone.
In our tests, the surface keyboard turned out to be good enough for composing short WhatsApp messages or entering search terms in the browser. For this article, we took notes with the keyboard while testing the new feature.
Videos by heise
The virtual keyboard is only suitable for more demanding tasks such as word processing to a limited extent. The lack of key feedback and the higher error rate quickly become tiring. Furthermore, important keys that are familiar from physical keyboards, including the arrow keys, Tab, and not least Ctrl, are missing in the current version. You also won't find umlaut keys, as currently only an English keyboard layout is supported.
Speaking of missing key feedback: Since you don't feel keys under your fingers and it's difficult to estimate finger position on the smooth surface, typing without looking at the keys is significantly harder. Therefore, Meta displays a text field for input directly above the keyboard. This way, the keyboard, and text remain in view simultaneously.
Virtual touchpad surprisingly useful
Technically, the surface keyboard is a small feat of engineering: it relies solely on optical hand tracking and a neural network that can often predict the finest finger movements even when fingers obscure each other and are only partially captured by the Quest's cameras.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes Video (TargetVideo GmbH) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (TargetVideo GmbH) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
However, the purely optical operating principle has inherent disadvantages. You cannot type in the dark because the cameras no longer recognize your hands. And if the viewing angle is too flat, for example, because you lean back too far, the system works less reliably.
According to Meta, the virtual touchpad supports left-click by tapping, drag-and-drop by double-tapping and then moving, and vertical and horizontal scrolling with the index and middle fingers. In our test, we found that a right-click, for example to open a context menu, is also possible by tapping simultaneously with both fingers. Curiously, clicking with a finger apart from the index finger is not supported. In the settings, the sensitivity of the touchpad tracking, scroll speed, and other parameters can be adjusted.
In practice, we liked the virtual touchpad better than the touch keyboard because it is more versatile: it is well-suited for system navigation as well as for tasks such as browsing and multitasking, without having to resort to controllers or classic hand tracking.
Innovation with work in progress
Meta's solution cannot replace a physical keyboard. However, it is significantly superior to the previous "air keyboard" of many VR headsets in terms of typing speed and haptics, and it also introduces a virtual touchpad. What seems like a curiosity today is likely to become a standard feature of VR headsets in a few years.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes Video (TargetVideo GmbH) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (TargetVideo GmbH) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
There is still a lot of room for improvement in terms of functionality. However, Meta has already indicated that this will not be the first version. It will be interesting to see if and how Meta's Neural Band will complement this input method: it could support haptics and refine keyboard input via muscular signal detection.
The solution is held back by Meta's VR operating system, which still feels like a collection of disparate control concepts that get in each other's way rather than complementing each other organically. It remains to be seen whether Meta will get this problem under control with Horizon OS 2 and the recruitment of Apple's UI design chief.
(nie)
