Symptoms of drunkenness: researchers investigate the effects of mixed reality

Contents

Mixed reality headsets differ fundamentally from AR glasses and conventional visual aids: Instead of looking directly through glass at their surroundings, users see the world mediated by cameras, displays and special lenses. This technology is called "passthrough" and is still relatively new in consumer electronics. Its effects on perception and physical sensation are correspondingly little researched.

Researchers at the Norwegian University of Science and Technology in Gjøvik and Trondheim have investigated how passthrough devices affect walking, dexterity and whole-body coordination.

The study involved 20 people between the ages of 18 and 77. They were randomly divided into two groups. The first group used the "Merge" low-end headset, a foam holder based on the Google Cardboard principle, into which a Pixel 6 smartphone was inserted as a display. Its camera also enables pass-through. The second group used Meta Quest 3, a current and technically much more advanced mixed reality headset.

Negative effects with both devices

With the passthrough activated, the participants had to complete several tasks that tested walking and orientation skills, dexterity in handling everyday objects as well as body coordination and balance. The last task was modeled on motor tests from alcohol tests: Participants were asked to walk from a point heel-to-toe in a straight line to a target point and back, and then balance on one foot for 30 seconds.

Various questionnaires, self-reports and qualitative findings from observations and interviews with the participants were used for the evaluation. The researchers came to the conclusion that both passthrough devices have a negative impact on orientation, well-being and everyday task accomplishment. "Both systems fall short of the promise of seamless interaction and integration of reality. Participants showed symptoms similar to alcoholization, including loss of coordination, general malaise, and difficulty maintaining focus and concentration," the researchers write in their summary.

Following the cybersickness known from virtual reality, the researchers suggest a separate research category for pass-through-specific symptoms, which they call "CybAR sickness", in reference to the augmented reality that pass-through systems generate.

Mixed reality headsets are bad glasses

A look at the individual results shows that the group with Quest 3 performed significantly better in most tasks and at the same time showed fewer and milder symptoms. However, use was not symptom-free here either: the participants showed a certain degree of hesitation and uncertainty with both devices. The researchers also emphasize that the tasks were simple and were carried out in an ideal environment. This meant that many challenges that users might encounter in real-life scenarios, such as stairs, uneven terrain, changing light conditions or obstacles at ground level, were not included.

With this in mind, it is not surprising that Meta and Apple recommend only using their mixed reality headsets in safe environments. Usually at home rather than outdoors.

The limitations of current passthrough technology are well known: The world conveyed by cameras, displays and pancake lenses loses color, sharpness, depth and brightness. The field of view is only 100 degrees, which is about half the width of the natural human field of vision. Natural focusing is not possible, and if the light in the surroundings decreases, the image quickly appears grainy. As the external cameras cannot be mounted in the eye position, algorithms have to correct the image in real time, which can lead to high latency and distortion. Mixed reality headsets therefore significantly impair human vision.

Long-term consequences are unknown

This problem could be mitigated in the future. Meta Quest 3 and Apple Vision Pro are among the first devices to seriously test pass-through technology. In future generations, hardware and software will mature considerably and thus mitigate one or two of the limitations. Thanks to software updates, the pass-through of the Quest 3 is already significantly better than when the headset was launched. For their study, the researchers used version 59 of the Quest operating system, which did not yet benefit from later improvements in resolution, image quality and distortion correction.

Incidentally, the long-term effect of passthrough was not investigated. The tasks deliberately lasted less than 15 minutes. But what will happen if mixed reality headsets are one day so light and powerful that you can work with them or wear them for hours on end in everyday life? What effects would they have on the visual system, the ability to concentrate, cognitive strain and general well-being? Research has yet to find answers to these questions.

The study entitled "Are you drunk? No, I am CybAR sick!" is freely accessible online.

(afl)