Apple Vision Pro: What does the new "Foveated Streaming" bring?
Apple is introducing "Foveated Streaming" for Apple Vision Pro with visionOS 26.4 Beta. The same technique will also be used for Steam Frame in the future.
A new framework for visionOS supports streaming PC and cloud content to Vision Pro.
(Image: Apple)
Apple has introduced visionOS 26.4 Beta, and among the new features is a framework for Foveated Streaming.
Apple's Vision Pro has so far offered limited options for accessing the computing power of an external computer. While the 2D interface of a Mac can be mirrored to a large virtual display, VR apps cannot be displayed through it. Apple announced a corresponding solution at WWDC 2025 under the name "Spatial Rendering," but further details have been scarce since then.
An alternative is offered by the open-source tool ALVR. It allows games and apps from SteamVR to be rendered on a Windows or Linux computer and streamed to the high-resolution displays of the Apple Vision Pro. Since the tool supports PSVR2 controllers, it is suitable for much more than just flight and racing simulations.
With the release of the Foveated Streaming framework as part of the visionOS 26.4 Beta, Apple is making progress in this area again. The framework gives visionOS developers the ability to stream 2D and VR content from an external computer or the cloud to Apple Vision Pro. Developers must implement Nvidia CloudXR's streaming protocols in their application, but can run them on a local machine without using Nvidia's servers.
Videos by heise
In the developer documentation, Apple lists a number of use cases for the new framework: transferring high-quality games from a powerful computer, CAD software, and accessing a remote desktop.
Furthermore, locally rendered content can be combined with streamed content: As an example, the company cites a flight simulator where the cockpit is calculated on the Apple Vision Pro, while the graphically demanding external environment is rendered by an external computer and transmitted to the VR headset.
Steam Frame also relies on Foveated Streaming
"Foveated Streaming" refers not only to the framework but also to a technique of the same name: it uses the headset's eye-tracking to optimize the encoding quality of the stream for the area of the image the user is currently looking at, while the peripheral vision area is more heavily compressed. This increases the perceived image quality while reducing bandwidth requirements. The user normally doesn't notice this trick.
An existing application that could benefit from this technique is the aforementioned open-source project ALVR. We have inquired whether an implementation is being considered and will update this article if we receive a response.
Empfohlener redaktioneller Inhalt
Mit Ihrer Zustimmung wird hier ein externes Video (TargetVideo GmbH) geladen.
Ich bin damit einverstanden, dass mir externe Inhalte angezeigt werden. Damit können personenbezogene Daten an Drittplattformen (TargetVideo GmbH) übermittelt werden. Mehr dazu in unserer Datenschutzerklärung.
Foveated Streaming is also one of the central features of Valve's upcoming VR headset Steam Frame. According to Valve, Foveated Streaming can increase image quality and effective bandwidth by more than a factor of ten.
Foveated Streaming is not to be confused with Foveated Rendering. In the latter technique, it is not the encoding quality of the stream that is optimized, but the image rendering in the central field of vision, which can save a lot of computing power.
Both terms refer to the fovea, the tiny area in the center of the retina responsible for sharp vision when focusing on a point. Outside this area, visual acuity decreases significantly, which is hardly noticeable due to constant eye movements. This is an effect that these methods exploit.
(dahe)
