Apple releases visionOS 26.4 Beta 1: Collaborates with NVIDIA to unlock "cloud-based mods," 3A games bid farewell to graphics anxiety

IT House February 18 News, Apple announced the release of visionOS 26.4 Beta 1 yesterday, introducing support for NVIDIA CloudXR and “Foveated Streaming” technology.
Supporting NVIDIA CloudXR
IT House Note: NVIDIA CloudXR is a streaming solution based on NVIDIA RTX technology, supporting the transmission of high-fidelity virtual reality (VR) and augmented reality (AR) content from remote servers to portable devices.

In simpler terms, it’s a technology that allows the headset to “borrow” a remote computer’s graphics card, similar to cloud gaming on a phone—complex visuals are rendered by a powerful remote computer and streamed directly to your headset.

visionOS 26.4 further breaks down ecosystem barriers. Since NVIDIA CloudXR is currently a widely adopted third-party streaming technology in the VR field, native support for this protocol means that existing large libraries of PC VR games and high-performance industrial software can be ported to the Apple Vision Pro platform at lower cost and with better experience.
Introduction of “Foveated Streaming” Technology

The core logic of “Foveated Streaming” is intelligent allocation of computing power and bandwidth. The system will real-time utilize the eye-tracking data of Apple Vision Pro to accurately determine the user’s gaze focus. In the area where the user is looking, the system transmits the highest quality images; in peripheral vision areas, the quality is appropriately reduced. This strategy, which the user almost cannot perceive, greatly reduces data transmission pressure and ensures smooth cloud streaming experience.

The most exciting feature of this update is support for “Hybrid Layer Rendering.” According to Apple developer documentation, developers can overlay locally generated spatial content with remotely streamed content.

To demonstrate the potential of this technology, Apple provided two specific examples: racing games and flight simulators:

In developing a racing game, close-up details such as the dashboard and steering wheel can be rendered locally on the device using RealityKit to ensure instant responsiveness; while the highly graphics-intensive dynamic scenery and lighting effects outside the window are rendered by a remote high-performance computer or cloud server and streamed to the headset.

In a flight simulation application, the complex cockpit instruments are driven by the local chip of Apple Vision Pro to ensure clear and sharp readings; meanwhile, the vast landscape outside and weather systems are rendered via the cloud.