Remote Rendering with OpenXR

Magic Leap 2 offers remote rendering capabilities through OpenXR. OpenXR helps address application compatibility issues across devices by exposing a set of standardized APIs that allow your application to run on any system that works with OpenXR. This guide provides an overview of common features you may want to use, and best practices for working with them through OpenXR.

note

Suggestions in this guide may be optional in the specification, but are helpful when you want to get optimal performance for remote rendering.

Environment Blending

After the compositor blends and flattens all layers, including layers from the system, the image is presented to the Magic Leap 2 headset's display. When possible, use ALPHA_BLEND as the blend mode to submit frames, and set the alpha portion of the RGBA image as appropriate for your headset.

When setting alpha for your image, it is expected that the alpha is "Pre multiplied." An alpha value of 0 will result in no segmented dimming, but will not reduce the brightness of the RGB image, creating a blend of real and virtual. Stronger values will increase the strength of the segmented dimming as well as the opacity in camera capture, up until full strength is achieved. If you want to do simple dimming at full strength, set alpha to the maximum value.

You can read more about environment blend modes in section 10.4.6 Environment Blend Mode of the OpenXR Specification.

Depth

You can provide depth information to your application with XR_KHR_composition_layer_depth. XR_KHR_composition_layer_depth allows your application to submit depth information from its renderer to the OpenXR runtime.

Support for XR Controllers

Magic Leap 2 has its own controller profile. However, you can also access it using the Simple Controller profile.

You can use the sample code to get started, it is straightforward to use. In general, you define explicit actions your user can do, for example 'Teleport.' Then you map what action maps to what button on the controller.

If you need the 3D position of the controller, it's done by creating and mapping an Action Space for it. Then you query the space (using the same methods as you would for any XrSpace) and obtain the controller's location.

Magic Leap Remote Render OpenXR Extensions

Magic Leap Remote Render utilizes OpenXR to standardize XR application development. The table below lists the supported OpenXR extensions, with links to the official documentation and descriptions.

Latest Extensions

See the official Khronos Documentation to see the latest extensions supported by the Magic Leap Remote Render OpenXR Runtime.

Extension	Description
XR_EXT_dpad_binding	Enables the binding of directional pad (D-pad) inputs to actions for more precise user interactions.
XR_EXT_eye_gaze_interaction	Supports eye gaze as an input method, allowing applications to track where the user is looking for interaction.
XR_EXT_hand_interaction	Provides support for hand-based interactions, including gestures and pose tracking.
XR_EXT_hand_tracking	Enables precise tracking of the user’s hands, allowing for natural hand interactions in virtual environments.
XR_EXT_plane_detection	Detects horizontal and vertical planes in the environment, used for placing virtual objects or interactions within the real world.
XR_KHR_D3D11_enable	Provides support for Direct3D 11, enabling rendering on Windows platforms.
XR_KHR_D3D12_enable	Provides support for Direct3D 12, enabling advanced rendering features on Windows platforms.
XR_KHR_composition_layer_cylinder	Supports rendering content on cylindrical layers, useful for displaying panoramic content.
XR_KHR_composition_layer_depth	Allows applications to submit depth information for accurate compositing of virtual objects with real-world environments.
XR_KHR_composition_layer_equirect2	Enables rendering of 360-degree content using equirectangular layers.
XR_KHR_opengl_enable	Provides OpenGL support for cross-platform rendering, including Windows and Linux.
XR_KHR_vulkan_enable	Enables Vulkan support, providing high-performance graphics rendering on compatible platforms.
XR_KHR_vulkan_enable2	Extends Vulkan support with additional features and optimizations for advanced rendering capabilities.
XR_KHR_win32_convert_performance_counter_time	Converts Win32 performance counter time to OpenXR time, facilitating synchronization across systems.
XR_ML_global_dimmer	Controls global dimmer settings, adjusting the brightness of the real-world view through the headset.
XR_ML_ml2_controller_interaction	Defines specific interaction profiles for Magic Leap 2 controllers, ensuring precise and consistent input handling.
XR_KHR_binding_modification	Allows applications to dynamically modify the binding of actions to inputs, providing flexibility in user interactions.
XR_MND_swapchain_usage_input_attachment_bit	Supports input attachment usage for swapchains, enabling advanced rendering techniques.
XR_EXT_debug_utils	Provides utilities for debugging OpenXR applications, aiding in troubleshooting and optimization.