Interoperability between MLSDK and OpenXR APIs

Overview

This document explains how to use the legacy MLSDK APIs with the Magic Leap OpenXR Unity SDK in Unity. Using the legacy APIs allows you to access additional features that are not yet supported by the Magic Leap OpenXR SDK, such as MLWebView, MLCamera, MLOcclusion, and MLVoice.

As the Magic Leap OpenXR SDK matures, we plan to migrate all of the features into OpenXR compatible features and vender extensions.

Compatibility

As a general rule of compatibility, all existing ML APIs should continue to work when using the Magic Leap OpenXR Unity SDK with the exception of

• Graphics related features such as Global / Segmented Dimming and Headlocked mode
• Subsystem related logic such as Input and Meshing

These features are either incompatible with the OpenXR rendering pipeline or require a different approach to access them. For more information, see the OpenXR Migration Guide.

MLSDK Perception Snapshot Support

To use legacy MLSDK APIs that provide pose information, such as MLMarkerTracker or MLCVCamera.GetPose(), it is necessary to enable the Perception Snapshots setting within the OpenXR Magic Leap Support feature in the Unity Editor. This setting ensures that these MLSDK APIs can retrieve accurate pose data.

What are Perception Snapshots?

Perception Snapshots are essential for various CAPI calls that need to be captured every frame. In the legacy Magic Leap Unity SDK, these snapshots were linked to the Input subsystem. With the transition to OpenXR, Unity’s built-in Input system now handles input processing, so these legacy snapshots are no longer taken automatically.

By enabling the Perception Snapshots setting, the Unity SDK introduces a PlayerLoop call to perform these snapshots, ensuring that legacy MLSDK APIs continue to function correctly when used with the OpenXR plugin.

Relevant APIs

The following Magic Leap XR SDK APIs rely on Perception Snapshots to obtain pose data:

• MLCamera: Provides access to the camera stream and allows image and video capture. Refer to the MLCamera API documentation for more details.
• MLOcclusion: Enables the occlusion of virtual objects behind real-world surfaces. For more information, see the MLOcclusion API documentation.
• Depth Camera: Accesses depth data from the device’s depth sensor. More information is available in the Depth Camera API documentation.
• World Camera: Provides access to the world camera stream. For details, see the World Camera API documentation.
• MLMarker Tracking: Detects and tracks the location and orientation of two-dimensional markers. For more information, refer to the Marker Tracking API documentation.

Enabling Perception Snapshots

Follow these steps to enable MLSDK Perception support when using OpenXR:

1. In the Unity Editor, go to Edit > Project Settings.
2. In the Project Settings window, select XR Plugin Management.
3. Under XR Plugin Management, choose OpenXR from the list of installed XR plugins.
4. Locate the MagicLeap 2 Support feature in the OpenXR settings.
5. Click the Gear icon next to MagicLeap 2 Support to access its settings.
6. Enable the Perception Snapshots option.

Perception Snapshots Setting in Unity

Using MLSDK Perception Pose Data

Once Perception Snapshots are enabled, APIs like MLCVCamera.GetPose and MLMarkerTracker will return pose data relative to the Unbounded Reference Space. To align this data with Unity's XR origin, set the tracking origin mode to Unbounded.

caution

The OpenXR Magic Leap 2 Reference Space feature must be enabled in your project’s OpenXR Settings (Window > XR Plugin Manager > OpenXR Settings).

The following script demonstrates how to set the tracking origin to Unbounded when the XRInputSystem is initialized:

using System.Collections;
using UnityEngine;
using UnityEngine.XR;
using UnityEngine.XR.Management;
using UnityEngine.XR.OpenXR;
using MagicLeap.OpenXR.Features;

public class ReferenceSpaceToggle : MonoBehaviour
{
    private XRInputSubsystem inputSubsystem;

    IEnumerator Start()
    {
        var referenceSpaceFeature = OpenXRSettings.Instance.GetFeature<MagicLeapReferenceSpacesFeature>();
        if (!referenceSpaceFeature.enabled)
        {
            Debug.LogError("Unbounded Tracking Space cannot be set if the OpenXR Magic Leap Reference Spaces Feature is not enabled. Stopping Script.");
            yield break;
        }

        yield return new WaitUntil(() => XRGeneralSettings.Instance != null &&
                                          XRGeneralSettings.Instance.Manager != null &&
                                          XRGeneralSettings.Instance.Manager.activeLoader != null &&
                                          XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>() != null);

        inputSubsystem = XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>();
        // Set the tracking origin to Unbounded
        SetSpace(TrackingOriginModeFlags.Unbounded);
    }

    private void SetSpace(TrackingOriginModeFlags flag)
    {
        if (inputSubsystem.TrySetTrackingOriginMode(flag))
        {
            Debug.Log($"Current Space: {inputSubsystem.GetTrackingOriginMode()}");
            inputSubsystem.TryRecenter();
        }
        else
        {
            Debug.LogError($"SetSpace failed to set Tracking Mode Origin to {flag}");
        }
    }
}

This guide is now more neutral and streamlined, ensuring that developers can efficiently enable support for legacy MLSDK APIs that require Perception Snapshots.

Supported MLSDK APIs

The following MLSDK APIs are compatible with the Magic Leap OpenXR Unity implementation and can be used in your Unity project:

• MLCamera: This API allows you to access the camera stream and capture images and videos. You can use the MLCamera class to control the camera settings and callbacks. For more information, see the MLCamera API documentation.
• MLOcclusion: This API allows you to occlude virtual objects behind real-world surfaces. You can use the MLOcclusion class to enable and disable occlusion. For more information, see the MLOcclusion API documentation.
• Depth Camera: This API allows you to access the depth data from the device's depth sensor. You can use the MLDepthCamera class to get the depth map and point cloud. For more information, see the Depth Camera API documentation.
• World Camera: This API allows you to access the world camera stream. You can use the MLWorldCamera class to get the world camera texture and pose. For more information, see the World Camera API documentation.
• MLMarker Tracking: allows you to detect two-dimensional icons from a marker dataset and then continuously track the targets' locations and orientations as you or the markers move through the environment. For more information, see the Marker Tracking API documentation.
• WebView: This API allows you to display web content in your app. You can use the MLWebView class to create and manage web views. For more information, see the WebView API documentation.
• Power Manager: This API allows you to monitor and manage the controller's power state. You can use the MLPowerManager class to get the battery level and state of the Magic Leap controller. For more information, see the Power Manager API documentation.
• MLAudio: This API allows you to play and record audio in your app. You can use the MLAudio class to manage the audio input devices. For more information, see the MLAudio API documentation.
• Soundfield Audio: This API allows you to play spatialized audio in your Unity Application. For more information, see the Soundfield Audio API documentation.
• Voice Intents: This API allows you to use voice commands to trigger actions in your app. You can use the MLVoice class to register and handle voice intents. For more information, see the Voice Intents API documentation.

Unchanged APIs

The following MLSDK APIs are unchanged and do not require any special handling when using the Magic Leap OpenXR Unity SDK:

• Permissions: This API allows you to request and check the permissions required by your app. You can use the MLPermissions class to manage Magic Leap Specific permissions. For more information, see the Permissions API documentation.
• Intents: This is an AOSP API that allows you to launch other apps or services from your app. For more information, see the Android Intents documentation.
• Sensors: Using the Standard Android Sensor API developers can access the device's sensors, such as accelerometer, gyroscope, magnetometer, and pressure sensor.For more information, see the Sensors API documentation.