MLCamera

The Magic Leap 2 MLCamera API allows developers to capture both real and virtual content within their applications. Although the Magic Leap 2 has only one physical camera for capturing content, two separate streams can be accessed from the camera simultaneously. This enables applications to stream the user's point of view while also performing computer vision tasks using the same camera.

tip

See the Unity MLSDK MLCamera Guide for complete details on how to use the ML Camera APIs in your project.

info

The Magic Leap Camera offers similar functionality to the Android Camera2 APIs and can be used when Android Camera2 functionality is not provided by the build tools—for example, when building with Unity.

Using MLCamera APIs in OpenXR

The MLCamera is part of the MLSDK and can be used alongside the OpenXR workflow in Unity. Before using the API, you will need to enable Perception Snapshots in your project and set the tracking space to Unbounded.

Enabling Perception Snapshots

Follow these steps to enable MLSDK Perception support when using OpenXR:

1. In the Unity Editor, go to Edit > Project Settings.
2. In the Project Settings window, select XR Plugin Management.
3. Under XR Plugin Management, choose OpenXR from the list of installed XR plugins.
4. Locate the MagicLeap 2 Support feature in the OpenXR settings.
5. Click the Gear icon next to MagicLeap 2 Support to access its settings.
6. Enable the Perception Snapshots option.

Perception Snapshots Setting in Unity

Using MLSDK Perception Pose Data

Once Perception Snapshots are enabled, APIs like MLCVCamera.GetPose and MLMarkerTracker will return pose data relative to the Unbounded Reference Space. To align this data with Unity's XR origin, set the tracking origin mode to Unbounded.

caution

The OpenXR Magic Leap 2 Reference Space feature must be enabled in your project’s OpenXR Settings (Window > XR Plugin Manager > OpenXR Settings).

The following script demonstrates how to set the tracking origin to Unbounded when the XRInputSystem is initialized:

using System.Collections;
using UnityEngine;
using UnityEngine.XR;
using UnityEngine.XR.Management;
using UnityEngine.XR.OpenXR;
using MagicLeap.OpenXR.Features;

public class ReferenceSpaceToggle : MonoBehaviour
{
    private XRInputSubsystem inputSubsystem;

    IEnumerator Start()
    {
        var referenceSpaceFeature = OpenXRSettings.Instance.GetFeature<MagicLeapReferenceSpacesFeature>();
        if (!referenceSpaceFeature.enabled)
        {
            Debug.LogError("Unbounded Tracking Space cannot be set if the OpenXR Magic Leap Reference Spaces Feature is not enabled. Stopping Script.");
            yield break;
        }

        yield return new WaitUntil(() => XRGeneralSettings.Instance != null &&
                                          XRGeneralSettings.Instance.Manager != null &&
                                          XRGeneralSettings.Instance.Manager.activeLoader != null &&
                                          XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>() != null);

        inputSubsystem = XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>();
        // Set the tracking origin to Unbounded
        SetSpace(TrackingOriginModeFlags.Unbounded);
    }

    private void SetSpace(TrackingOriginModeFlags flag)
    {
        if (inputSubsystem.TrySetTrackingOriginMode(flag))
        {
            Debug.Log($"Current Space: {inputSubsystem.GetTrackingOriginMode()}");
            inputSubsystem.TryRecenter();
        }
        else
        {
            Debug.LogError($"SetSpace failed to set Tracking Mode Origin to {flag}");
        }
    }
}

Using the API

To learn about the Unity API see the Magic Leap Camera Unity Documentation.

See Magic Leap Unity MLCamera