MLCamera

The Magic Leap 2 MLCamera API allows developers to capture real and virtual content inside their applications. While the Magic Leap 2 has only one camera for capturing content, two separate streams can be access from the camera at the same time. This allows you to create applications that can stream the user's point of view, while using the camera to perform computer vision tasks.

tip

The Magic Leap Camera provides the same functionality as the Android Camera2 APIs and can be used when Android Camera 2 functionality is not provided by the build tools. For example, when building with Unity3D.

Camera Streams

Magic Leap 2 allows developers to access two streams from the same physical camera. The camera streams are accessed as devices in the Unity API and have the following identifiers:

0. Main Camera - provides access to compressed video and still images. This device allows you to capture virtual, real-world, mixed reality content and is the performed choice if you are not performing computer vision tasks on the output or if it is being used for streaming, broadcasting, or images.

1. CV Camera - best used for Computer vision scenarios, uncompressed, raw frames. If you use this device to do CV in you application, you can use the record/stream gameplay using the Capture Service, but you cannot perform Image or marker tracking using the SDK.

You can use the same stream to capture different types of content. For example, you can capture Video, Images, and a Preview from the Main Camera stream, while also using the CV Camera at the same time.

Mixed Reality Capture

The Main Camera stream is the only stream that can capture Mixed Reality content. However, the Mixed Reality Capture stream only supports a single concurrent capture mode which means that you need to choose whether to capture Video OR Images but not both.

API

The MLCamera is part of the MLSDK and can be used along side the OpenXR workflow in Unity. Before using the API you will need to Enable Perception Snapshots in your project.

Enabling Perception Snapshots

Follow these steps to enable MLSDK Perception support when using OpenXR:

1. In the Unity Editor, go to Edit > Project Settings.
2. In the Project Settings window, select XR Plugin Management.
3. Under XR Plugin Management, choose OpenXR from the list of installed XR plugins.
4. Locate the MagicLeap 2 Support feature in the OpenXR settings.
5. Click the Gear icon next to MagicLeap 2 Support to access its settings.
6. Enable the Perception Snapshots option.

Perception Snapshots Setting in Unity

Using MLSDK Perception Pose Data

Once Perception Snapshots are enabled, APIs like MLCVCamera.GetPose and MLMarkerTracker will return pose data relative to the Unbounded Reference Space. To align this data with Unity's XR origin, set the tracking origin mode to Unbounded.

caution

The OpenXR Magic Leap 2 Reference Space feature must be enabled in your project’s OpenXR Settings (Window > XR Plugin Manager > OpenXR Settings).

The following script demonstrates how to set the tracking origin to Unbounded when the XRInputSystem is initialized:

using System.Collections;
using UnityEngine;
using UnityEngine.XR;
using UnityEngine.XR.Management;
using UnityEngine.XR.OpenXR;
using MagicLeap.OpenXR.Features;

public class ReferenceSpaceToggle : MonoBehaviour
{
    private XRInputSubsystem inputSubsystem;

    IEnumerator Start()
    {
        var referenceSpaceFeature = OpenXRSettings.Instance.GetFeature<MagicLeapReferenceSpacesFeature>();
        if (!referenceSpaceFeature.enabled)
        {
            Debug.LogError("Unbounded Tracking Space cannot be set if the OpenXR Magic Leap Reference Spaces Feature is not enabled. Stopping Script.");
            yield break;
        }

        yield return new WaitUntil(() => XRGeneralSettings.Instance != null &&
                                          XRGeneralSettings.Instance.Manager != null &&
                                          XRGeneralSettings.Instance.Manager.activeLoader != null &&
                                          XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>() != null);

        inputSubsystem = XRGeneralSettings.Instance.Manager.activeLoader.GetLoadedSubsystem<XRInputSubsystem>();
        // Set the tracking origin to Unbounded
        SetSpace(TrackingOriginModeFlags.Unbounded);
    }

    private void SetSpace(TrackingOriginModeFlags flag)
    {
        if (inputSubsystem.TrySetTrackingOriginMode(flag))
        {
            Debug.Log($"Current Space: {inputSubsystem.GetTrackingOriginMode()}");
            inputSubsystem.TryRecenter();
        }
        else
        {
            Debug.LogError($"SetSpace failed to set Tracking Mode Origin to {flag}");
        }
    }
}

Using the API

To learn about the Unity API see the Magic Leap Camera Unity Documentation.

See Magic Leap Unity MLCamera