Marker Understanding API Overview

This section provides an overview and API references for the the Magic Leap 2 Marker Understanding OpenXR Feature.

A marker detector can only handle single type of marker, specified by a value in MarkerDetectorSettings.MarkerType. To detect more than one marker type, you may create multiple MarkerDetector instances. However, note that you cannot detect multiple markers of the same type, such as both April and Aruco markers.

NameSpace

using MagicLeap.OpenXR.Features.MarkerUnderstanding;

caution

This feature requires the MARKER_TRACKING permission to be enabled in your project's Manifest Settings. (Edit > Project Settings > Magic Leap > Manifest Settings)

caution

This feature requires the Magic Leap 2 Marker Understanding OpenXR Feature to be enabled in your project's OpenXR Settings (Window > XR Plugin Manager > OpenXR Settings).

Marker Detector Settings

Marker Detector Settings tell the marker tracker what type of marker it should track and the Detector Profile, which includes information such as resolution, camera, etc., it should use to perform the detection. In this example, we create a Marker Detector that uses the Default Detector Profile and configure it to detect April Tags.

...

      // Create the Marker Detector Settings
      MarkerDetectorSettings detectorSettings =
          new MarkerDetectorSettings();

      // Set the detector profile to use when running the detection
      detectorSettings.MarkerDetectorProfile = MarkerDetectorProfile.Default;

      // Configure the detector settings
      detectorSettings.AprilTagSettings.AprilTagType = AprilTagType.Dictionary_36H11;
      detectorSettings.AprilTagSettings.AprilTagLength  = 0.115f;

      // Specify the target marker type to detect
      detectorSettings.MarkerType = MarkerType.AprilTag;

      // Get the OpenXR Marker Understanding Feature
      MagicLeapMarkerUnderstandingFeature markerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapMarkerUnderstandingFeature>();
      // Create the Marker Detector with the settings above
      MarkerDetector newMarkerDetector = markerFeature.CreateMarkerDetector(detectorSettings);

      if (newMarkerDetector != null)
      {
          // Save the marker detector and poll the data in the Update loop
      }

...

Marker Size Estimation

The Magic Leap 2 Marker Understanding feature can estimate the size of Aruco, April, and QR Code markers. Note that higher localization accuracy may be obtained by specifying the marker size. This feature can be enabled by setting ArucoSettings.EstimateArucoLength , AprilTagSettings.EstimateArucoLength or QRSettings.EstimateQRLength to true.

  // Specify the marker type to detect
  detectorSettings.QRSettings.EstimateArucoLength = true;
  detectorSettings.AprilTagSettings.EstimateArucoLength = true;
  detectorSettings.ArucoSettings.EstimateArucoLength = true;

Marker Tracker Profile

Marker Tracker Profiles allow developers to choose pre-configured or create custom marker tracking settings based on their use case. The pre-configured profiles are as follows:

• Default
  • Tracker profile that covers standard use cases.
• Speed
• Use this profile to reduce the compute load and increase detection/tracker speed. This can result poor poses.
• Accuracy
• Use this profile to optimize for accurate marker poses. This can cause increased load on the compute.
• SmallTargets
• Use this profile to optimize for markers that are small or for larger markers that need to detected from far.
• Large_FOV
• Use this profile to be able to detect markers across a larger Field Of View. Marker Tracker system will attempt to use multiple cameras to detect the markers.
• Custom
• Application can define a custom tracker profiler.

Developers can specify a variety of options when creating custom profiles. The example below shows how to create Marker Detector Settings using both custom and predefined profiles.

        // Create the Marker Detector Settings
        MarkerDetectorSettings detectorSettings =
            new MarkerDetectorSettings();

        // Specify the detector settings and the marker type to detect
        detectorSettings.AprilTagSettings.AprilTagType = AprilTagType.Dictionary_36H11;
        detectorSettings.ArucoSettings.EstimateArucoLength = true;
        detectorSettings.MarkerType = MarkerType.AprilTag;



        // Set the detector profile to use when running the detection
        detectorSettings.MarkerDetectorProfile = MarkerDetectorProfile.Custom;

        //Create the custom profile
        CustomProfileSettings customProfileSettings =
            new CustomProfileSettings();

        customProfileSettings.AnalysisInterval = MarkerDetectorFullAnalysisInterval.Medium;
        customProfileSettings.CameraHint = MarkerDetectorCamera.RGB;
        customProfileSettings.CornerRefinement = MarkerDetectorCornerRefineMethod.AprilTag;
        customProfileSettings.ResolutionHint = MarkerDetectorResolution.Medium;
        customProfileSettings.FPSHint = MarkerDetectorFPS.Medium;
        customProfileSettings.UseEdgeRefinement = false;

        detectorSettings.CustomProfileSettings = customProfileSettings;

Modify Marker Detector Settings

To update the Marker Detector Settings after creation, you can either destroy the detector and recreate it, or use the ModifyMarkerDetector function.

The example below demonstrates using ModifyMarkerDetector to update marker settings, ensuring the Marker Detector maintains its position in the MarkerFeature.MarkerDetectors list.

  //Get the OpenXR Marker Understanding Feature
   MagicLeapMarkerUnderstandingFeature markerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapMarkerUnderstandingFeature>();

   if (markerFeature.MarkerDetectors.Count > 0)
   {
       var markerDetector = markerFeature.MarkerDetectors[0];
       //Update the settings of a marker detector
       markerFeature.ModifyMarkerDetector(detectorSettings, ref markerDetector);
   }

Accessing the Marker Detector

After a marker detector is created, it is added to the feature instance's MarkerDetectors list. This list is readonly, allowing values to be obtained but not modified directly. Markers in the MarkerDetectors list are ordered according to when they were created.

  MagicLeapMarkerUnderstandingFeature markerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapMarkerUnderstandingFeature>();

  firstMarkerDetector = markerFeature.MarkerDetectors[0];

Updating Marker Detector Data

To update marker tracker data, simply call the UpdateMarkerDetectors method. The data for all the marker detectors in the MarkerDetectors list are updated simultaneously when this method is called.

  MagicLeapMarkerUnderstandingFeature markerFeature;

  void Start(){
    markerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapMarkerUnderstandingFeature>();
  }
  void Update()
  {
    markerFeature.UpdateMarkerDetectors();
  }

Destroying a Marker Detector

To destroy a tracker, call the DestroyMarkerDetector method and pass the desired marker detector as an argument. After a marker detector is destroyed, the index values of the marker detectors in the MarkerDetectors list will update accordingly.

  void OnDisable()
  {
    //Get the OpenXR Marker Understanding Feature
    MagicLeapMarkerUnderstandingFeature markerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapMarkerUnderstandingFeature>();

    if (markerFeature.MarkerDetectors.Count > 0)
    {
       var firstMarkerDetector = markerFeature.MarkerDetectors[0];
       //Update the settings of a marker detector
       markerFeature.DestroyMarkerDetector(firstMarkerDetector);
    }
  }

Destroying All Marker Detector

To destroy all trackers, call the DestroyAllMarkerDetectors method. Doing so will clear the feature's MarkerDetectors list.

void OnDestroy()
{
    //Get the OpenXR Marker Understanding Feature
    MagicLeapMarkerUnderstandingFeature markerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapMarkerUnderstandingFeature>();

    markerFeature.DestroyAllMarkerDetectors();
}

Query Marker Detector Data

The data from a marker detector can be queried directly, typically right after updating the detector. Before obtaining data, it is advisable to check the current status of the marker tracker and proceed only if it is of type MarkerDetectorStatus.Ready.

...
  void Update()
  {
    //Update the marker detector
    markerFeature.UpdateMarkerDetectors();

    // Iterate through all of the marker detectors
    for (int i = 0; i < markerFeature.MarkerDetectors.Count; i++)
    {
        //Verify that the marker detector is running
        if(markerFeature.MarkerDetectors[i].Status == MarkerDetectorStatus.Ready)
        {
            //Cycle through the detector's data and log it to the debug log
            var currentDetector = markerFeature.MarkerDetectors[i];
            for (int j = 0; j < currentDetector.Data.Count; j++)
            {
                //If the marker has a valid position log the value
                if (currentDetector.Data[j].MarkerPose.HasValue)
                {
                    Debug.Log(
                        $"Marker Value {currentDetector.Data[j].MarkerNumber}," +
                        $"Marker Pose {currentDetector.Data[j].MarkerPose}"
                    );
                }
                else
                {
                    Debug.Log(
                        $"Marker Value {currentDetector.Data[j].MarkerNumber}," +
                        $"Marker Pose Not Valid"
                    );
                }
            }
        }
    }
  }
...

Marker Data Pose

Developers should note that the Marker Marker Pose is in world coordinates. It is common practice to use Unity's XR Origin component as the reference to transform Marker Pose so it can share the same origin as the device's tracked pose drivers. The Marker Understanding Feature returns the marker pose as a nullable struct. Which means developers can check if the pose is valid using markerData.MarkerPose.HasValue and markerData.MarkerPose.Value.position/markerData.MarkerPose.Value.rotation.

// NameSpace for the XROrigin
using Unity.XR.CoreUtils;
...

// The XR Origin Component that is part of Unity's XR Rig 
public XROrigin XROrigin;

private void OnValidate()
{
    //Automatically find the XROrigin component if it's present in the scene
    if (XROrigin == null)
    {
        XROrigin = FindAnyObjectByType<XROrigin>();
    }
}

private void SetMarkerObjectPosition(GameObject marker, Pose markerPose, float markerSize)
{
    Transform originTransform = XROrigin.CameraFloorOffsetObject.transform;

    // Set the position of the marker. Since the pose is given relative to the XR Origin,
    // we need to transform it to world coordinates.
    marker.transform.position = originTransform.TransformPoint(markerPose.position);
    marker.transform.localRotation = originTransform.rotation * markerPose.rotation;

    // When marker size estimation is enabled, markers may take a few frames to scale to their appropriate size.
    if (marker.transform.localScale.x != markerSize)
    {
        marker.transform.localScale = new Vector3(markerSize, markerSize, markerSize);
    }
}

Reading String Data

String data encoded into a QR code or barcode marker can be read directly from the MarkerData struct. The following example shows a function that takes marker data and its type, then logs its values into the console.

public void LogMarkerData(MarkerData markerData, MarkerType currentMarkerType)
  {
      StringBuilder stringBuilder = new StringBuilder();
      stringBuilder.Clear();

      stringBuilder.Append($"MarkerLength: {markerData.MarkerLength}\n\n");

      switch (currentMarkerType)
      {
          case MarkerType.QR:
          case MarkerType.Code128:
              stringBuilder.Append($"MarkerString: {markerData.MarkerString}\n\n");
              break;
          default:
              stringBuilder.Append($"MarkerNumber: {markerData.MarkerNumber}\n\n");
              stringBuilder.Append($"ReprojectionErrorMeters: {markerData.ReprojectionErrorMeters}\n\n");
              break;
      }

      switch (currentMarkerType)
      {
          case MarkerType.Aruco:
          case MarkerType.QR:
          case MarkerType.AprilTag:
              if (markerData.MarkerPose.HasValue)
              {
                  stringBuilder.Append($"Position: {markerData.MarkerPose.Value.position}\n\n");
                  stringBuilder.Append($"Rotation: {markerData.MarkerPose.Value.rotation}");
              }
              break;
      }

      Debug.Log(stringBuilder.ToString());
  }

Additional Notes

• Tracker settings cannot be updated directly after being created. To change any details about the tracker, first destroy the old tracker and create a new one with the new settings.

• Some settings will only be applicable depending on the selected MarkerType:

  • For example, Aruco, AprilTag, and QR each have their categories of data.
  • The ArucoLength and QRLength variables will not be applied if the EstimateArucoLength or EstimateQRLength setting is chosen; the tracker will attempt to estimate these automatically.

• The CustomProfileSettings only apply if the MarkerDetectorProfile is set to the Custom type.

• Although the MarkerData type contains 5 variables, only certain ones are applicable depending on the selected MarkerType:

  • If a variable is not applicable to the marker type, it will return a default value.
  • Refer to the MagicLeapMarkerUnderstandingData file for more information about which variables apply to the different marker types.