Secondary View

The Magic Leap 2 Secondary View feature leverages the OpenXR XR_MSFT_first_person_observer extension to enhance Mixed Reality capture. By rendering a secondary view from the RGB camera's position, it ensures precise alignment of physical and virtual content. This feature addresses common issues in mixed reality (MR) capture, improving overall quality.

How the feature improves Mixed Reality Captures:

• Corrects offset alignment in MR captures involving hand tracking, controller tracking, or marker tracking applications.
• Eliminates cropping artifacts when recording mixed reality (MR) capture in 4:3 aspect ratios.
• Stabilizes MR capture regardless of focus distance for pixel registration

Left - Without Secondary View | Right - With Secondary View

Why Content Appears Offset Without Secondary View

When the XR_MSFT_first_person_observer feature is not enabled, the virtual content captured might be misaligned with the physical environment. This is because the default Mixed Reality capture is obtained from the left eye display, and needs to be warped to represent the RGB camera's position. As a result, objects at different depths can appear misaligned in the capture, leading to offset and reprojection artifacts.

Performance Impact

Enabling the XR_MSFT_first_person_observer extension requires the Magic Leap 2 to perform an additional render pass from the RGB camera's perspective. This can have a performance impact on the device, as it increases the rendering workload. When running an application and accessing the Mixed Reality Stream with these features enabled, the device needs to handle this extra rendering, which can affect overall performance.

Enabling the Feature in Unity

To use the Secondary View feature on Magic Leap 2, you need a Unity project with the Magic Leap SDK and the OpenXR package.

Enable Secondary View

1. Import Magic Leap SDK: Make sure your Unity project includes the Magic Leap SDK, which contains necessary libraries and tools.
2. Enable Magic Leap 2 Secondary View
   1. Go to Edit > Project Settings > XR Plug-in Management.
   2. Under the OpenXR tab, enable the Magic Leap 2 Secondary View option.

Checking MLXrSecondaryViewState

You can programmatically check the status of the Secondary View feature. This will be set to inactive until the application detects that a capture has started.

1. Enable the OpenXR Magic Leap 2 Secondary View setting.
2. Check the MLXrSecondaryViewState.IsActive flag and watch it change from inactive to active when you start a capture.

The following example script shows the user how to check the current status of the Secondary View feature. Attach the script to an object in your scene, assign a Text object in the inspector, and start a capture via the Magic Leap Hub or your preferred method of capturing content on the ML2 device.

using MagicLeap.OpenXR.Features;
using UnityEngine;
using UnityEngine.UI;

public class SecondaryViewStatus : MonoBehaviour
{
    private void Update()
    {
        Debug.Log($"Secondary View active = {MLXrSecondaryViewState.IsActive}");
    }
}

More Details

For lower-level technical specifications, refer to the OpenXR Specification for XR_MSFT_first_person_observer and XR_MSFT_secondary_view_configuration for more information about this feature.