Physical Occlusion

Overview

The Magic Leap Physical Occlusion Feature in Unity allows developers to manage and configure how virtual objects interact with real-world occluders. This feature enhances the realism of Augmented Reality experiences by allowing virtual objects to be hidden or partially blocked by real-world elements detected through various sources such as the environment, depth sensors, hands, and controllers.

Namespace

using MagicLeap.OpenXR.Features.PhysicalOcclusion;

Experimental API

This API is experimental and is subject to change or removal without prior notice. It may be unstable and is not recommended for use in production environments.

Known Issues

UI and Canvas Elements May Not Occlude Properly: Environment and Depth Sensor Occlusions do not interact as expected with Unity's UI and Canvas components. This is due to these UI elements not contributing data to the depth buffer, potentially leading to them appearing occluded or partially occluded even when the canvas is positioned in front of an occlusion source.

Occlusion Sources

The Magic Leap Physical Occlusion Feature supports several types of occlusion sources:

1. Environment : Occlusion based on static elements of the real-world environment. Creates an occlusion mesh using the depth sensor.
2. DepthSensor: The depth sensor can occlude objects within a specified range. If the range is set between 0 to 0.9 meter, occlusion operates at 30 fps. Beyond 0.9 meter, occlusion operates at 5 fps.
   • When the depth sensor is configured to occlude within a specific range, environment occlusion / spatial mesh will also be limited to this range.
3. Hands: Hand-based occlusion operates independent of the depth sensor.
4. Controller: Controller-based occlusion operates independent of the depth sensor.

caution

Both the DepthSensor and Environment occlusion sources rely on the depth sensor. Because environment occlusion uses the meshing system and relies on the long-range mode, if both depth and environment occlusion are enabled, then the far depth range is set to a minimum of 0.91 meters so that environment occlusion can function. Objects beyond the depth sensor's maximum range will not generate a spatial mesh.

Prerequisites

1. The Magic Leap Physical Occlusion OpenXR Feature must be enabled in your project's OpenXR Settings.
   1. Go to Edit > Project Settings
   2. Select OpenXR under XR Plug-in Management from the left-hand menu.
   3. Under Features, scroll down and enable the Magic Leap 2 Physical Occlusion feature.
2. Enable Depth Submission
   1. At the top of the OpenXR settings, set the Depth Submission Mode to 16 Bit or higher.

Example Usage

Below is a simplified example demonstrating how to use the Physical Occlusion feature in a Unity application. To use the example, Create a new script with the example script logic, then attach the componenet to a GameObject in the scene and adjust the parameters inside the inspector. The Occlusion Feature will initialize with the specified values as soon as the application is starts.

using MagicLeap.OpenXR.Features.PhysicalOcclusion;
using UnityEngine;
using UnityEngine.XR.OpenXR;

public class OcclusionTest : MonoBehaviour
{
    [SerializeField]
    [Tooltip("The sources to use for the Physical Occlusion Feature.")]
    private MagicLeapPhysicalOcclusionFeature.OcclusionSource  _occlusionSource = 
                        MagicLeapPhysicalOcclusionFeature.OcclusionSource.Controller |
                        MagicLeapPhysicalOcclusionFeature.OcclusionSource.DepthSensor |
                        MagicLeapPhysicalOcclusionFeature.OcclusionSource.Hands;
    
    [Tooltip("Depth sensor operates at 30fps when maximum range is 0.9 metters or lower. (min 0.3, max 7.5)")]
    [SerializeField]
    private float _nearRange = 0.3f;
    
    [Tooltip("Depth sensor operates at 30fps when maximum range is 0.9 metters or lower. (min 0.3, max 7.5)")]
    [SerializeField]
    private float _farRange = 0.9f;

    private MagicLeapPhysicalOcclusionFeature occlusionFeature;

    void Start()
    {
        // Initialize the Physical Occlusion feature
        occlusionFeature = OpenXRSettings.Instance.GetFeature<MagicLeapPhysicalOcclusionFeature>();

        if (occlusionFeature == null || !occlusionFeature.enabled)
        {
            Debug.LogError("Physical Occlusion feature must be enabled.");
            enabled = false;
            return;
        }

        // Enable occlusion and configure sources
        occlusionFeature.EnableOcclusion = true;
        occlusionFeature.EnabledOcclusionSource = _occlusionSource;

        // Configure depth sensor properties to the range specified in the inspector,
        // This will limit environment occlusion to this range as well
        occlusionFeature.DepthSensorNearRange = _nearRange;
        occlusionFeature.DepthSensorFarRange = _farRange;
    }

    void OnDestroy()
    {
        if (occlusionFeature != null && occlusionFeature.enabled)
        {
            // Disable occlusion when the script is destroyed
            occlusionFeature.EnableOcclusion = false;
            occlusionFeature.EnabledOcclusionSource = 0;
        }
    }
}

Retrieve the Minimum and Maximum ranges for the Depth Sensor at runtime.

You can retrieve the depth sensor's near and far range properties if you wish to display the limits inside your application. However, the feature clamps the range values automatically when setting the occlusionFeature.DepthSensorNearRange and occlusionFeature.DepthSensorFarRange properties.

Example

  (float minNear, float maxNear) = occlusionFeature.GetDepthSensorProperties().nearRange;
  (float minFar, float maxFar) = occlusionFeature.GetDepthSensorProperties().farRange;