Skip to main content
Version: 14 Oct 2024

Physical Occlusion

Overview​

The Magic Leap Physical Occlusion Feature in Unity allows developers to manage and configure how virtual objects interact with real-world occluders. This feature enhances the realism of Augmented Reality experiences by allowing virtual objects to be hidden or partially blocked by real-world elements detected through various sources such as the environment, depth sensors, hands, and controllers.

Namespace
using MagicLeap.OpenXR.Features.PhysicalOcclusion;
Experimental API

This API is experimental and is subject to change or removal without prior notice. It may be unstable and is not recommended for use in production environments.

Known Limitation​

UI and Canvas Elements May Not Occlude Properly: Environment and Depth Sensor Occlusions do not interact as expected with Unity's UI and Canvas components. This is due to these UI elements not contributing data to the depth buffer, potentially leading to them appearing occluded or partially occluded even when the canvas is positioned in front of an occlusion source.

caution

Semi-transparent materials, such as Unity UI elements may not work properly since their shaders often skip the depth buffer. To enable depth submission for UI elements, add the ZWrite flag to the top of the Pass block definition in the shader.

Modifying the Default UI Material Shader to Support Depth Submission

Step 1: Create a Custom Shader

  1. In Unity, navigate to Assets > Create > Shader > Unlit Shader.
  2. Name the new shader UIDepthWriteShader.
  3. Open the newly created shader file.
  4. Replace the shader code with the code from Unity's default Sprite Shader as a starting point. You can find Unity’s default shaders in the Unity Shader Archive.
    • Select the Unity version you're using.
    • Go to See all downloads > Other Installs > Shaders.
    • Download the shader package, locate the Default-Sprite.shader, and copy its contents into your new shader.

Step 2: Modify the Shader for Depth Submission

  1. Inside the shader file, locate the Pass block (typically around lines 40-50).

  2. Add the following lines to ensure the shader writes to the depth buffer and always passes the depth test:

    ZWrite On
    ZTest Always
    • ZWrite On: Enables depth writing, ensuring the shader submits depth information to the buffer.
    • ZTest Always: Ensures that the object is always drawn, even if there is other content in the scene. This is useful for UI elements that must appear regardless of scene depth.

  3. Save the shader file.

Step 3: Create a Material Using the Custom Shader

  1. In Unity, navigate to Assets > Create > Material.
  2. Name the material UIDepthMaterial.
  3. In the Inspector, set the Shader for this material to UIDepthWriteShader (the shader you created in Step 1).

Step 4: Apply the Material to UI Elements

  1. Select the UI element(s) you want to modify in your scene.
  2. In the Inspector panel, locate the CanvasRenderer or Image component.
  3. Under the Material property, assign the UIDepthMaterial you created in Step 3.

Step 5: Verify Depth Submission

  1. Enter Play Mode and observe how the UI behaves relative to 3D objects in your scene.
  2. Ensure that the UI elements are now occluded by other objects when appropriate. If the UI elements are still drawn in front of other objects, review the shader settings to confirm the depth submission flags are applied correctly.

Occlusion Sources​

The Magic Leap Physical Occlusion Feature supports several types of occlusion sources:

  1. Environment : Occlusion based on static elements of the real-world environment. Creates an occlusion mesh using the depth sensor.
  2. DepthSensor: The depth sensor can occlude objects within a specified range. If the range is set between 0 to 0.9 meter, occlusion operates at 30 fps. Beyond 0.9 meter, occlusion operates at 5 fps.
    • When the depth sensor is configured to occlude within a specific range, environment occlusion / spatial mesh will also be limited to this range.
  3. Hands: Hand-based occlusion operates independent of the depth sensor.
  4. Controller: Controller-based occlusion operates independent of the depth sensor.
caution

Both the DepthSensor and Environment occlusion sources rely on the depth sensor. Because environment occlusion uses the meshing system and relies on the long-range mode, if both depth and environment occlusion are enabled, then the far depth range is set to a minimum of 0.91 meters so that environment occlusion can function. Objects beyond the depth sensor's maximum range will not generate a spatial mesh.

Prerequisites​

  1. The Magic Leap Physical Occlusion OpenXR Feature must be enabled in your project's OpenXR Settings.
    1. Go to Edit > Project Settings
    2. Select OpenXR under XR Plug-in Management from the left-hand menu.
    3. Under Features, scroll down and enable the Magic Leap 2 Physical Occlusion feature.
  2. Enable Depth Submission
    1. At the top of the OpenXR settings, set the Depth Submission Mode to 16 Bit or higher.

Example Usage​

Below is a simplified example demonstrating how to use the Physical Occlusion feature in a Unity application. To use the example, Create a new script with the example script logic, then attach the componenet to a GameObject in the scene and adjust the parameters inside the inspector. The Occlusion Feature will initialize with the specified values as soon as the application is starts.

using MagicLeap.OpenXR.Features.PhysicalOcclusion;
using UnityEngine;
using UnityEngine.XR.OpenXR;

public class OcclusionTest : MonoBehaviour
{
    [SerializeField]
    [Tooltip("The sources to use for the Physical Occlusion Feature.")]
    private MagicLeapPhysicalOcclusionFeature.OcclusionSource  _occlusionSource = 
                        MagicLeapPhysicalOcclusionFeature.OcclusionSource.Controller |
                        MagicLeapPhysicalOcclusionFeature.OcclusionSource.DepthSensor |
                        MagicLeapPhysicalOcclusionFeature.OcclusionSource.Hands;
    
    [Tooltip("Depth sensor operates at 30fps when maximum range is 0.9 metters or lower. (min 0.3, max 7.5)")]
    [SerializeField]
    private float _nearRange = 0.3f;
    
    [Tooltip("Depth sensor operates at 30fps when maximum range is 0.9 metters or lower. (min 0.3, max 7.5)")]
    [SerializeField]
    private float _farRange = 0.9f;

    private MagicLeapPhysicalOcclusionFeature occlusionFeature;

    void Start()
    {
        // Initialize the Physical Occlusion feature
        occlusionFeature = OpenXRSettings.Instance.GetFeature<MagicLeapPhysicalOcclusionFeature>();

        if (occlusionFeature == null || !occlusionFeature.enabled)
        {
            Debug.LogError("Physical Occlusion feature must be enabled.");
            enabled = false;
            return;
        }

        // Enable occlusion and configure sources
        occlusionFeature.EnableOcclusion = true;
        occlusionFeature.EnabledOcclusionSource = _occlusionSource;

        // Configure depth sensor properties to the range specified in the inspector,
        // This will limit environment occlusion to this range as well
        occlusionFeature.DepthSensorNearRange = _nearRange;
        occlusionFeature.DepthSensorFarRange = _farRange;
    }

    void OnDestroy()
    {
        if (occlusionFeature != null && occlusionFeature.enabled)
        {
            // Disable occlusion when the script is destroyed
            occlusionFeature.EnableOcclusion = false;
            occlusionFeature.EnabledOcclusionSource = 0;
        }
    }
}
Retrieve the Minimum and Maximum ranges for the Depth Sensor at runtime.
You can retrieve the depth sensor's near and far range properties if you wish to display the limits inside your application. However, the feature clamps the range values automatically when setting the occlusionFeature.DepthSensorNearRange and occlusionFeature.DepthSensorFarRange properties.
Example
  (float minNear, float maxNear) = occlusionFeature.GetDepthSensorProperties().nearRange;
  (float minFar, float maxFar) = occlusionFeature.GetDepthSensorProperties().farRange;