Skip to main content
Version: 10 Jul 2024

Dynamic Occlusion

The XR_ML_physical_world_occlusion extension allows applications to specify occlusion settings on a per-layer basis in OpenXR. This extension provides the ability to occlude virtual content with real-world objects detected by different sources, improving the sense of presence and realism.

note

As of OS release 1.7.0, Dynamic Occlusion is still an experimental extension not included in the official OpenXR registry and is subject to change.

User Experience​

If enabled, the selected sources will be automatically occlude virtual objects behind them from the user's view. This also applies to the segmented dimmer visibility.

For example, if an application has controller occlusion enabled and the controller is held in front of a virtual object, both the virtual object and the segmented dimmer around it will not be visible to the user in the areas occluded by the controller.

>Hand Occlusion Example
>Depth Occlusion Example

Occlusion Sources​

There are 4 types of occlusion available to developers: Controller, Hands, Depth and Environment. Note that the Depth and Environment occlusion sources both rely on the depth sensor. Click the OpenXR specs below to view the full extension definition:

  • Controller :
    • XR_TYPE_PHYSICAL_WORLD_OCCLUSION_SOURCE_CONTROLLER_ML
  • Hands :
    • XR_TYPE_PHYSICAL_WORLD_OCCLUSION_SOURCE_HANDS_ML
  • Environment :
    • XR_TYPE_PHYSICAL_WORLD_OCCLUSION_SOURCE_ENVIRONMENT_ML
  • Depth :
    • XR_TYPE_PHYSICAL_WORLD_OCCLUSION_SOURCE_DEPTH_SENSOR_ML

Limitations​

  • Opting for this feature will cause apps to have less GPU time per frame compare to not opting in and apps should budget in the additional cost.

  • Both the depth and environment occlusion sources rely on the depth sensor. Because environment occlusion uses the meshing system and relies on the long range mode, if both depth and environment occlusion are enabled then the far depth range is set to a minimum of 0.91 so that environment occlusion can function.

  • Switching between sources or changing depth ranges can cause very noticeable delays, depending on the source:

    • Controller -- no delay
    • Hand tracking -- medium delay
    • Depth sensor -- large delay
    • Environment -- large delay

  • Currently, enabling the first occlusion source can cause an additional delay, due to the occlusion feature starting up, regardless of occlusion source.

  • While there is no direct frame rate dropoff at the 0.91 depth cutoff, any increase in depth range will increase the amount of geometry being rendered and have some performance impact.

Best Practices​

  • Developers can request a specific range for the depth sensor when using depth occlusion. A far range of \<= 0.9 means the short-range 30 Hz depth sensor mode is active, while a value above 0.9 means the long range 5 Hz mode is active.

  • Developers who wish to use the depth sensor source at 30 Hz should use this configuration:

    • All layers that use depth sensor occlusion should have a far range of less than or equal to 0.9m
    • No layers should use environment occlusion

  • It is recommended to use depth occlusion for any moving real-world objects, such as handheld objects, pets and other people. Use environmental occlusion for any other static surroundings.