Light Estimation API Overview

This section provides an overview and API references for the the Magic Leap 2 Light Estimation OpenXR Feature.

caution

This feature requires the CAMERA permission to be enabled in your project's Manifest Settings. (Edit > Project Settings > Magic Leap > Manifest Settings)

caution

This feature requires the Magic Leap 2 Light Estimation OpenXR Feature to be enabled in your project's OpenXR Settings (Window > XR Plugin Manager > OpenXR Settings).

caution

This feature uses the CV Camera and will prevent other features from working such as the Marker Understanding Feature.

Creating Light Estimation

Light Estimation is created as an instance and accepts a HDRCubemapFaceResolution as an argument. This represents the resolution of each face on our cubemap.

...
    void Start()
    {
        // Obtain the Light Estimation Feature.
        _lightEstimationFeature = OpenXRSettings.Instance.GetFeature<MagicLeapLightEstimationFeature>();

        // Create a variable that contains the resolution of each face on the cubemap and set it to 256x256.
        HDRCubemapFaceResolution cubemapFaceResolution
            = HDRCubemapFaceResolution.Resolution_256x256;

        // Create the light estimation.
        _lightEstimationFeature.CreateLightEstimation(cubemapFaceResolution);
    }
...

Obtaining Light Estimation Data

In order to obtain the data from the LightEstimation instance, you must first make sure that two different criteria are met.

• Firstly, you must ensure that our light estimation instance has been created. You can check this in your script by using the LightEstimationCreated member of your light estimation instance.
• Second, you must ensure that the light estimation data is ready to be queried. You can check this by using the CheckEstimationEstimateReadiness method of your light estimation instance

bool IsLightEstimationDataReadyToBeObtained()
{
    // In order to obtain light estimation data, you must first ensure that
    // the light estimation feature has been created and that our light
    // estimation data is ready to be obtained.
    return _lightEstimationFeature.LightEstimationCreated && _lightEstimationFeature.CheckEstimationEstimateReadiness();
}

The light estimate data is represented as a EstimateData data type. This contains information about directional light and environment mapping.

Directional Light

To obtain directional lighting data, you can access the DirectionalLight property. This directional light data provides the color of the light and a direction vector that represents the direction of the environment's main light source.

private void ProcessEstimateData(EstimateData estimateData)
{
    // Create variables for the directional light.
    Vector3 estimatedLightDirection = estimateData.DirectionalLight.Direction;
    Color estimatedLightColor = estimateData.DirectionalLight.Color;

    // Make the directional light in the scene point in the correct direction.
    _directionalLight.transform.LookAt(
    _directionalLight.transform.position + estimatedLightDirection);

    // Set the color of the directional light in the scene.
    _directionalLight.color = estimatedLightColor;
...

Environment Mapping

• The user’s environment is mapped in the form of a cubemap which can then be mapped to a mesh (e.g. sphere).

• The cubemap data is stored in the Cubemap property as part of the stored estimate data and contains two properties.

  • Pixels is a float array of pixel data of the cubemap representing the color of each pixel. The data is in the order of (R, G, B, A) with values between 0 and 1, and is organized into 6 faces in the order of (+X, -X, +Y, -Y, +Z, -Z). The raw data will appear upside down if mapped directly into Unity as-is.
  • FaceDimension represents the width/height dimension resolution of each face on the cubemap.

• The light estimation feature contains a method that returns a full cubemap in the proper orientation in the form of Unity’s Cubemap type called GetEstimateCubemap. This method takes two arguments.

  • The first argument is a float array intended for the raw pixels data.
  • The second argument is an int intended to specify the resolution of the face dimension of the cubemap.

• A common use-case is to apply the cubemap directly to a cubemap texture in a shader on a material.

...
    Cubemap cubemap = _lightEstimationFeature.GetEstimateCubemap(estimateData.CubeMap.Pixels, // You can obtain the pixel data from the cubemap.
                (int)estimateData.CubeMap.FaceDimension);

        // Update our renderer with our cubemap data.
        _cubemapRenderer.material.SetTexture("_Tex", cubemap);
}

You can also use a custom environment reflection and set the cubemap there. This way, all metallic/specular reflections will use the data captured in real time.

Go to Window > Rendering > Lighting > Environment Tab and set the "Source" field under "Environment Reflections" to Custom.

...
        RenderSettings.customReflectionTexture = cubemap;
...

Destroying Light Estimation

To release the light estimation data call DestroyLightEstimation on your instance of light estimation. This will free the resources that were used to generate the estimation.

private void OnDestroy()
{
    _lightEstimationFeature.DestroyLightEstimation();
}