Magic Leap Eye Tracker API Overview

The MagicLeapEyeTrackerFeature is an experimental OpenXR extension for Unity, enabling developers to integrate eye-tracking capabilities into applications running on Magic Leap 2. This API provides comprehensive access to data such as gaze direction, eye positions, pupil sizes, and gaze behaviors, essential for creating immersive and interactive user experiences.

Namespace

using MagicLeap.OpenXR.Features.EyeTracker;

Types of Data Available

The eye-tracking data is categorized into several types:

1. Geometric Data: Information about the geometric properties of the eyes, such as eye openness and position within the skull.
2. Pupil Data: Data related to the pupil size of each eye. (Gated by the PUPIL_SIZE permission)
3. Gaze Behavior Data: Details about the user's gaze behaviors, such as fixation, saccade, and pursuit.
4. Static Data: Static information about the hardware, like the maximum width and height of the eye cameras.
5. Pose Data: Positional data of the eyes, including the gaze direction and position of the left and right eyes.

Prerequisites

Before using the eye tracker, ensure that your project is configured for Magic Leap OpenXR and that the MagicLeapEyeTrackerFeature is enabled in the XR Plugin Management settings.

API Functions Overview

Requesting Permissions

Before accessing eye-tracking data, you must request permissions from the user. Required permissions include EyeTracking for basic tracking capabilities and PupilSize for accessing pupil dilation metrics. Make sure the permissions are also declared in your project's manifest. (Project Settings > Magic Leap > Permissions)

This example demonstrates how to request the necessary permissions from the user before attempting to initialize and use the eye tracker.

using MagicLeap.Android;
using UnityEngine;

public class EyeTrackerPermissions : MonoBehaviour
{
    void Start()
    {
        Permissions.RequestPermissions(new string[] { Permissions.EyeTracking, Permissions.PupilSize }, OnPermissionGranted, OnPermissionDenied);
    }

    private void OnPermissionGranted(string permission)
    {
        Debug.Log($"{permission} granted.");
        // Initialize the eye tracker here or notify other components that permission has been granted.
    }

    private void OnPermissionDenied(string permission)
    {
        Debug.LogError($"{permission} denied. Eye tracking will not function.");
    }
}

Creating and Destroying the Eye Tracker

• CreateEyeTracker: Initializes the eye tracker. This method must be invoked before attempting to retrieve any eye-tracking data.
• DestroyEyeTracker: Releases the eye tracker and its resources. This should be called when the eye tracker is no longer needed, typically upon disabling the component or closing the application.

This example shows how to initialize the eye tracker, and assumes that permissions have already been granted. It uses the CreateEyeTracker method to start the eye tracker and prepares it to gather data.

using UnityEngine;
using UnityEngine.XR.OpenXR;
using MagicLeap.OpenXR.Features.EyeTracker;

public class EyeTrackerInitializer : MonoBehaviour
{
    private MagicLeapEyeTrackerFeature eyeTrackerFeature;

    void Start()
    {
        eyeTrackerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapEyeTrackerFeature>();
        if (eyeTrackerFeature != null && eyeTrackerFeature.enabled)
        {
            eyeTrackerFeature.CreateEyeTracker();
            Debug.Log("Eye Tracker initialized.");
        }
    }

    void OnDestroy()
    {
        if (eyeTrackerFeature != null && eyeTrackerFeature.enabled)
        {
            eyeTrackerFeature.DestroyEyeTracker();
            Debug.Log("Eye Tracker destroyed.");
        }
    }
}

Retrieving Eye Tracking Data

GetEyeTrackerData: Fetches a comprehensive set of eye-tracking data, encapsulating various metrics such as static data about the hardware, geometric data about the eyes' positions, pupil data, and behavioral data concerning the user's gaze.

This example illustrates how to retrieve and log various eye-tracking data using the GetEyeTrackerData method.

using UnityEngine;
using MagicLeap.OpenXR.Features.EyeTracker;

public class EyeTrackerDataLogger : MonoBehaviour
{
    private MagicLeapEyeTrackerFeature eyeTrackerFeature;

    void Start()
    {
        eyeTrackerFeature = OpenXRSettings.Instance.GetFeature<MagicLeapEyeTrackerFeature>();
        if (eyeTrackerFeature != null && eyeTrackerFeature.enabled)
        {
            eyeTrackerFeature.CreateEyeTracker();
        }
    }

    void Update()
    {
        if (eyeTrackerFeature != null && eyeTrackerFeature.enabled)
        {
            EyeTrackerData data = eyeTrackerFeature.GetEyeTrackerData();

            // Static Data
            StaticData staticData = eyeTrackerFeature.GetEyeTrackerData().StaticData;
            Debug.Log($"Eye Width Max: {staticData.EyeWidthMax}, Eye Height Max: {staticData.EyeHeightMax}");

            // Pupil Data
            PupilData[] pupilData = data.PupilData;
            foreach (var data in pupilData)
            {
                Debug.Log($"Eye: {data.Eye}, Pupil Diameter: {data.PupilDiameter} meters");
            }

            // GazeBehavior Data
            GazeBehavior gazeBehavior = data.GazeBehaviorData;
            Debug.Log($"Gaze Behavior: {gazeBehavior.GazeBehaviorType}, Time: {gazeBehavior.Time}, Duration: {gazeBehavior.Duration}");
            // Log Gaze Direction
            Vector3 gazePosition = gazeBehavior.MetaData.Direction;
            Debug.Log($"Gaze Position: {gazePosition}");

            // GeometricData Data | 0 = Left 1 = Right
            GeometricData[] geometricData = data.GeometricData;
            foreach (var data in geometricData)
            {
                Debug.Log($"Eye: {data.Eye}, Openness: {data.EyeOpenness}, Position: {data.EyeInSkullPosition}");
            }

        }
    }

    void OnDestroy()
    {
        if (eyeTrackerFeature != null && eyeTrackerFeature.enabled)
        {
            eyeTrackerFeature.DestroyEyeTracker();
        }
    }
}

Data Types

GeometricData

GeometricData geometricData = new GeometricData
{
    Eye = (Eye)geometricDataNative.Eye,            // Indicates whether the data is for the left or right eye
    Time = geometricDataNative.Time,               // Timestamp of when the geometric data was captured
    Valid = geometricDataNative.Valid,             // Indicates whether the geometric data is valid
    EyeOpenness = geometricDataNative.EyeOpenness, // Represents the openness of the eye, ranging from 0.0 (closed) to 1.0 (fully open)
    EyeInSkullPosition = new Vector2
    {
        x = geometricDataNative.EyeInSkullPosition.x, // X position of the eye in the skull
        y = geometricDataNative.EyeInSkullPosition.y  // Y position of the eye in the skull
    }
};

PupilData

PupilData pupilData = new PupilData
{
    Eye = (Eye)pupilDataNative.Eye,              // Indicates whether the data is for the left or right eye.
    Time = pupilDataNative.Time,                 // Timestamp of when the pupil data was captured
    Valid = pupilDataNative.Valid,               // Indicates whether the pupil data is valid
    PupilDiameter = pupilDataNative.PupilDiameter // Diameter of the pupil in meters
};

GazeBehavior

GazeBehavior gazeData = new GazeBehavior
{
    MetaData = new GazeBehaviorMetaData
    {
        Valid = gazeMetaDataNative.Valid,                // Indicates whether the gaze behavior metadata is valid
        Amplitude = gazeMetaDataNative.Amplitude,        // Represents the displacement of the gaze in degrees of visual angle
        Direction = gazeMetaDataNative.Direction,        // Angle of the gaze movement in radians
        Velocity = gazeMetaDataNative.Velocity           // Speed of the gaze movement in degrees per second
    },
    Time = gazeDataNative.Time,                          // Timestamp of when the gaze behavior data was captured
    Valid = gazeDataNative.Valid,                        // Indicates whether the gaze behavior data is valid
    GazeBehaviorType = (GazeBehaviorType)gazeDataNative.GazeBehaviorType, // Type of the gaze behavior detected (e.g., fixation, saccade)
    OnsetTime = gazeDataNative.OnsetTime,                // Time when the current gaze behavior started
    Duration = gazeDataNative.Duration                   // Duration of the current gaze behavior
};

StaticData

StaticData staticData = new StaticData
{
    EyeWidthMax = staticDataNative.EyeWidthMax, // Maximum width of the eye in the skull
    EyeHeightMax = staticDataNative.EyeHeightMax // Maximum height of the eye in the skull
};