Skip to main content
Version: 21 Aug 2024

Eye Tracking

Magic Leap 2 provides a set of APIs that gives you the ability to access the user's eye data. These APIs can be used to understand the position and rotation of the eyes, as well as the ability to track what the user is looking at. This page explains how you can benefit from eye tracking for various use cases, and what to look for when designing eye-gaze-based user interactions.

The predicted eye-gaze is approximately within 1 degree in visual angle around the actual target. Slight imprecisions are expected, so plan for some margin around this lower-bound value.

Eye Tracking information can be accessed using the Gaze Recognition API. You can also access raw grayscale frames from four Magic Leap 2 eye cameras using the Eye Camera API.

note

The Eye Camera API is an experimental API.

For the best eye tracking performance, it is crucial that the user completes eye calibration and fit process. You can determine the status of current eye calibration and fit with the Eye Calibration and Fit APIs. Prompt users to go through the calibration and fit processes if needed.

What follows on this page are details about all available APIs for eye tracking, and resources for learning more about each. Sections include:

Eye Tracking API​

Eye tracking on Magic Leap 2 allows you to design natural and intuitive input and interaction scenarios. The Eye Tracking API provides information about what the user is looking at as a single eye-gaze ray at approximately 60 FPS (60 Hz). For eye tracking to work accurately, each user is required to go through an eye calibration.

The Eye Tracking APIs have been designed with a user’s privacy in mind; it avoids the passing of any identifiable information, particularly any biometric data, to the application.

The API requires the com.magicleap.permission.EYE_TRACKING permission to be declared in the application manifest. This is a dangerous permission and requires application user consent for the API to expose the data.

note

If your application collects, stores, transfers or otherwise uses data off the Magic Leap 2 device that is received via this API, then you must comply with the Magic Leap 2 Eye Tracking Data Transparency Policy.

Please refer to the API documentation for more information. SDK package contains a sample app demonstrating the use of the API.

API Documentation:

Sample Apps:

Gaze Recognition API​

The Gaze Recognition API reports an estimate of the user’s oculomotor behavior, metadata about the current eye behavior, and useful high-precision eye position signals at a given point in time.

The following mutually exclusive behaviors are currently available:

Eye Gaze Behaviors

The Gaze Recognition APIs have been designed with a user’s privacy in mind; it avoids the passing of any identifiable information, particularly any biometric data.

The API requires the com.magicleap.permission.EYE_TRACKING permission to be declared in the application manifest. This is a dangerous permission and requires application user consent for the API to expose the data. Please refer to API documentation for more information. SDK package contains a sample app demonstrating the use of the API.

API Documentation:

Sample Apps:

Eye Camera API​

The API requires com.magicleap.permission.EYE_CAMERA permission to be declared in the application manifest. This is a dangerous permission and requires application user consent for the API to expose the data.

Please refer to API documentation for more information. SDK package contains a sample app demonstrating the use of the API.

Image of an Eye Taken By Right Nasal and Right Temple Eye Cameras

API Documentation:

  • CAPI
  • Unity
  • Unreal - CAPI eye camera is not available in Unreal.

Sample Apps:

Eye Calibration API​

You can use the Eye Calibration API to determine the current status of the eye calibration system.

note

If your application collects, stores, transfers or otherwise uses data off the Magic Leap 2 device that is received via this API, then you must comply with the Magic Leap 2 Eye Tracking Data Transparency Policy.

The Eye Calibration API returns one of three outputs:

  • Fine - eye-tracking is precise enough to use as input for selecting buttons or icons.
  • Coarse - eye-tracking is usable for basic interactions such as scrolling, but not precise enough to use for selecting buttons or icons.
  • None - Eye calibration did not run or was not completed successfully.

The success of eye calibration depends on these things:

  • Most importantly, the success of fit; if the eyes are not seen via the Headset Fit API then they won’t be tracked via the Eye Calibration API.
  • Prescription (if the user needs prescription but isn’t wearing it).
  • Eye conditions.

API Documentation:

  • CAPI
  • Unity
  • Unreal - CAPI eye camera is not available in Unreal.

Sample Apps:

  • CAPI - N/A
  • Unity - N/A
  • Unreal - CAPI eye calibration is not available in Unreal.

Headset Fit API​

You can use the Headset Fit API to check whether the device is correctly fitted to the user or if users need to complete Custom Fit again.

The Magic Leap 2 device is constantly checking for headset fit. You can obtain the current state by using the Headset Fit API. If you’d like to check fit while running your application, call up the Headset Fit API, which will make sure the user’s eyes are seen.

The Headset Fit API returns one of four options:

  • Unknown: device needs to be rebooted.
  • Not Worn: eyes are not visible for 2 consecutive seconds; Either the headset was taken off or the fit is so bad that the user’s eyes aren’t visible at all.
  • Good Fit: eyes are visible and the device is properly fitted to the user.
  • Bad Fit: eyes are not in the correct location. Eye Tracking will not be functional.

The Headset Fit API judges fit in five second cycles. It is recommended to check for the Headset Fit API 5-10 seconds into your application and prompt the user to launch the Custom Fit app if the Custom Fit API returns a “Bad Fit” state. The Custom Fit App can be launched using the standard Android intent mechanism. Please refer to this article for more details.

API Documentation:

  • CAPI
  • Unity
  • Unreal - CAPI eye calibration is not available in Unreal.

Sample Apps:

  • CAPI - N/A
  • Unity - N/A
  • Unreal - CAPI headset fit is not available in Unreal.

Headset Fit Tips​

We strongly recommend that users complete Custom Fit to ensure proper device fit

Completing the Custom Fit app helps maximize Magic Leap 2 performance by fitting the device to each user’s unique facial dimensions and features. Use Headset Fit API to determine if Custom Fit was completed and the fit is good. If fit is not good, alert app users and send them to the Custom Fit app to complete the fit procedure. If your users have completed the Custom Fit app, then the device should be able to see the user’s eyes.

Encourage your users to launch the Custom Fit app from the main menu to position the back and side bands to rest above the ears and choose suitable nose and forehead pads from the Fit Kit. Keep in mind that when multiple users are sharing a device, the Custom Fit app should be re-launched for each user.

Poor Eye-Fit​

  • Eye strain and headaches as a result of vertical and/or horizontal vergence discrepancy
  • Segmented dimmer misalignment
  • Mismatched content depth placement
  • Poor Iris ID performance
  • Poor Gaze Tracking performance
  • Poor iris or pupil measurement performance
  • Significantly reduced virtual display FOV

Virtual display FOV of a worst-case scenario user that has not completed Custom Fit (Vertical: 39°/55°; Horizontal: 33°/45°; Diagonal: 37°/71°)

Worst-case scenario