Skip to main content
Version: 20 Mar 2024

Hand Tracking Design Guide

This guide provides design guidelines and best practices for the hand-tracking capabilities of Magic Leap 2.

You can use hand tracking as an input method within the apps you develop by using the available hand tracking and gesture classification APIs. In addition, users can choose to enable more hand-tracking features available in the Magic Leap 2 OS, although this doesn't affect the hand-tracking capabilities of your apps.

For information on developing apps that include hand tracking, see the Hand tracking developer guide in this developer portal.

For information about the hand-tracking features available to end users in the Magic Leap 2 OS, the Hand Tracking Guide in the Magic Leap Care portal.

Field of Sensing and Display Zone Near Boundary​

Hand gestures are only detected within the field of sensing, which is the field of view of the Magic Leap headset world camera. Near the edges of the field of sensing, hand tracking might work less effectively.

The near boundary of the Display Zone, also known as the near clipping plane, is the limit of how close virtual content can be displayed. Any virtual content attached to the user’s hand disappears when it crosses the near boundary of the Display Zone.

Users can interact with content within the field of sensing and Display Zone.

caution

For maximum comfort, the near boundary of the Display Zone is set to 37 cm by default. At closer distances, there is an increased likelihood of discomfort, such as eye strain, blurring, double vision, dizziness, and nausea. For more information about how users experience virtual content close to the near boundary of the Display Zone, see Display Zone and Clipping Plane.

A person wearing a Magic Leap 2 headset with their hand in the safe area within the field of sensing and further from them than the near boundary of the Display Zone (called the clipping plane in this illustration).

Content Placement​

The Magic Leap OS has two options for virtual content and user interface (UI) location, and corresponding interaction modes:

  • Near field interaction: Interaction with virtual content or UI elements at approximately 50 cm distance.
  • Far field interaction: Interaction with virtual content or UI elements at approximately 1 m distance.

Near Field Interaction​

For near field interaction, ideally content is placed about 45 cm in front and about 20 cm lower than the eye level to avoid arm fatigue. The lower the two-dimensional content is from the user’s eye level, the more tilted it should be from the vertical. This ensures comfort when interacting with content.

A person wearing a Magic Leap 2 headset, using their hand to interact with virtual content about 45 cm away from them and at least 20 cm below their eye level. The content positioned lower is tilted closer to horizontal than the content positioned higher.

Far Field Interaction​

For far field interaction, content should be placed between 60 cm and 1.7 m from the user. This is far enough to interact comfortably with the pinch gesture and close enough to be visibly legible. The further the object is from the user, the larger the UI buttons should be.

A person wearing a Magic Leap 2 headset, standing in front of two sets of virtual content and pointing toward the virtual content with their index finger. One set of virtual content is 60 cm away from the user and as wide as it is high. The other set of virtual content is 1.7 m away from the user and 2.83 times wider than it is high.

Interacting With Content​

When designing interactions with content, different gestures are needed for near field interactions and far field interactions.

tip

Many basic hand-tracking interactions are available out of the box in MRTK. See Magic Leap MRTK3 Package.

Near Field Gestures​

Use poke and pinch gestures for near field interactions.

Select. Poking in the near field is the most comfortable and intuitive gesture. Do not place content where the poking finger might be occluded from the headset's camera.

This short animation shows a user’s hand selecting a virtual UI item by touching it with the tip of the index finger.
This short animation shows a user’s hand selecting a virtual UI item by touching it with the tip of the index finger.

Scroll vertically or scroll horizontally. Use the poke gesture for scrolling in the near field:

  1. Poke and hold.
  2. Move in the desired direction and release.
This short animation shows a user’s hand selecting and moving virtual UI items by touching the UI with the tip of the index finger.
This short animation shows a user’s hand selecting and moving virtual UI items by touching the UI with the tip of the index finger.

Grab: Use the pinch gesture for moving a virtual object:

  1. Pinch and hold.
  2. Move in the desired direction and release.
This short animation shows a user’s hand moving a virtual cube by grasping it between the thumb and index finger and releasing it after placing it in a new position.
This short animation shows a user’s hand moving a virtual cube by grasping it between the thumb and index finger and releasing it after placing it in a new position.

Far Field Gestures​

Use pinch gestures for far field interactions.

Select. Use the pinch gesture for selecting an object in the far field:

  1. Aim.
  2. Pinch to select.
This short animation shows a user’s hand making a pinch gesture with the thumb and the index finger slightly separated to aim a ray at a virtual UI item and then touching the tip of the thumb and index finger together to select the item.
This short animation shows a user’s hand making a pinch gesture with the thumb and the index finger slightly separated to aim a ray at a virtual UI item and then touching the tip of the thumb and index finger together to select the item.

Scroll vertically or scroll horizontally. Use the pinch gesture for scrolling in the far field:

  1. Aim.
  2. Pinch and hold.
  3. Move in the desired direction and release.
This short animation shows a user’s hand selecting a virtual UI item by making a pinch gesture with the thumb and the index finger slightly separated to aim a ray at the item and then touching the tip of the thumb and index finger together to select and move the item.
This short animation shows a user’s hand selecting a virtual UI item by making a pinch gesture with the thumb and the index finger slightly separated to aim a ray at the item and then touching the tip of the thumb and index finger together to select and move the item.

Grab: Use the pinch gesture for moving a virtual object:

  1. Aim.
  2. Pinch and hold.
  3. Move in the desired direction and release.
This short animation shows a user’s hand making a pinch gesture with the thumb and index finger slightly separated to aim a ray at a virtual cube and then touching the tip of the thumb and index finger together to select the cube and move it. After moving the cube to a new location, the user releases the cube by separating the tips of their thumb and index finger.
This short animation shows a user’s hand making a pinch gesture with the thumb and index finger slightly separated to aim a ray at a virtual cube and then touching the tip of the thumb and index finger together to select the cube and move it. After moving the cube to a new location, the user releases the cube by separating the tips of their thumb and index finger.

Interactions With the Controller​

If you are designing interactions that include input from the Magic Leap 2 controller and user's hands:

  • Clearly separate these inputs.
  • Design a method to recognize which type of input is being used.

More Design Guidelines​

For general guidelines about user comfort and content placement for Magic Leap 2, see Comfort and Content Placement.