Version: 21 Aug 2024

Hand Interaction

Hand joint pose tracking is provided by Epic's OpenXRHandTracking plugin, but this does not include functionality for detecting hand gestures.

You can use the "Magic Leap Hand Interaction" plugin to detect hand gestures and standard poses in your project.

For detailed information about the origin of each pose relative to the hand see the OpenXR Hand Interaction Extension documentation.

The plugin can be enabled from the plugin browser. In the Unreal editor, select Edit -> Plugins. Enable "Magic Leap Hand Interaction" to enable the plugin.

After enabling the plugin, detectable gestures can be selected from Edit -> Project Settings -> Plugins - Magic Leap Hand Interaction

Gestures will only be detected if they are enabled in the settings panel. Note that no gestures are enabled by default.

Blueprint Function Library

The plugin includes a library of blueprint functions that can be used to get the current state of a tracked interaction.

BP Accessibility	Function Signature	Description
BlueprintPure	`bool GetInteractionReady(EControllerHand Hand, EMagicLeapHandInteractionType Type)`	Outputs a bool value indicating whether a interaction is currently detected
BlueprintPure	`bool GetInteractionValue(EControllerHand Hand, EMagicLeapHandInteractionType Type, float& OutValue)`	Outputs a float value indicating the state of a interaction (see supported interactions bellow)
BlueprintPure	`bool GetInteractionPose(EControllerHand Hand, EMagicLeapHandInteractionType Type, FTransform& OutPose)`	Outputs a transform value indicating a world space pose for a tracked interaction (see supported interactions bellow)

Note that a particular hand must be specified. "AnyHand" is not supported.

Supported Interactions

The following interactions are supported:

Name	InteractionReady	InteractionValue	InteractionPose
Pinch	true if the gesture is detected (Value will be non-zero)	[0.0f, 1.0f] Indicates how close the tips of the user's thumb and index finger are to touching each other, where a value of 1.0 indicates that the finger tips are touching.	The thumb and index finger are flexed. The tips of the thumb and index finger are close to each other but not necessarily touching. The other fingers are flexed toward the palm.
Poke	-NA- (always false)	-NA- (always 0.0)	Pose that tracks the user's index fingertip when it is outstretched.
Grip	true if pose is tracked (pose will be a valid world space transform)	non-zero when tracked	Pose that can be used to render a virtual object held in the user’s hand.
Aim	true if pose is tracked (pose will be a valid world space transform)	non-zero when tracked	Pose that allows applications to point in the world using a ray from their hand pointed towards a target.
Palm (native only)	true if pose is tracked (pose will be a valid world space transform)	non-zero when tracked	Pose that allows applications to reliably anchor visual content relative to the user’s physical hand.