Hand Tracking
Support for common OpenXR features is provided by Epic's own OpenXR plugins, which are included in Unreal Engine 5.
Support for hand joint tracking is provided by Epic's OpenXRHandTracking plugin.
Note that enabling OpenXRHandTracking plugin will only enable joint tracking. See Hand Interaction for information about how to enable gesture detection in your project.
Use the plugin browser (Edit menu -> Plugins from the UE editor window) to enable both plugins to enable hand tracking in your project.
Make sure that you declare the com.magicleap.permission.HAND_TRACKING permission in your apps manifest.
This can be done by adding extra permissions in the APK packaging settings in Project Settings.
See this page for details- Permissions
When the plugins are enabled, your hands will be detected as motion controllers by Unreal.
See Unreal documentation on motion controllers for more details-
https://dev.epicgames.com/documentation/en-us/unreal-engine/motion-controller-component-setup-in-unreal-engine
In a blueprint, you can use a 'Get Motion Controller Data' node to get joint pose data.
You can use the blueprint function library provided by Epic's XRVisualization plugin to render a simple debug visualization of you hand joints.
This blueprint uses the 'Render Motion Controller' function provided by the visualization library to render hand joints.
Joint data is surfaced in the HandKeyPositions and HandKeyRotations fields of the FXRMotionControllerData structure that is output.
Example blueprint that renders an axis widget at each right hand joint-
The index of each value corresponds to the EHandKeypoint enum.
| Index | Joint |
|---|---|
| 0 | Palm |
| 1 | Wrist |
| 2 | ThumbMetacarpal |
| 3 | ThumbProximal |
| 4 | ThumbDistal |
| 5 | ThumbTip |
| 6 | IndexMetacarpal |
| 7 | IndexProximal |
| 8 | IndexIntermediate |
| 9 | IndexDistal |
| 10 | IndexTip |
| 11 | MiddleMetacarpal |
| 12 | MiddleProximal |
| 13 | MiddleIntermediate |
| 14 | MiddleDistal |
| 15 | MiddleTip |
| 16 | RingMetacarpal |
| 17 | RingProximal |
| 18 | RingIntermediate |
| 19 | RingDistal |
| 20 | RingTip |
| 21 | LittleMetacarpal |
| 22 | LittleProximal |
| 23 | LittleIntermediate |
| 24 | LittleDistal |
| 25 | LittleTip |