Version: 14 Oct 2024

ml_hand_tracking.h

Classes

	Name
struct	MLHandTrackingCFUIDs MLCoordinateFrameUIDs for the keypoints.
struct	MLHandTrackingStaticData Static information about a hand tracker.
struct	MLHandTrackingHandState State of a single hand.
struct	MLHandTrackingData Data which is received when querying hand tracker from MLHandTrackingGetData().
struct	MLHandTrackingSettings Settings of the hand tracking system.

Types

	Name
typedef struct MLHandTrackingCFUIDs	MLHandTrackingCFUIDs MLCoordinateFrameUIDs for the keypoints.
typedef struct MLHandTrackingStaticData	MLHandTrackingStaticData Static information about a hand tracker.
typedef struct MLHandTrackingHandState	MLHandTrackingHandState State of a single hand.
typedef struct MLHandTrackingData	MLHandTrackingData Data which is received when querying hand tracker from MLHandTrackingGetData().
typedef struct MLHandTrackingSettings	MLHandTrackingSettings Settings of the hand tracking system.

Enums

	Name
enum	MLHandTrackingHandType { MLHandTrackingHandType_Left = 0, MLHandTrackingHandType_Right = 1, MLHandTrackingHandType_Count = 2, MLHandTrackingHandType_Ensure32Bits = 0x7FFFFFFF }
enum	Anonymous Enum 12 { MLHandTrackingStaticData_MaxKeyPoints = 28 }
enum	MLHandTrackingKeyPoint { MLHandTrackingKeyPoint_Thumb_Tip = 0, MLHandTrackingKeyPoint_Thumb_IP, MLHandTrackingKeyPoint_Thumb_MCP, MLHandTrackingKeyPoint_Thumb_CMC, MLHandTrackingKeyPoint_Index_Tip, MLHandTrackingKeyPoint_Index_DIP, MLHandTrackingKeyPoint_Index_PIP, MLHandTrackingKeyPoint_Index_MCP, MLHandTrackingKeyPoint_Middle_Tip, MLHandTrackingKeyPoint_Middle_DIP, MLHandTrackingKeyPoint_Middle_PIP, MLHandTrackingKeyPoint_Middle_MCP, MLHandTrackingKeyPoint_Ring_Tip, MLHandTrackingKeyPoint_Ring_DIP, MLHandTrackingKeyPoint_Ring_PIP, MLHandTrackingKeyPoint_Ring_MCP, MLHandTrackingKeyPoint_Pinky_Tip, MLHandTrackingKeyPoint_Pinky_DIP, MLHandTrackingKeyPoint_Pinky_PIP, MLHandTrackingKeyPoint_Pinky_MCP, MLHandTrackingKeyPoint_Wrist_Center, MLHandTrackingKeyPoint_Wrist_Ulnar, MLHandTrackingKeyPoint_Wrist_Radial, MLHandTrackingKeyPoint_Hand_Center, MLHandTrackingKeyPoint_Index_Meta, MLHandTrackingKeyPoint_Middle_Meta, MLHandTrackingKeyPoint_Ring_Meta, MLHandTrackingKeyPoint_Pinky_Meta, MLHandTrackingKeyPoint_Count = MLHandTrackingStaticData_MaxKeyPoints, MLHandTrackingKeyPoint_Ensure32Bits = 0x7FFFFFFF } Keypoint index ordering.
enum	MLHandTrackingSettingsFlags { MLHandTrackingSettingsFlags_None = 0, MLHandTrackingSettingsFlags_UnfilteredKeypoints = 1 << 0, MLHandTrackingSettingsFlags_Ensure32Bits = 0x7FFFFFFF } Enumeration of flags to select settings related to hand tracking.

Functions

	Name
void	MLHandTrackingStaticDataInit(MLHandTrackingStaticData * inout_attr) Initializes default values for MLHandTrackingStaticData.
void	MLHandTrackingDataInit(MLHandTrackingData * inout_attr) Initializes values for MLHandTrackingData.
void	MLHandTrackingSettingsInit(MLHandTrackingSettings * inout_settings) Initializes default values for MLHandTrackingSettings.
MLResult	MLHandTrackingCreate(MLHandle * out_handle) Creates a hand tracker.
MLResult	MLHandTrackingCreateEx(const MLHandTrackingSettings settings, MLHandle out_handle) Creates a hand tracker.
MLResult	MLHandTrackingUpdateSettings(MLHandle hand_tracker, const MLHandTrackingSettings * settings) Updates settings.
MLResult	MLHandTrackingDestroy(MLHandle hand_tracker) Destroys a hand tracker.
MLResult	MLHandTrackingGetData(MLHandle hand_tracker, MLHandTrackingData * out_data) Queries the state of the hand tracker.
MLResult	MLHandTrackingGetStaticData(MLHandle hand_tracker, MLHandTrackingStaticData * out_data) Gets static information about hand tracking system.

Enums Documentation

MLHandTrackingHandType

Enumerator	Value	Description
MLHandTrackingHandType_Left	0	Left hand.
MLHandTrackingHandType_Right	1	Right hand.
MLHandTrackingHandType_Count	2	Number of hands.
MLHandTrackingHandType_Ensure32Bits	0x7FFFFFFF	Ensure enum is represented as 32 bits.

Available hand types.

Anonymous Enum 12

Enumerator	Value	Description
MLHandTrackingStaticData_MaxKeyPoints	28	Maximum number of key points per hand.

MLHandTrackingKeyPoint

Enumerator	Value	Description
MLHandTrackingKeyPoint_Thumb_Tip	0
MLHandTrackingKeyPoint_Thumb_IP
MLHandTrackingKeyPoint_Thumb_MCP
MLHandTrackingKeyPoint_Thumb_CMC
MLHandTrackingKeyPoint_Index_Tip
MLHandTrackingKeyPoint_Index_DIP
MLHandTrackingKeyPoint_Index_PIP
MLHandTrackingKeyPoint_Index_MCP
MLHandTrackingKeyPoint_Middle_Tip
MLHandTrackingKeyPoint_Middle_DIP
MLHandTrackingKeyPoint_Middle_PIP
MLHandTrackingKeyPoint_Middle_MCP
MLHandTrackingKeyPoint_Ring_Tip
MLHandTrackingKeyPoint_Ring_DIP
MLHandTrackingKeyPoint_Ring_PIP
MLHandTrackingKeyPoint_Ring_MCP
MLHandTrackingKeyPoint_Pinky_Tip
MLHandTrackingKeyPoint_Pinky_DIP
MLHandTrackingKeyPoint_Pinky_PIP
MLHandTrackingKeyPoint_Pinky_MCP
MLHandTrackingKeyPoint_Wrist_Center
MLHandTrackingKeyPoint_Wrist_Ulnar
MLHandTrackingKeyPoint_Wrist_Radial
MLHandTrackingKeyPoint_Hand_Center
MLHandTrackingKeyPoint_Index_Meta
MLHandTrackingKeyPoint_Middle_Meta
MLHandTrackingKeyPoint_Ring_Meta
MLHandTrackingKeyPoint_Pinky_Meta
MLHandTrackingKeyPoint_Count	MLHandTrackingStaticData_MaxKeyPoints	Maximum number of key points per gesture.
MLHandTrackingKeyPoint_Ensure32Bits	0x7FFFFFFF

Keypoint index ordering.

The index ordering of 28 keypoints exposed in array keypoints_mask[MLHandTrackingStaticData_MaxKeyPoints] and left_frame/right_frame[MLHandTrackingStaticData_MaxKeyPoints].

API Level:

MLHandTrackingSettingsFlags

Enumerator	Value	Description
MLHandTrackingSettingsFlags_None	0	No additional settings used.
MLHandTrackingSettingsFlags_UnfilteredKeypoints	1 << 0	Enable unfiltered keypoints. See MLHandTrackingGetStaticData for more details.
MLHandTrackingSettingsFlags_Ensure32Bits	0x7FFFFFFF	Ensure enum is represented as 32 bits.

Enumeration of flags to select settings related to hand tracking.

Filtered keypoints are always available, no matter the settings.

API Level:

Types Documentation

MLHandTrackingCFUIDs

typedef struct MLHandTrackingCFUIDs MLHandTrackingCFUIDs;

MLCoordinateFrameUIDs for the keypoints.

See MLHandTrackingKeyPoint for more details.

More Info

API Level:

MLHandTrackingStaticData

typedef struct MLHandTrackingStaticData MLHandTrackingStaticData;

Static information about a hand tracker.

This structure must be initialized by calling MLHandTrackingStaticDataInit() before use.

More Info

API Level:

MLHandTrackingHandState

typedef struct MLHandTrackingHandState MLHandTrackingHandState;

State of a single hand.

Same for both filtered and unfiltered keypoints.

More Info

API Level:

MLHandTrackingData

typedef struct MLHandTrackingData MLHandTrackingData;

Data which is received when querying hand tracker from MLHandTrackingGetData().

This structure must be initialized by calling MLHandTrackingDataInit() before use.

More Info

API Level:

MLHandTrackingSettings

typedef struct MLHandTrackingSettings MLHandTrackingSettings;

Settings of the hand tracking system.

This structure must be initialized by calling MLHandTrackingSettingsInit().

More Info

API Level:

Functions Documentation

MLHandTrackingStaticDataInit

static inline void MLHandTrackingStaticDataInit(
    MLHandTrackingStaticData * inout_attr
)

Initializes default values for MLHandTrackingStaticData.

Parameters


MLHandTrackingStaticData *	inout_attr	The object to initialize with default values.

API Level:

MLHandTrackingDataInit

static inline void MLHandTrackingDataInit(
    MLHandTrackingData * inout_attr
)

Initializes values for MLHandTrackingData.

Parameters


MLHandTrackingData *	inout_attr	The object to initialize.

API Level:

MLHandTrackingSettingsInit

static inline void MLHandTrackingSettingsInit(
    MLHandTrackingSettings * inout_settings
)

Initializes default values for MLHandTrackingSettings.

Parameters


MLHandTrackingSettings *	inout_settings	The object to initialize with default settings.

API Level:

MLHandTrackingCreate

MLResult MLHandTrackingCreate(
    MLHandle * out_handle
)

Creates a hand tracker.

Parameters


MLHandle *	out_handle	A pointer to an MLHandle which will contain the handle of the hand tracker. If this operation fails, out_handle will be ML_INVALID_HANDLE.

Returns


MLResult	MLResult_InvalidParam	Failed due to an invalid parameter.
MLResult	MLResult_Ok	The tracker was created successfully.
MLResult	MLResult_PermissionDenied	The application lacks the required permission.
MLResult	MLResult_UnspecifiedFailure	Failed to create the tracker.

Required Permissions:

com.magicleap.permission.HAND_TRACKING (protection level: normal)

Deprecated

Deprecated since 1.3.0. Scheduled for removal. Replaced by MLHandTrackingCreateEx.

API Level:

MLHandTrackingCreateEx

MLResult MLHandTrackingCreateEx(
    const MLHandTrackingSettings * settings,
    MLHandle * out_handle
)

Creates a hand tracker.

Parameters


const MLHandTrackingSettings *	settings	A pointer to a previously initialized MLHandTrackingSettings struct.
MLHandle *	out_handle	A pointer to an MLHandle which will contain the handle of the hand tracker. If this operation fails, out_handle will be ML_INVALID_HANDLE.

Returns


MLResult	MLResult_InvalidParam	Failed due to an invalid parameter.
MLResult	MLResult_Ok	The tracker was created successfully.
MLResult	MLResult_PermissionDenied	The application lacks the required permission.
MLResult	MLResult_UnspecifiedFailure	Failed to create the tracker.

Required Permissions:

com.magicleap.permission.HAND_TRACKING (protection level: normal)

API Level:

MLHandTrackingUpdateSettings

MLResult MLHandTrackingUpdateSettings(
    MLHandle hand_tracker,
    const MLHandTrackingSettings * settings
)

Updates settings.

Parameters


MLHandle	hand_tracker	A handle to a Hand Tracker created by MLHandTrackingCreateEx().
const MLHandTrackingSettings *	settings	A pointer to a previously initialized MLHandTrackingSettings struct.

Returns


MLResult	MLResult_InvalidParam	Failed due to an invalid parameter.
MLResult	MLResult_Ok	The settings were updated successfully.
MLResult	MLResult_UnspecifiedFailure	Failed to update the settings.

Required Permissions:

None

API Level:

MLHandTrackingDestroy

MLResult MLHandTrackingDestroy(
    MLHandle hand_tracker
)

Destroys a hand tracker.

Parameters


MLHandle	hand_tracker	A handle to a Hand Tracker created by MLHandTrackingCreateEx().

Returns


MLResult	MLResult_Ok	It successfully destroyed the tracker.
MLResult	MLResult_UnspecifiedFailure	Failed to destroy the tracker.

Required Permissions:

None

API Level:

MLHandTrackingGetData

MLResult MLHandTrackingGetData(
    MLHandle hand_tracker,
    MLHandTrackingData * out_data
)

Queries the state of the hand tracker.

Parameters


MLHandle	hand_tracker	A handle to a Hand Tracker created by MLHandTrackingCreateEx().
MLHandTrackingData *	out_data	Pointer to a variable that receives information about the tracked hands.

Returns


MLResult	MLResult_InvalidParam	Failed due to an invalid parameter.
MLResult	MLResult_Ok	The hand information was available and the information in out_data is valid.
MLResult	MLResult_UnspecifiedFailure	Failed to get the hand information.

Required Permissions:

None

The MLHandTrackingData is the same for both filtered and unfiltered keypoints.

API Level:

MLHandTrackingGetStaticData

MLResult MLHandTrackingGetStaticData(
    MLHandle hand_tracker,
    MLHandTrackingStaticData * out_data
)

Gets static information about hand tracking system.

Parameters


MLHandle	hand_tracker	A handle to a Hand Tracker created by MLHandTrackingCreateEx().
MLHandTrackingStaticData *	out_data	Pointer to a variable that receives static data about the hand tracker.

Returns


MLResult	MLResult_InvalidParam	Failed due to an invalid parameter.
MLResult	MLResult_Ok	The hand information was available and the information in out_data is valid.
MLResult	MLResult_UnspecifiedFailure	Failed to get the hand information.

Required Permissions:

None

See MLHandTrackingStaticData for more details.

API Level:

Source code

// %BANNER_BEGIN%
// ---------------------------------------------------------------------
// %COPYRIGHT_BEGIN%
// Copyright (c) 2017 Magic Leap, Inc. All Rights Reserved.
// Use of this file is governed by the Software License Agreement,
// located here: https://www.magicleap.com/software-license-agreement-ml2
// Terms and conditions applicable to third-party materials accompanying
// this distribution may also be found in the top-level NOTICE file
// appearing herein.
// %COPYRIGHT_END%
// ---------------------------------------------------------------------
// %BANNER_END%

#pragma once

#include "ml_api.h"
#include "ml_types.h"

#include <string.h>

ML_EXTERN_C_BEGIN

typedef enum MLHandTrackingHandType {
  MLHandTrackingHandType_Left = 0,
  MLHandTrackingHandType_Right = 1,
  MLHandTrackingHandType_Count = 2,
  MLHandTrackingHandType_Ensure32Bits = 0x7FFFFFFF
} MLHandTrackingHandType;

enum {
  MLHandTrackingStaticData_MaxKeyPoints = 28
};

typedef enum MLHandTrackingKeyPoint {
  MLHandTrackingKeyPoint_Thumb_Tip = 0,
  MLHandTrackingKeyPoint_Thumb_IP,
  MLHandTrackingKeyPoint_Thumb_MCP,
  MLHandTrackingKeyPoint_Thumb_CMC,
  MLHandTrackingKeyPoint_Index_Tip,
  MLHandTrackingKeyPoint_Index_DIP,
  MLHandTrackingKeyPoint_Index_PIP,
  MLHandTrackingKeyPoint_Index_MCP,
  MLHandTrackingKeyPoint_Middle_Tip,
  MLHandTrackingKeyPoint_Middle_DIP,
  MLHandTrackingKeyPoint_Middle_PIP,
  MLHandTrackingKeyPoint_Middle_MCP,
  MLHandTrackingKeyPoint_Ring_Tip,
  MLHandTrackingKeyPoint_Ring_DIP,
  MLHandTrackingKeyPoint_Ring_PIP,
  MLHandTrackingKeyPoint_Ring_MCP,
  MLHandTrackingKeyPoint_Pinky_Tip,
  MLHandTrackingKeyPoint_Pinky_DIP,
  MLHandTrackingKeyPoint_Pinky_PIP,
  MLHandTrackingKeyPoint_Pinky_MCP,
  MLHandTrackingKeyPoint_Wrist_Center,
  MLHandTrackingKeyPoint_Wrist_Ulnar,
  MLHandTrackingKeyPoint_Wrist_Radial,
  MLHandTrackingKeyPoint_Hand_Center,
  MLHandTrackingKeyPoint_Index_Meta,
  MLHandTrackingKeyPoint_Middle_Meta,
  MLHandTrackingKeyPoint_Ring_Meta,
  MLHandTrackingKeyPoint_Pinky_Meta,
  MLHandTrackingKeyPoint_Count = MLHandTrackingStaticData_MaxKeyPoints,
  MLHandTrackingKeyPoint_Ensure32Bits = 0x7FFFFFFF
} MLHandTrackingKeyPoint;


typedef struct MLHandTrackingCFUIDs {
  MLCoordinateFrameUID keypoint_cfuids[MLHandTrackingStaticData_MaxKeyPoints];
} MLHandTrackingCFUIDs;

typedef struct MLHandTrackingStaticData {
  uint32_t version;
  MLHandTrackingCFUIDs hand_cfuids[MLHandTrackingHandType_Count];
  MLHandTrackingCFUIDs unfiltered_hand_cfuids[MLHandTrackingHandType_Count];
} MLHandTrackingStaticData;

ML_STATIC_INLINE void MLHandTrackingStaticDataInit(MLHandTrackingStaticData *inout_attr) {
  if (NULL != inout_attr) {
    memset(inout_attr, 0, sizeof(MLHandTrackingStaticData));
    inout_attr->version = 3u;
  }
}

typedef struct MLHandTrackingHandState {
  bool is_hand_detected;
  float hand_confidence;
  bool keypoints_mask[MLHandTrackingStaticData_MaxKeyPoints];
} MLHandTrackingHandState;

typedef struct MLHandTrackingData {
  uint32_t version;

  MLHandTrackingHandState hand_state[MLHandTrackingHandType_Count];

  MLTime timestamp_ns;

} MLHandTrackingData;

ML_STATIC_INLINE void MLHandTrackingDataInit(MLHandTrackingData *inout_attr) {
  if (NULL != inout_attr) {
    memset(inout_attr, 0, sizeof(MLHandTrackingData));
    inout_attr->version = 3u;
  }
}

typedef enum MLHandTrackingSettingsFlags {
  MLHandTrackingSettingsFlags_None = 0,
  MLHandTrackingSettingsFlags_UnfilteredKeypoints = 1 << 0,
  MLHandTrackingSettingsFlags_Ensure32Bits = 0x7FFFFFFF
} MLHandTrackingSettingsFlags;

typedef struct MLHandTrackingSettings {
  uint32_t version;
  uint32_t flags;
} MLHandTrackingSettings;

ML_STATIC_INLINE void MLHandTrackingSettingsInit(MLHandTrackingSettings *inout_settings) {
  if (NULL != inout_settings) {
    memset(inout_settings, 0, sizeof(MLHandTrackingSettings));
    inout_settings->version = 1u;
    inout_settings->flags = MLHandTrackingSettingsFlags_None;
  }
}

ML_DEPRECATED_MSG("Replaced by MLHandTrackingCreateEx.")
ML_API MLResult ML_CALL MLHandTrackingCreate(MLHandle *out_handle);

ML_API MLResult ML_CALL MLHandTrackingCreateEx(const MLHandTrackingSettings* settings, MLHandle *out_handle);

ML_API MLResult ML_CALL MLHandTrackingUpdateSettings(MLHandle hand_tracker, const MLHandTrackingSettings* settings);

ML_API MLResult ML_CALL MLHandTrackingDestroy(MLHandle hand_tracker);

ML_API MLResult ML_CALL MLHandTrackingGetData(MLHandle hand_tracker, MLHandTrackingData *out_data);

ML_API MLResult ML_CALL MLHandTrackingGetStaticData(MLHandle hand_tracker, MLHandTrackingStaticData *out_data);

ML_EXTERN_C_END

ml_hand_tracking.h

Classes​

Types​

Enums​

Functions​

Enums Documentation​

MLHandTrackingHandType​

Anonymous Enum 12​

MLHandTrackingKeyPoint​

MLHandTrackingSettingsFlags​

Types Documentation​

MLHandTrackingCFUIDs​

MLHandTrackingStaticData​

MLHandTrackingHandState​

MLHandTrackingData​

MLHandTrackingSettings​

Functions Documentation​

MLHandTrackingStaticDataInit​

MLHandTrackingDataInit​

MLHandTrackingSettingsInit​

MLHandTrackingCreate​

MLHandTrackingCreateEx​

MLHandTrackingUpdateSettings​

MLHandTrackingDestroy​

MLHandTrackingGetData​

MLHandTrackingGetStaticData​

Source code​

Classes

Types

Enums

Functions

Enums Documentation

MLHandTrackingHandType

Anonymous Enum 12

MLHandTrackingKeyPoint

MLHandTrackingSettingsFlags

Types Documentation

MLHandTrackingCFUIDs

MLHandTrackingStaticData

MLHandTrackingHandState

MLHandTrackingData

MLHandTrackingSettings

Functions Documentation

MLHandTrackingStaticDataInit

MLHandTrackingDataInit

MLHandTrackingSettingsInit

MLHandTrackingCreate

MLHandTrackingCreateEx

MLHandTrackingUpdateSettings

MLHandTrackingDestroy

MLHandTrackingGetData

MLHandTrackingGetStaticData

Source code