Skip to main content
Version: 17 Jan 2024

Perception

APIs for the Perception system. More...

Classes

Name
structMLCoordinateFrameUID
structMLPerceptionSettings
structMLTransformDerivatives
Velocity and acceleration derivatives for a related transform.
structMLSnapshotStaticData
Static information about the snapshot system.

Types

Name
typedef struct MLCoordinateFrameUIDMLCoordinateFrameUID
typedef struct MLPerceptionSettingsMLPerceptionSettings
typedef struct MLTransformDerivativesMLTransformDerivatives
Velocity and acceleration derivatives for a related transform.
typedef struct MLSnapshotStaticDataMLSnapshotStaticData
Static information about the snapshot system.
typedef struct MLSnapshotMLSnapshot

Enums

Name
enumAnonymous Enum 27
{
MLResultAPIPrefix_Snapshot = ( 0x87b8 << 16)
}
enumMLSnapshotResult
{
MLSnapshotResult_DerivativesNotCalculated = MLResultAPIPrefix_Snapshot,
MLSnapshotResult_Ensure32Bits = 0x7FFFFFFF
}

Functions

Name
MLResultMLPerceptionInitSettings(MLPerceptionSettings * out_settings)
Initializes the perception system with the passed in settings.
MLResultMLPerceptionStartup(MLPerceptionSettings * settings)
Starts the perception system.
MLResultMLPerceptionShutdown()
Shuts down the perception system and cleans up all resources used by it.
MLResultMLPerceptionGetSnapshot(MLSnapshot ** out_snapshot)
Pulls in the latest state of all persistent transforms and all enabled trackers extrapolated to the next frame time.
MLResultMLPerceptionGetPredictedSnapshot(MLTime timestamp, MLSnapshot ** out_snapshot)
Pulls in the state of all persistent transforms and all enabled trackers extrapolated to the provided timestamp.
MLResultMLPerceptionReleaseSnapshot(MLSnapshot * snap)
Releases specified #MLSnapshot object.
voidMLTransformDerivativesInit(MLTransformDerivatives * inout_data)
Intializes the default values for MLTransformDerivatives.
voidMLSnapshotStaticDataInit(MLSnapshotStaticData * inout_data)
Intializes the default values for MLSnapshotStaticData.
MLResultMLSnapshotGetStaticData(MLSnapshotStaticData * out_static_data)
Get the static data pertaining to the snapshot system.
MLResultMLSnapshotGetTransform(const MLSnapshot snapshot, const MLCoordinateFrameUID id, MLTransform * out_transform)
Get transform between world origin and the coordinate frame "id.
MLResultMLSnapshotGetTransformWithDerivatives(const MLSnapshot snapshot, const MLCoordinateFrameUID id, MLTransform out_transform, MLTransformDerivatives out_derivatives)
Get transform between world origin and the coordinate frame "id as well as any derivatives that have been calculated.
MLResultMLSnapshotGetPoseInBase(const MLSnapshot snapshot, const MLCoordinateFrameUID base_id, const MLCoordinateFrameUID id, MLPose out_pose)
Get transform between coordinate frame 'base_id' and the coordinate frame "id as well as any derivatives that have been calculated.
const char *MLSnapshotGetResultString(MLResult result_code)
Returns an ASCII string representation for each result code.

Detailed Description

APIs for the Perception system.

Shared Object:

  • perception.magicleap*

Enums Documentation

Anonymous Enum 27

EnumeratorValueDescription
MLResultAPIPrefix_Snapshot( 0x87b8 << 16)Defines the prefix for MLSnapshotResult codes.

MLSnapshotResult

EnumeratorValueDescription
MLSnapshotResult_DerivativesNotCalculatedMLResultAPIPrefix_SnapshotDerivatives not calculated for requested coordinate frame.
MLSnapshotResult_Ensure32Bits0x7FFFFFFFEnsure enum is represented as 32 bits.

Return values for Snapshot API calls.


Types Documentation

MLCoordinateFrameUID

typedef struct MLCoordinateFrameUID MLCoordinateFrameUID;

A unique identifier which represents a coordinate frame.

More Info


MLPerceptionSettings

typedef struct MLPerceptionSettings MLPerceptionSettings;

Settings for initializing the perception system.

More Info


MLTransformDerivatives

typedef struct MLTransformDerivatives MLTransformDerivatives;

Velocity and acceleration derivatives for a related transform.

More Info

API Level:

  • 8

MLSnapshotStaticData

typedef struct MLSnapshotStaticData MLSnapshotStaticData;

Static information about the snapshot system.

Initalize this structure with MLSnapshotStaticDataInit and populate with MLSnapshotGetStaticData().

More Info

API Level:

  • 30

MLSnapshot

typedef struct MLSnapshot MLSnapshot;

Opaque snapshot of system state.

More Info


Functions Documentation

MLPerceptionInitSettings

MLResult MLPerceptionInitSettings(
MLPerceptionSettings * out_settings
)

Initializes the perception system with the passed in settings.

Parameters

MLPerceptionSettings *out_settingsInitializes the perception system with these settings.

Returns

MLResultMLResult_InvalidParamFailed to initialize the perception settings due to an invalid input parameter.
MLResultMLResult_OkSuccessfully initialized the perception settings.
MLResultMLResult_UnspecifiedFailureFailed to initialize the perception settings due to an unknown error.

Required Permissions:

  • None

MLPerceptionStartup

MLResult MLPerceptionStartup(
MLPerceptionSettings * settings
)

Starts the perception system.

Parameters

MLPerceptionSettings *settingsThe perception system starts with these settings.

Returns

MLResultMLResult_OkSuccessfully started perception system.
MLResultMLResult_UnspecifiedFailureFailed to start perception system due to an unknown failure.

Required Permissions:

  • None

This function should be called before any perception functions are called.


MLPerceptionShutdown

MLResult MLPerceptionShutdown()

Shuts down the perception system and cleans up all resources used by it.

Returns

MLResultMLResult_OkSuccessfully shut down the perception system.
MLResultMLResult_UnspecifiedFailureFailed to shut down the perception system because of an uknown failure.

Required Permissions:

  • None

This function should be called prior to exiting the program if a call to [MLPerceptionStartup()](/docs/api-ref/api/Modules/group___perception/#mlresult-mlperceptionstartup) was called.


MLPerceptionGetSnapshot

MLResult MLPerceptionGetSnapshot(
MLSnapshot ** out_snapshot
)

Pulls in the latest state of all persistent transforms and all enabled trackers extrapolated to the next frame time.

Parameters

MLSnapshot **out_snapshotPointer to a pointer containing an MLSnapshot on success.

Returns

MLResultMLResult_OkSuccessfully created snapshot.
MLResultMLResult_InvalidParamout_snapshot parameter was not valid (null).
MLResultMLResult_PerceptionSystemNotStartedPerception System has not been started.

Required Permissions:

  • None

Returns a MLSnapshot with this latest state. This snap should be used for the duration of the frame being constructed and then released with a call to [MLPerceptionReleaseSnapshot()](/docs/api-ref/api/Modules/group___perception/#mlresult-mlperceptionreleasesnapshot).


MLPerceptionGetPredictedSnapshot

MLResult MLPerceptionGetPredictedSnapshot(
MLTime timestamp,
MLSnapshot ** out_snapshot
)

Pulls in the state of all persistent transforms and all enabled trackers extrapolated to the provided timestamp.

Parameters

MLTimetimestampTimestamp representing the time for which to predict poses.
MLSnapshot **out_snapshotPointer to a pointer containing an MLSnapshot on success.

Returns

MLResultMLResult_OkSuccessfully created snapshot.
MLResultMLResult_InvalidTimestampTimestamp is either more than 100ms in the future or too old for cached state.
MLResultMLResult_InvalidParamOutput parameter was not valid (null).
MLResultMLResult_PerceptionSystemNotStartedPerception System has not been started.

Required Permissions:

  • None

This timestamp typically comes from out_frame_info.predicted_display_time out parameter from the MLGraphicsBeginFrameEx function.

Returns a MLSnapshot with this latest state. This snap should be used for the duration of the frame being constructed and then released with a call to [MLPerceptionReleaseSnapshot()](/docs/api-ref/api/Modules/group___perception/#mlresult-mlperceptionreleasesnapshot).

API Level:

  • 27

MLPerceptionReleaseSnapshot

MLResult MLPerceptionReleaseSnapshot(
MLSnapshot * snap
)

Releases specified #MLSnapshot object.

Parameters

MLSnapshot *snapPointer to a valid snap object.

Returns

MLResultMLResult_OkSuccessfully released snapshot.
MLResultMLResult_InvalidParamsnapshot parameter was not valid (null).
MLResultMLResult_PerceptionSystemNotStartedPerception System has not been started.

Required Permissions:

  • None

This function should be called exactly once for each call to [MLPerceptionGetSnapshot()](/docs/api-ref/api/Modules/group___perception/#mlresult-mlperceptiongetsnapshot).


MLTransformDerivativesInit

static inline void MLTransformDerivativesInit(
MLTransformDerivatives * inout_data
)

Intializes the default values for MLTransformDerivatives.

Parameters

MLTransformDerivatives *inout_dataThe object that will be initialized with default values.

API Level:

  • 8

MLSnapshotStaticDataInit

static inline void MLSnapshotStaticDataInit(
MLSnapshotStaticData * inout_data
)

Intializes the default values for MLSnapshotStaticData.

Parameters

MLSnapshotStaticData *inout_dataThe object that will be initialized with default values.

API Level:

  • 30

MLSnapshotGetStaticData

MLResult MLSnapshotGetStaticData(
MLSnapshotStaticData * out_static_data
)

Get the static data pertaining to the snapshot system.

Parameters

MLSnapshotStaticData *out_static_dataTarget to populate the data about snapshot system.

Returns

MLResultMLResult_InvalidParamFailed to obtain static data due to invalid parameter.
MLResultMLResult_OkObtained static data successfully.
MLResultMLResult_UnspecifiedFailureFailed to obtain static data due to internal error.

Required Permissions:

  • None

API Level:

  • 30

MLSnapshotGetTransform

MLResult MLSnapshotGetTransform(
const MLSnapshot * snapshot,
const MLCoordinateFrameUID * id,
MLTransform * out_transform
)

Get transform between world origin and the coordinate frame "id.

Parameters

const MLSnapshot *snapshotA snapshot of tracker state. Can be obtained with MLPerceptionGetSnapshot().
const MLCoordinateFrameUID *idLook up the transform between the current origin and this coordinate frame id.
MLTransform *out_transformValid pointer to a MLTransform. To be filled out with requested transform data.

Returns

MLResultMLResult_InvalidParamFailed to obtain transform due to invalid parameter.
MLResultMLResult_OkObtained transform successfully.
MLResultMLResult_PoseNotFoundCoordinate Frame is valid, but not found in the current pose snapshot.
MLResultMLResult_UnspecifiedFailureFailed to obtain transform due to internal error.

Required Permissions:

  • None

MLSnapshotGetTransformWithDerivatives

MLResult MLSnapshotGetTransformWithDerivatives(
const MLSnapshot * snapshot,
const MLCoordinateFrameUID * id,
MLTransform * out_transform,
MLTransformDerivatives * out_derivatives
)

Get transform between world origin and the coordinate frame "id as well as any derivatives that have been calculated.

Parameters

const MLSnapshot *snapshotA snapshot of tracker state. Can be obtained with MLPerceptionGetSnapshot().
const MLCoordinateFrameUID *idLook up the transform between the current origin and this coordinate frame id.
MLTransform *out_transformValid pointer to a MLTransform. To be filled out with requested transform data.
MLTransformDerivatives *out_derivativesValid pointer to a MLTransformDerivatives. To be filled out with the derivatives of the transform if available.

Returns

MLResultMLResult_InvalidParamFailed to obtain transform due to invalid parameter.
MLResultMLResult_OkObtained transform successfully.
MLResultMLResult_PoseNotFoundCoordinate Frame is valid, but not found in the current pose snapshot.
MLResultMLResult_UnspecifiedFailureFailed to obtain transform due to internal error.
MLResultMLSnapshotResult_DerivativesNotCalculatedDerivatives are not available for the requested coordinate Frame.

Required Permissions:

  • None

API Level:

  • 8

MLSnapshotGetPoseInBase

MLResult MLSnapshotGetPoseInBase(
const MLSnapshot * snapshot,
const MLCoordinateFrameUID * base_id,
const MLCoordinateFrameUID * id,
MLPose * out_pose
)

Get transform between coordinate frame 'base_id' and the coordinate frame "id as well as any derivatives that have been calculated.

Parameters

const MLSnapshot *snapshotA snapshot of tracker state. Can be obtained with MLPerceptionGetSnapshot().
const MLCoordinateFrameUID *base_idThe coordinate frame in which to locate 'id'.
const MLCoordinateFrameUID *idThe coordinate frame which needs to be located in the base_id coordinate frame.
MLPose *out_poseValid pointer to a MLPose. To be filled out with requested pose data.

Returns

MLResultMLResult_InvalidParamFailed to obtain transform due to invalid parameter.
MLResultMLResult_OkObtained transform successfully.
MLResultMLResult_PoseNotFoundCoordinate Frame is valid, but not found in the current pose snapshot.
MLResultMLResult_UnspecifiedFailureFailed to obtain transform due to internal error.

Required Permissions:

  • None
// Get the CFUID for World Origin
MLSnapshotStaticData snapshot_static_data = {};
MLSnapshotStaticDataInit(&snapshot_static_data);
MLSnapshotGetStaticData(&snapshot_static_data);

// Get the CFUID for Headset Origin
MLHeadTrackingStaticData head_static_data_ = {};
MLHeadTrackingGetStaticData(head_tracker_, &head_static_data_);

// Get the CFUID for left eye
MLEyeTrackingStaticData eye_static_data_ = {};
MLEyeTrackingGetStaticData(eye_tracker_, &eye_static_data_);

MLPose left_eye_center = {};
MLSnapshot *snapshot = nullptr;
bool gaze_in_device_coordinate_frame = true;
MLPerceptionGetSnapshot(&snapshot);
if(gaze_in_device_coordinate_frame) { // Left eye gaze in device coordinate frame.
MLSnapshotGetPoseInBase(snapshot, &head_static_data_.coord_frame_head, &eye_static_data_.left_center, &left_eye_center);
}
else { // Left eye gaze in world origin coordinate frame.
MLSnapshotGetPoseInBase(snapshot, &snapshot_static_data.coord_world_origin, &eye_static_data_.left_center, &left_eye_center);
}
MLPerceptionReleaseSnapshot(snapshot);

API Level:

  • 30

MLSnapshotGetResultString

const char * MLSnapshotGetResultString(
MLResult result_code
)

Returns an ASCII string representation for each result code.

Parameters

MLResultresult_codeMLResult type to be converted to string.

Required Permissions:

  • None

This call returns strings for all of the global MLResult and MLSnapshotResult codes.

Return: ASCII string containing readable version of the result code.