Version: 20 Mar 2024

Perception

APIs for the Perception system. More...

Classes

	Name
struct	MLCoordinateFrameUID
struct	MLPerceptionSettings
struct	MLTransformDerivatives Velocity and acceleration derivatives for a related transform.
struct	MLSnapshotStaticData Static information about the snapshot system.

Types

	Name
typedef struct MLCoordinateFrameUID	MLCoordinateFrameUID
typedef struct MLPerceptionSettings	MLPerceptionSettings
typedef struct MLTransformDerivatives	MLTransformDerivatives Velocity and acceleration derivatives for a related transform.
typedef struct MLSnapshotStaticData	MLSnapshotStaticData Static information about the snapshot system.
typedef struct MLSnapshot	MLSnapshot

Enums

	Name
enum	Anonymous Enum 27 { MLResultAPIPrefix_Snapshot = ( 0x87b8 << 16) }
enum	MLSnapshotResult { MLSnapshotResult_DerivativesNotCalculated = MLResultAPIPrefix_Snapshot, MLSnapshotResult_Ensure32Bits = 0x7FFFFFFF }

Functions

	Name
MLResult	MLPerceptionInitSettings(MLPerceptionSettings * out_settings) Initializes the perception system with the passed in settings.
MLResult	MLPerceptionStartup(MLPerceptionSettings * settings) Starts the perception system.
MLResult	MLPerceptionShutdown() Shuts down the perception system and cleans up all resources used by it.
MLResult	MLPerceptionGetSnapshot(MLSnapshot ** out_snapshot) Pulls in the latest state of all persistent transforms and all enabled trackers extrapolated to the next frame time.
MLResult	MLPerceptionGetPredictedSnapshot(MLTime timestamp, MLSnapshot ** out_snapshot) Pulls in the state of all persistent transforms and all enabled trackers extrapolated to the provided timestamp.
MLResult	MLPerceptionReleaseSnapshot(MLSnapshot * snap) Releases specified `#MLSnapshot` object.
void	MLTransformDerivativesInit(MLTransformDerivatives * inout_data) Intializes the default values for MLTransformDerivatives.
void	MLSnapshotStaticDataInit(MLSnapshotStaticData * inout_data) Intializes the default values for MLSnapshotStaticData.
MLResult	MLSnapshotGetStaticData(MLSnapshotStaticData * out_static_data) Get the static data pertaining to the snapshot system.
MLResult	MLSnapshotGetTransform(const MLSnapshot snapshot, const MLCoordinateFrameUID id, MLTransform * out_transform) Get transform between world origin and the coordinate frame "id.
MLResult	MLSnapshotGetTransformWithDerivatives(const MLSnapshot snapshot, const MLCoordinateFrameUID id, MLTransform out_transform, MLTransformDerivatives out_derivatives) Get transform between world origin and the coordinate frame "id as well as any derivatives that have been calculated.
MLResult	MLSnapshotGetPoseInBase(const MLSnapshot snapshot, const MLCoordinateFrameUID base_id, const MLCoordinateFrameUID id, MLPose out_pose) Get transform between coordinate frame 'base_id' and the coordinate frame "id as well as any derivatives that have been calculated.
const char *	MLSnapshotGetResultString(MLResult result_code) Returns an ASCII string representation for each result code.

Detailed Description

APIs for the Perception system.

Shared Object:

perception.magicleap*

Enums Documentation

Anonymous Enum 27

Enumerator	Value	Description
MLResultAPIPrefix_Snapshot	( 0x87b8 << 16)	Defines the prefix for MLSnapshotResult codes.

MLSnapshotResult

Enumerator	Value	Description
MLSnapshotResult_DerivativesNotCalculated	MLResultAPIPrefix_Snapshot	Derivatives not calculated for requested coordinate frame.
MLSnapshotResult_Ensure32Bits	0x7FFFFFFF	Ensure enum is represented as 32 bits.

Return values for Snapshot API calls.

Types Documentation

MLCoordinateFrameUID

typedef struct MLCoordinateFrameUID MLCoordinateFrameUID;

A unique identifier which represents a coordinate frame.

More Info

MLPerceptionSettings

typedef struct MLPerceptionSettings MLPerceptionSettings;

Settings for initializing the perception system.

More Info

MLTransformDerivatives

typedef struct MLTransformDerivatives MLTransformDerivatives;

Velocity and acceleration derivatives for a related transform.

More Info

API Level:

MLSnapshotStaticData

typedef struct MLSnapshotStaticData MLSnapshotStaticData;

Static information about the snapshot system.

Initalize this structure with MLSnapshotStaticDataInit and populate with MLSnapshotGetStaticData().

More Info

API Level:

MLSnapshot

typedef struct MLSnapshot MLSnapshot;

Opaque snapshot of system state.

More Info

Functions Documentation

MLPerceptionInitSettings

MLResult MLPerceptionInitSettings(
    MLPerceptionSettings * out_settings
)

Initializes the perception system with the passed in settings.

Parameters


MLPerceptionSettings *	out_settings	Initializes the perception system with these settings.

Returns


MLResult	MLResult_InvalidParam	Failed to initialize the perception settings due to an invalid input parameter.
MLResult	MLResult_Ok	Successfully initialized the perception settings.
MLResult	MLResult_UnspecifiedFailure	Failed to initialize the perception settings due to an unknown error.

Required Permissions:

None

MLPerceptionStartup

MLResult MLPerceptionStartup(
    MLPerceptionSettings * settings
)

Starts the perception system.

Parameters


MLPerceptionSettings *	settings	The perception system starts with these settings.

Returns


MLResult	MLResult_Ok	Successfully started perception system.
MLResult	MLResult_UnspecifiedFailure	Failed to start perception system due to an unknown failure.

Required Permissions:

None

This function should be called before any perception functions are called.

MLPerceptionShutdown

MLResult MLPerceptionShutdown()

Shuts down the perception system and cleans up all resources used by it.

Returns


MLResult	MLResult_Ok	Successfully shut down the perception system.
MLResult	MLResult_UnspecifiedFailure	Failed to shut down the perception system because of an uknown failure.

Required Permissions:

None

This function should be called prior to exiting the program if a call to [MLPerceptionStartup()](/docs/api-ref/api/Modules/group___perception/#mlresult-mlperceptionstartup) was called.

MLPerceptionGetSnapshot

MLResult MLPerceptionGetSnapshot(
    MLSnapshot ** out_snapshot
)

Pulls in the latest state of all persistent transforms and all enabled trackers extrapolated to the next frame time.

Parameters


MLSnapshot **	out_snapshot	Pointer to a pointer containing an MLSnapshot on success.

Returns


MLResult	MLResult_Ok	Successfully created snapshot.
MLResult	MLResult_InvalidParam	out_snapshot parameter was not valid (null).
MLResult	MLResult_PerceptionSystemNotStarted	Perception System has not been started.

Required Permissions:

None

Returns a MLSnapshot with this latest state. This snap should be used for the duration of the frame being constructed and then released with a call to [MLPerceptionReleaseSnapshot()](/docs/api-ref/api/Modules/group___perception/#mlresult-mlperceptionreleasesnapshot).

MLPerceptionGetPredictedSnapshot

MLResult MLPerceptionGetPredictedSnapshot(
    MLTime timestamp,
    MLSnapshot ** out_snapshot
)

Pulls in the state of all persistent transforms and all enabled trackers extrapolated to the provided timestamp.

Parameters


MLTime	timestamp	Timestamp representing the time for which to predict poses.
MLSnapshot **	out_snapshot	Pointer to a pointer containing an MLSnapshot on success.

Returns


MLResult	MLResult_Ok	Successfully created snapshot.
MLResult	MLResult_InvalidTimestamp	Timestamp is either more than 100ms in the future or too old for cached state.
MLResult	MLResult_InvalidParam	Output parameter was not valid (null).
MLResult	MLResult_PerceptionSystemNotStarted	Perception System has not been started.

Required Permissions:

None

This timestamp typically comes from out_frame_info.predicted_display_time out parameter from the MLGraphicsBeginFrameEx function.

API Level:

MLPerceptionReleaseSnapshot

MLResult MLPerceptionReleaseSnapshot(
    MLSnapshot * snap
)

Releases specified #MLSnapshot object.

Parameters


MLSnapshot *	snap	Pointer to a valid snap object.

Returns


MLResult	MLResult_Ok	Successfully released snapshot.
MLResult	MLResult_InvalidParam	snapshot parameter was not valid (null).
MLResult	MLResult_PerceptionSystemNotStarted	Perception System has not been started.

Required Permissions:

None

This function should be called exactly once for each call to [MLPerceptionGetSnapshot()](/docs/api-ref/api/Modules/group___perception/#mlresult-mlperceptiongetsnapshot).

MLTransformDerivativesInit

static inline void MLTransformDerivativesInit(
    MLTransformDerivatives * inout_data
)

Intializes the default values for MLTransformDerivatives.

Parameters


MLTransformDerivatives *	inout_data	The object that will be initialized with default values.

API Level:

MLSnapshotStaticDataInit

static inline void MLSnapshotStaticDataInit(
    MLSnapshotStaticData * inout_data
)

Intializes the default values for MLSnapshotStaticData.

Parameters


MLSnapshotStaticData *	inout_data	The object that will be initialized with default values.

API Level:

MLSnapshotGetStaticData

MLResult MLSnapshotGetStaticData(
    MLSnapshotStaticData * out_static_data
)

Get the static data pertaining to the snapshot system.

Parameters


MLSnapshotStaticData *	out_static_data	Target to populate the data about snapshot system.

Returns


MLResult	MLResult_InvalidParam	Failed to obtain static data due to invalid parameter.
MLResult	MLResult_Ok	Obtained static data successfully.
MLResult	MLResult_UnspecifiedFailure	Failed to obtain static data due to internal error.

Required Permissions:

None

API Level:

MLSnapshotGetTransform

MLResult MLSnapshotGetTransform(
    const MLSnapshot * snapshot,
    const MLCoordinateFrameUID * id,
    MLTransform * out_transform
)

Get transform between world origin and the coordinate frame "id.

Parameters


const MLSnapshot *	snapshot	A snapshot of tracker state. Can be obtained with MLPerceptionGetSnapshot().
const MLCoordinateFrameUID *	id	Look up the transform between the current origin and this coordinate frame id.
MLTransform *	out_transform	Valid pointer to a MLTransform. To be filled out with requested transform data.

Returns


MLResult	MLResult_InvalidParam	Failed to obtain transform due to invalid parameter.
MLResult	MLResult_Ok	Obtained transform successfully.
MLResult	MLResult_PoseNotFound	Coordinate Frame is valid, but not found in the current pose snapshot.
MLResult	MLResult_UnspecifiedFailure	Failed to obtain transform due to internal error.

Required Permissions:

None

MLSnapshotGetTransformWithDerivatives

MLResult MLSnapshotGetTransformWithDerivatives(
    const MLSnapshot * snapshot,
    const MLCoordinateFrameUID * id,
    MLTransform * out_transform,
    MLTransformDerivatives * out_derivatives
)

Get transform between world origin and the coordinate frame "id as well as any derivatives that have been calculated.

Parameters


const MLSnapshot *	snapshot	A snapshot of tracker state. Can be obtained with MLPerceptionGetSnapshot().
const MLCoordinateFrameUID *	id	Look up the transform between the current origin and this coordinate frame id.
MLTransform *	out_transform	Valid pointer to a MLTransform. To be filled out with requested transform data.
MLTransformDerivatives *	out_derivatives	Valid pointer to a MLTransformDerivatives. To be filled out with the derivatives of the transform if available.

Returns


MLResult	MLResult_InvalidParam	Failed to obtain transform due to invalid parameter.
MLResult	MLResult_Ok	Obtained transform successfully.
MLResult	MLResult_PoseNotFound	Coordinate Frame is valid, but not found in the current pose snapshot.
MLResult	MLResult_UnspecifiedFailure	Failed to obtain transform due to internal error.
MLResult	MLSnapshotResult_DerivativesNotCalculated	Derivatives are not available for the requested coordinate Frame.

Required Permissions:

None

API Level:

MLSnapshotGetPoseInBase

MLResult MLSnapshotGetPoseInBase(
    const MLSnapshot * snapshot,
    const MLCoordinateFrameUID * base_id,
    const MLCoordinateFrameUID * id,
    MLPose * out_pose
)

Get transform between coordinate frame 'base_id' and the coordinate frame "id as well as any derivatives that have been calculated.

Parameters


const MLSnapshot *	snapshot	A snapshot of tracker state. Can be obtained with MLPerceptionGetSnapshot().
const MLCoordinateFrameUID *	base_id	The coordinate frame in which to locate 'id'.
const MLCoordinateFrameUID *	id	The coordinate frame which needs to be located in the base_id coordinate frame.
MLPose *	out_pose	Valid pointer to a MLPose. To be filled out with requested pose data.

Returns


MLResult	MLResult_InvalidParam	Failed to obtain transform due to invalid parameter.
MLResult	MLResult_Ok	Obtained transform successfully.
MLResult	MLResult_PoseNotFound	Coordinate Frame is valid, but not found in the current pose snapshot.
MLResult	MLResult_UnspecifiedFailure	Failed to obtain transform due to internal error.

Required Permissions:

None

// Get the CFUID for World Origin
MLSnapshotStaticData snapshot_static_data = {};
MLSnapshotStaticDataInit(&snapshot_static_data);
MLSnapshotGetStaticData(&snapshot_static_data);

// Get the CFUID for Headset Origin
MLHeadTrackingStaticData head_static_data_ = {};
MLHeadTrackingGetStaticData(head_tracker_, &head_static_data_);

// Get the CFUID for left eye
MLEyeTrackingStaticData eye_static_data_ = {};
MLEyeTrackingGetStaticData(eye_tracker_, &eye_static_data_);

MLPose left_eye_center = {};
MLSnapshot *snapshot = nullptr;
bool gaze_in_device_coordinate_frame = true;
MLPerceptionGetSnapshot(&snapshot);
if(gaze_in_device_coordinate_frame) {  // Left eye gaze in device coordinate frame.
  MLSnapshotGetPoseInBase(snapshot, &head_static_data_.coord_frame_head, &eye_static_data_.left_center, &left_eye_center);
}
else {  // Left eye gaze in world origin coordinate frame.
  MLSnapshotGetPoseInBase(snapshot, &snapshot_static_data.coord_world_origin, &eye_static_data_.left_center, &left_eye_center);
}
MLPerceptionReleaseSnapshot(snapshot);

API Level:

MLSnapshotGetResultString

const char * MLSnapshotGetResultString(
    MLResult result_code
)

Returns an ASCII string representation for each result code.

Parameters


MLResult	result_code	MLResult type to be converted to string.

Required Permissions:

None

This call returns strings for all of the global MLResult and MLSnapshotResult codes.

Return: ASCII string containing readable version of the result code.

Perception

Classes​

Types​

Enums​

Functions​

Detailed Description​

Enums Documentation​

Anonymous Enum 27​

MLSnapshotResult​

Types Documentation​

MLCoordinateFrameUID​

MLPerceptionSettings​

MLTransformDerivatives​

MLSnapshotStaticData​

MLSnapshot​

Functions Documentation​

MLPerceptionInitSettings​

MLPerceptionStartup​

MLPerceptionShutdown​

MLPerceptionGetSnapshot​

MLPerceptionGetPredictedSnapshot​

MLPerceptionReleaseSnapshot​

MLTransformDerivativesInit​

MLSnapshotStaticDataInit​

MLSnapshotGetStaticData​

MLSnapshotGetTransform​

MLSnapshotGetTransformWithDerivatives​

MLSnapshotGetPoseInBase​

MLSnapshotGetResultString​

Classes

Types

Enums

Functions

Detailed Description

Enums Documentation

Anonymous Enum 27

MLSnapshotResult

Types Documentation

MLCoordinateFrameUID

MLPerceptionSettings

MLTransformDerivatives

MLSnapshotStaticData

MLSnapshot

Functions Documentation

MLPerceptionInitSettings

MLPerceptionStartup

MLPerceptionShutdown

MLPerceptionGetSnapshot

MLPerceptionGetPredictedSnapshot

MLPerceptionReleaseSnapshot

MLTransformDerivativesInit

MLSnapshotStaticDataInit

MLSnapshotGetStaticData

MLSnapshotGetTransform

MLSnapshotGetTransformWithDerivatives

MLSnapshotGetPoseInBase

MLSnapshotGetResultString