Skip to main content
Version: 21 Aug 2024

ml_world_camera.h

Classes

Name
structMLWorldCameraSettings
A structure to encapsulate the camera settings.
structMLWorldCameraIntrinsics
World camera intrinsic parameters.
structMLWorldCameraFrameBuffer
A structure to encapsulate per plane info for each camera frame.
structMLWorldCameraFrame
A structure to encapsulate output data for each camera sensor.
structMLWorldCameraData
A structure to encapsulate output data for each camera sensor.

Types

Name
typedef struct MLWorldCameraSettingsMLWorldCameraSettings
A structure to encapsulate the camera settings.
typedef struct MLWorldCameraIntrinsicsMLWorldCameraIntrinsics
World camera intrinsic parameters.
typedef struct MLWorldCameraFrameBufferMLWorldCameraFrameBuffer
A structure to encapsulate per plane info for each camera frame.
typedef struct MLWorldCameraFrameMLWorldCameraFrame
A structure to encapsulate output data for each camera sensor.
typedef struct MLWorldCameraDataMLWorldCameraData
A structure to encapsulate output data for each camera sensor.

Enums

Name
enumAnonymous Enum 32
{
MLWorldCameraIntrinsics_MaxRadialDistortionCoefficients = 4,
MLWorldCameraIntrinsics_MaxTangentialDistortionCoefficients = 2
}
enumMLWorldCameraIdentifier
{
MLWorldCameraIdentifier_None = 0,
MLWorldCameraIdentifier_Left = 1 << 0,
MLWorldCameraIdentifier_Right = 1 << 1,
MLWorldCameraIdentifier_Center = 1 << 2,
MLWorldCameraIdentifier_All = MLWorldCameraIdentifier_Left
MLWorldCameraIdentifier_Right
MLWorldCameraIdentifier_Center,
MLWorldCameraIdentifier_Ensure32Bits = 0x7FFFFFFF
}
Camera Identifier.
enumMLWorldCameraMode
{
MLWorldCameraMode_Unknown = 0,
MLWorldCameraMode_LowExposure = 1 << 0,
MLWorldCameraMode_NormalExposure = 1 << 1,
MLWorldCameraMode_Ensure32Bits = 0x7FFFFFFF
}
Enumeration of world camera modes.
enumMLWorldCameraFrameType
{
MLWorldCameraFrameType_Unknown = 0,
MLWorldCameraFrameType_LowExposure = 1,
MLWorldCameraFrameType_NormalExposure = 2,
MLWorldCameraFrameType_Ensure32Bits = 0x7FFFFFFF
}
Enumeration of camera mode used when capturing a frame.

Functions

Name
voidMLWorldCameraSettingsInit(MLWorldCameraSettings * inout_handle)
Initialize the camera settings structure. Shall be called before calling MLWorldCameraConnect().
voidMLWorldCameraDataInit(MLWorldCameraData * inout_world_camera_data)
Initialize MLWorldCameraData with version.
MLResultMLWorldCameraConnect(const MLWorldCameraSettings settings, MLHandle out_handle)
Connect to world cameras.
MLResultMLWorldCameraUpdateSettings(MLHandle handle, const MLWorldCameraSettings * settings)
Update the world camera settings.
MLResultMLWorldCameraGetLatestWorldCameraData(MLHandle handle, uint64_t timeout_ms, MLWorldCameraData ** out_data)
Poll for Frames.
MLResultMLWorldCameraReleaseCameraData(MLHandle handle, MLWorldCameraData * world_camera_data)
Releases specified MLWorldCameraData object.
MLResultMLWorldCameraDisconnect(MLHandle handle)
Disconnect from world camera.

Enums Documentation

Anonymous Enum 32

EnumeratorValueDescription
MLWorldCameraIntrinsics_MaxRadialDistortionCoefficients4Default radial distortion vector size.
MLWorldCameraIntrinsics_MaxTangentialDistortionCoefficients2Default tangential distortion vector size.

MLWorldCameraIdentifier

EnumeratorValueDescription
MLWorldCameraIdentifier_None0None.
MLWorldCameraIdentifier_Left1 << 0Left World camera.
MLWorldCameraIdentifier_Right1 << 1Right World camera.
MLWorldCameraIdentifier_Center1 << 2Center World camera.
MLWorldCameraIdentifier_AllMLWorldCameraIdentifier_Left
MLWorldCameraIdentifier_Right
MLWorldCameraIdentifier_CenterAll World cameras.
MLWorldCameraIdentifier_Ensure32Bits0x7FFFFFFFEnsure enum is represented as 32 bits.

Camera Identifier.

Enumeration of all the available world camera sensors.


MLWorldCameraMode

EnumeratorValueDescription
MLWorldCameraMode_Unknown0None.
MLWorldCameraMode_LowExposure1 << 0Low exposure mode. This mode is currently only available when the controller is being tracked.
MLWorldCameraMode_NormalExposure1 << 1Normal exposure mode.
MLWorldCameraMode_Ensure32Bits0x7FFFFFFFEnsure enum is represented as 32 bits.

Enumeration of world camera modes.


MLWorldCameraFrameType

EnumeratorValueDescription
MLWorldCameraFrameType_Unknown0None.
MLWorldCameraFrameType_LowExposure1Frame captured using MLWorldCameraMode_LowExposure mode.
MLWorldCameraFrameType_NormalExposure2Frame captured using MLWorldCameraMode_NormalExposure mode.
MLWorldCameraFrameType_Ensure32Bits0x7FFFFFFFEnsure enum is represented as 32 bits.

Enumeration of camera mode used when capturing a frame.


Types Documentation

MLWorldCameraSettings

typedef struct MLWorldCameraSettings MLWorldCameraSettings;

A structure to encapsulate the camera settings.

This structure must be initialized by calling MLWorldCameraSettingsInit before use.

More Info

API Level:

  • 23

MLWorldCameraIntrinsics

typedef struct MLWorldCameraIntrinsics MLWorldCameraIntrinsics;

World camera intrinsic parameters.

More Info

API Level:

  • 23

MLWorldCameraFrameBuffer

typedef struct MLWorldCameraFrameBuffer MLWorldCameraFrameBuffer;

A structure to encapsulate per plane info for each camera frame.

More Info

API Level:

  • 23

MLWorldCameraFrame

typedef struct MLWorldCameraFrame MLWorldCameraFrame;

A structure to encapsulate output data for each camera sensor.

More Info

API Level:

  • 23

MLWorldCameraData

typedef struct MLWorldCameraData MLWorldCameraData;

A structure to encapsulate output data for each camera sensor.

This structure must be initialized by calling MLWorldCameraDataInit before use.

More Info

API Level:

  • 23

Functions Documentation

MLWorldCameraSettingsInit

static inline void MLWorldCameraSettingsInit(
MLWorldCameraSettings * inout_handle
)

Initialize the camera settings structure. Shall be called before calling MLWorldCameraConnect().

Parameters

MLWorldCameraSettings *inout_handleMLWorldCameraSettings structure to initialize.

Required Permissions:

  • None

API Level:

  • 23

MLWorldCameraDataInit

static inline void MLWorldCameraDataInit(
MLWorldCameraData * inout_world_camera_data
)

Initialize MLWorldCameraData with version.

Parameters

MLWorldCameraData *inout_world_camera_dataSet up the version for inout_world_camera_data.

API Level:

  • 23

MLWorldCameraConnect

MLResult MLWorldCameraConnect(
const MLWorldCameraSettings * settings,
MLHandle * out_handle
)

Connect to world cameras.

Parameters

const MLWorldCameraSettings *settingsA pointer to MLWorldCameraSettings structure.
MLHandle *out_handleA pointer to camera handle to be used in later APIs.

Returns

MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkConnected to camera device(s) successfully.
MLResultMLResult_PermissionDeniedNecessary permission is missing.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.

Required Permissions:

  • android.permission.CAMERA (protection level: dangerous)

API Level:

  • 23

MLWorldCameraUpdateSettings

MLResult MLWorldCameraUpdateSettings(
MLHandle handle,
const MLWorldCameraSettings * settings
)

Update the world camera settings.

Parameters

MLHandlehandleCamera handle obtained from MLWorldCameraConnect.
const MLWorldCameraSettings *settingsPointer to MLWorldCameraSettings.

Returns

MLResultMLResult_InvalidParamInvalid handle.
MLResultMLResult_OkSettings updated successfully.
MLResultMLResult_UnspecifiedFailureFailed due to internal error.

Required Permissions:

  • None

API Level:

  • 23

MLWorldCameraGetLatestWorldCameraData

MLResult MLWorldCameraGetLatestWorldCameraData(
MLHandle handle,
uint64_t timeout_ms,
MLWorldCameraData ** out_data
)

Poll for Frames.

Parameters

MLHandlehandleCamera handle obtained from MLWorldCameraConnect.
uint64_ttimeout_msTimeout in milliseconds.
MLWorldCameraData **out_dataWorld camera data. Will be set to NULL if no valid data is available at this time.

Returns

MLResultMLResult_InvalidParamInvalid handle.
MLResultMLResult_OkWorld camera data fetched successfully.
MLResultMLResult_TimeoutReturned because no new frame available at this time.
MLResultMLResult_UnspecifiedFailureFailed due to internal error.

Required Permissions:

  • None

Returns MLWorldCameraData with this latest data when available. The memory is owned by the system. Application should copy the data it needs to cache and release the memory by calling MLWorldCameraReleaseCameraData.

This is a blocking call. API is not thread safe.

If there are no new camera frames within the timeout_ms duration then the API will return MLResult_Timeout.

API Level:

  • 23

MLWorldCameraReleaseCameraData

MLResult MLWorldCameraReleaseCameraData(
MLHandle handle,
MLWorldCameraData * world_camera_data
)

Releases specified MLWorldCameraData object.

Parameters

MLHandlehandleCamera handle obtained from MLWorldCameraConnect.
MLWorldCameraData *world_camera_dataPointer to a valid MLWorldCameraData object.

Returns

MLResultMLResult_OkSuccessfully released world camera data.
MLResultMLResult_InvalidParamworld_camera_data parameter was not valid (NULL).
MLResultMLResult_UnspecifiedFailureFailed due to internal error.

Required Permissions:

  • None

This function should be called exactly once for each successfull call to #MLWorldCameraGetLatestCameraData.


MLWorldCameraDisconnect

MLResult MLWorldCameraDisconnect(
MLHandle handle
)

Disconnect from world camera.

Parameters

MLHandlehandleCamera handle obtained from MLWorldCameraConnect.

Returns

MLResultMLResult_InvalidParamInvalid handle.
MLResultMLResult_OkDisconnected camera successfully.
MLResultMLResult_UnspecifiedFailureFailed to disconnect camera.

Required Permissions:

  • None

This will disconnect from all the world camera currently connected.

API Level:

  • 23

Source code

// %BANNER_BEGIN%
// ---------------------------------------------------------------------
// %COPYRIGHT_BEGIN%
// Copyright (c) 2022 Magic Leap, Inc. All Rights Reserved.
// Use of this file is governed by the Software License Agreement,
// located here: https://www.magicleap.com/software-license-agreement-ml2
// Terms and conditions applicable to third-party materials accompanying
// this distribution may also be found in the top-level NOTICE file
// appearing herein.
// %COPYRIGHT_END%
// ---------------------------------------------------------------------
// %BANNER_END%

#pragma once

#include "ml_api.h"
#include "ml_types.h"

#include <string.h>

ML_EXTERN_C_BEGIN

enum {
MLWorldCameraIntrinsics_MaxRadialDistortionCoefficients = 4,
MLWorldCameraIntrinsics_MaxTangentialDistortionCoefficients = 2
};

typedef enum MLWorldCameraIdentifier{
MLWorldCameraIdentifier_None = 0,
MLWorldCameraIdentifier_Left = 1 << 0,
MLWorldCameraIdentifier_Right = 1 << 1,
MLWorldCameraIdentifier_Center = 1 << 2,
MLWorldCameraIdentifier_All = MLWorldCameraIdentifier_Left |
MLWorldCameraIdentifier_Right |
MLWorldCameraIdentifier_Center,
MLWorldCameraIdentifier_Ensure32Bits = 0x7FFFFFFF
} MLWorldCameraIdentifier;

typedef enum MLWorldCameraMode {
MLWorldCameraMode_Unknown = 0,
MLWorldCameraMode_LowExposure = 1 << 0,
MLWorldCameraMode_NormalExposure = 1 << 1,
MLWorldCameraMode_Ensure32Bits = 0x7FFFFFFF
} MLWorldCameraMode;

typedef enum MLWorldCameraFrameType {
MLWorldCameraFrameType_Unknown = 0,
MLWorldCameraFrameType_LowExposure = 1,
MLWorldCameraFrameType_NormalExposure = 2,
MLWorldCameraFrameType_Ensure32Bits = 0x7FFFFFFF
} MLWorldCameraFrameType;

typedef struct MLWorldCameraSettings {
uint32_t version;
uint32_t mode;
uint32_t cameras;
} MLWorldCameraSettings;

ML_STATIC_INLINE void MLWorldCameraSettingsInit(MLWorldCameraSettings *inout_handle) {
if (inout_handle) {
memset(inout_handle, 0, sizeof(MLWorldCameraSettings));
inout_handle->version = 1;
inout_handle->mode = MLWorldCameraMode_NormalExposure;
inout_handle->cameras = MLWorldCameraIdentifier_All;
}
}

typedef struct MLWorldCameraIntrinsics {
uint32_t width;
uint32_t height;
MLVec2f focal_length;
MLVec2f principal_point;
float fov;
double radial_distortion[MLWorldCameraIntrinsics_MaxRadialDistortionCoefficients];
double tangential_distortion[MLWorldCameraIntrinsics_MaxTangentialDistortionCoefficients];
} MLWorldCameraIntrinsics;

typedef struct MLWorldCameraFrameBuffer {
uint32_t width;
uint32_t height;
uint32_t stride;
uint32_t bytes_per_pixel;
uint32_t size;
uint8_t *data;
} MLWorldCameraFrameBuffer;


typedef struct MLWorldCameraFrame {
MLWorldCameraIdentifier id;
int64_t frame_number;
MLTime timestamp;
MLWorldCameraIntrinsics intrinsics;
MLTransform camera_pose;
MLWorldCameraFrameBuffer frame_buffer;
MLWorldCameraFrameType frame_type;
} MLWorldCameraFrame;


typedef struct MLWorldCameraData {
uint32_t version;
uint8_t frame_count;
MLWorldCameraFrame *frames;
} MLWorldCameraData;

ML_STATIC_INLINE void MLWorldCameraDataInit(MLWorldCameraData *inout_world_camera_data) {
if (inout_world_camera_data) {
memset(inout_world_camera_data, 0, sizeof(MLWorldCameraData));
inout_world_camera_data->version = 1;
}
}

ML_API MLResult ML_CALL MLWorldCameraConnect(const MLWorldCameraSettings *settings, MLHandle *out_handle);

ML_API MLResult ML_CALL MLWorldCameraUpdateSettings(MLHandle handle, const MLWorldCameraSettings *settings);

ML_API MLResult ML_CALL MLWorldCameraGetLatestWorldCameraData(MLHandle handle, uint64_t timeout_ms, MLWorldCameraData **out_data);

ML_API MLResult ML_CALL MLWorldCameraReleaseCameraData(MLHandle handle, MLWorldCameraData *world_camera_data);

ML_API MLResult ML_CALL MLWorldCameraDisconnect(MLHandle handle);

ML_EXTERN_C_END