Depth Camera
Module: Pixel Sensors
APIs to access the depth camera data. More...
Classes
Name | |
---|---|
struct | MLDepthCameraStreamConfig Structure to encapsulate the camera config for a specific stream. |
struct | MLDepthCameraSettings Structure to encapsulate the camera settings. |
struct | MLDepthCameraIntrinsics Depth camera intrinsic parameters. |
struct | MLDepthCameraFrameBuffer Structure to encapsulate per plane info for each camera frame. |
struct | MLDepthCameraFrame Structure to encapsulate output data for each camera sensor. |
struct | MLDepthCameraData Structure to encapsulate output data for each camera stream. |
struct | MLDepthCameraStreamCapability Structure to encapsulate a possible configuration for a single stream. Can be used to understand possible values for a specific #stream_configs element in MLDepthCameraSettings. |
struct | MLDepthCameraCapability Structure to encapsulate a possible set of streams configuration. Such set describes a possible way of setting #stream_configs in MLDepthCameraSettings and may hold multiple MLDepthCameraStreamCapability for different streams. |
struct | MLDepthCameraCapabilityList Structure to encapsulate a list of possible stream configurations. |
struct | MLDepthCameraCapabilityFilter Structure to encapsulate camera capabilities filtering. |
Types
Name | |
---|---|
typedef struct MLDepthCameraStreamConfig | MLDepthCameraStreamConfig Structure to encapsulate the camera config for a specific stream. |
typedef struct MLDepthCameraSettings | MLDepthCameraSettings Structure to encapsulate the camera settings. |
typedef struct MLDepthCameraIntrinsics | MLDepthCameraIntrinsics Depth camera intrinsic parameters. |
typedef struct MLDepthCameraFrameBuffer | MLDepthCameraFrameBuffer Structure to encapsulate per plane info for each camera frame. |
typedef MLDepthCameraFrameBuffer | MLDepthCameraDepthImage Depth image. |
typedef MLDepthCameraFrameBuffer | MLDepthCameraConfidenceBuffer Confidence score. |
typedef MLDepthCameraFrameBuffer | MLDepthCameraDepthFlagsBuffer Flags. |
typedef MLDepthCameraFrameBuffer | MLDepthCameraAmbientRawDepthImage Ambient raw depth image. |
typedef MLDepthCameraFrameBuffer | MLDepthCameraRawDepthImage Raw depth image. |
typedef struct MLDepthCameraFrame | MLDepthCameraFrame Structure to encapsulate output data for each camera sensor. |
typedef struct MLDepthCameraData | MLDepthCameraData Structure to encapsulate output data for each camera stream. |
typedef struct MLDepthCameraStreamCapability | MLDepthCameraStreamCapability Structure to encapsulate a possible configuration for a single stream. Can be used to understand possible values for a specific #stream_configs element in MLDepthCameraSettings. |
typedef struct MLDepthCameraCapability | MLDepthCameraCapability Structure to encapsulate a possible set of streams configuration. Such set describes a possible way of setting #stream_configs in MLDepthCameraSettings and may hold multiple MLDepthCameraStreamCapability for different streams. |
typedef struct MLDepthCameraCapabilityList | MLDepthCameraCapabilityList Structure to encapsulate a list of possible stream configurations. |
typedef struct MLDepthCameraCapabilityFilter | MLDepthCameraCapabilityFilter Structure to encapsulate camera capabilities filtering. |
Enums
Functions
Detailed Description
APIs to access the depth camera data.
\experimental This is an experimental API which may be modified or removed without any prior notice.
Using the depth camera may result in the following APIs returning stale data:
- Meshing (ml_meshing2.h).
- Planes (ml_planes.h).
Enums Documentation
Anonymous Enum 8
Enumerator | Value | Description |
---|---|---|
MLDepthCameraIntrinsics_MaxDistortionCoefficients | 5 | Default distortion vector size. |
MLDepthCameraStream
Enumerator | Value | Description |
---|---|---|
MLDepthCameraStream_None | 0 | None. |
MLDepthCameraStream_LongRange | 1 << 0 | Long range stream. Under normal operations long range stream has a maximum frequency of 5fps and a range from 1m up to 5m, in some cases this can go as far 7.5m. |
MLDepthCameraStream_ShortRange | 1 << 1 | Short range stream. Under normal operations short range stream has a maximum frequency of 60fps and a range from 0.2m up to 0.9m. |
MLDepthCameraStream_Ensure32Bits | 0x7FFFFFFF | Ensure enum is represented as 32 bits. |
Enumeration of depth camera streams.
API Level:
- 29
MLDepthCameraFrameType
Enumerator | Value | Description |
---|---|---|
MLDepthCameraFrameType_LongRange | 0 | Frame captured using MLDepthCameraStream_LongRange stream. |
MLDepthCameraFrameType_ShortRange | 1 | Frame captured using MLDepthCameraStream_ShortRange stream. |
MLDepthCameraFrameType_Count | 2 | |
MLDepthCameraFrameType_Ensure32Bits | 0x7FFFFFFF | Ensure enum is represented as 32 bits. |
Enumeration of camera stream used when capturing a frame.
API Level:
- 29
MLDepthCameraFrameRate
Enumerator | Value | Description |
---|---|---|
MLDepthCameraFrameRate_1FPS | 0 | |
MLDepthCameraFrameRate_5FPS | 1 | |
MLDepthCameraFrameRate_25FPS | 2 | |
MLDepthCameraFrameRate_30FPS | 3 | |
MLDepthCameraFrameRate_50FPS | 4 | |
MLDepthCameraFrameRate_60FPS | 5 | |
MLDepthCameraFrameRate_Ensure32Bits | 0x7FFFFFFF | Ensure enum is represented as 32 bits. |
Enumeration of possible frame rates.
API Level:
- 29
MLDepthCameraFlags
Enumerator | Value | Description |
---|---|---|
MLDepthCameraFlags_None | 0 | None. |
MLDepthCameraFlags_DepthImage | 1 << 0 | Enable MLDepthCameraDepthImage. See MLDepthCameraDepthImage for more details. |
MLDepthCameraFlags_Confidence | 1 << 1 | Enable MLDepthCameraConfidenceBuffer. See MLDepthCameraConfidenceBuffer for more details. |
MLDepthCameraFlags_DepthFlags | 1 << 2 | Enable MLDepthCameraDepthFlagsBuffer. See MLDepthCameraDepthFlagsBuffer for more details. |
MLDepthCameraFlags_AmbientRawDepthImage | 1 << 3 | Enable MLDepthCameraAmbientRawDepthImage. See MLDepthCameraAmbientRawDepthImage for more details. |
MLDepthCameraFlags_RawDepthImage | 1 << 4 | Enable MLDepthCameraRawDepthImage. See MLDepthCameraRawDepthImage for more details. |
MLDepthCameraFlags_Ensure32Bits | 0x7FFFFFFF | Ensure enum is represented as 32 bits. |
Enumeration of flags to select data requested from depth camera.
MLDepthCameraDepthFlags
Enumerator | Value | Description |
---|---|---|
MLDepthCameraDepthFlags_Valid | 0 << 0 | Valid pixel. Indicates that there is no additional flag data for this pixel. |
MLDepthCameraDepthFlags_Invalid | 1 << 0 | Invalid. This bit is set to one to indicate that one or more flags from below have been set. Depending on the use case the application can correlate the flag data and corresponding pixel data to determine how to handle the pixel data. |
MLDepthCameraDepthFlags_Saturated | 1 << 1 | Pixel saturated. The pixel intensity is either below the min or the max threshold value. |
MLDepthCameraDepthFlags_Inconsistent | 1 << 2 | Inconsistent data. Inconsistent data received when capturing frames. This can happen due to fast motion. |
MLDepthCameraDepthFlags_LowSignal | 1 << 3 | Low signal. Pixel has very low signal to noise ratio. One example of when this can happen is for pixels in far end of the range. |
MLDepthCameraDepthFlags_FlyingPixel | 1 << 4 | Flying pixel. This typically happens when there is step jump in the distance of adjoining pixels in the scene. Example: When you open a door looking into the room the edges along the door's edges can cause flying pixels. |
MLDepthCameraDepthFlags_Masked | 1 << 5 | Masked. If this bit is on it indicates that the corresponding pixel may not be within the illuminator's illumination cone. |
MLDepthCameraDepthFlags_SBI | 1 << 8 | SBI. This bit will be set when there is high noise. |
MLDepthCameraDepthFlags_StrayLight | 1 << 9 | Stray light. This could happen when there is another light source apart from the depth camera illuminator. This could also lead to MLDepthCameraDepthFlags_LowSignal. |
MLDepthCameraDepthFlags_ConnectedComponent | 1 << 10 | Connected component. If a small group of MLDepthCameraDepthFlags_Valid is surrounded by a set of MLDepthCameraDepthFlags_Invalid then this bit will be set to 1. |
MLDepthCameraDepthFlags_Ensure32Bits | 0x7FFFFFFF | Ensure enum is represented as 32 bits. |
Enumeration of flags to select data requested from depth camera.
Types Documentation
MLDepthCameraStreamConfig
typedef struct MLDepthCameraStreamConfig MLDepthCameraStreamConfig;
Structure to encapsulate the camera config for a specific stream.
API Level:
- 29
MLDepthCameraSettings
typedef struct MLDepthCameraSettings MLDepthCameraSettings;
Structure to encapsulate the camera settings.
This structure must be initialized by calling MLDepthCameraSettingsInit before use.
These are supported configurations: Streams Exposure limits [us] Frame rate [FPS] Long Range 250 - 2000 1, 5 Short Range 50 - 375 On 50Hz PLC: 5, 25, 50On 60Hz PLC: 5, 30, 60 Use MLDepthCameraGetCapabilities to get a list of available configurations. \note The system may not be able to service all the requested streams at any given time. This parameter is treated as a hint and data will be provided for the requested streams if available. \note The API only supports a single stream. In a future release support for multiple streams will be added. @par API Level:\n 29
MLDepthCameraIntrinsics
typedef struct MLDepthCameraIntrinsics MLDepthCameraIntrinsics;
Depth camera intrinsic parameters.
API Level:
- 29
MLDepthCameraFrameBuffer
typedef struct MLDepthCameraFrameBuffer MLDepthCameraFrameBuffer;
Structure to encapsulate per plane info for each camera frame.
API Level:
- 29
MLDepthCameraDepthImage
typedef MLDepthCameraFrameBuffer MLDepthCameraDepthImage;
Depth image.
Depth image stores the depth data from the depth camera. Depth is represented in meters and gives the radial distance of the real world location from the depth camera coordinate frame. See MLDepthCameraIntrinsics and MLDepthCameraData.camera_pose for more details.
Cast the MLDepthCameraFrameBuffer.data to float data type.
MLDepthCameraConfidenceBuffer
typedef MLDepthCameraFrameBuffer MLDepthCameraConfidenceBuffer;
Confidence score.
Confidence buffer stores the confidence values for each pixel in the MLDepthCameraDepthImage. The confidence score is derived from the sensor noise and it is not normalized. The higher the value the higher the confidence. Applications can determine what confidence threshold to use based on their use case.
Cast the MLDepthCameraFrameBuffer.data to float data type.
MLDepthCameraDepthFlagsBuffer
typedef MLDepthCameraFrameBuffer MLDepthCameraDepthFlagsBuffer;
Flags.
Each pixel in the MLDepthCameraDepthImage will have an associated set of flags.
Cast the MLDepthCameraFrameBuffer.data to uint32_t data type.
MLDepthCameraAmbientRawDepthImage
typedef MLDepthCameraFrameBuffer MLDepthCameraAmbientRawDepthImage;
Ambient raw depth image.
This is the raw depth camera sensor data captured without the depth camera illumination and corresponds to the amount of ambient light incident on the sensor.
Cast the MLDepthCameraFrameBuffer.data to float data type.
MLDepthCameraRawDepthImage
typedef MLDepthCameraFrameBuffer MLDepthCameraRawDepthImage;
Raw depth image.
This is the raw depth camera sensor data captured with the depth camera illumination and corresponds to the amount of total light incident on the sensor.
Cast the MLDepthCameraFrameBuffer.data to float data type.
MLDepthCameraFrame
typedef struct MLDepthCameraFrame MLDepthCameraFrame;
Structure to encapsulate output data for each camera sensor.
API Level:
- 29
MLDepthCameraData
typedef struct MLDepthCameraData MLDepthCameraData;
Structure to encapsulate output data for each camera stream.
This structure must be initialized by calling MLDepthCameraDataInit before use.
API Level:
- 29
MLDepthCameraStreamCapability
typedef struct MLDepthCameraStreamCapability MLDepthCameraStreamCapability;
Structure to encapsulate a possible configuration for a single stream. Can be used to understand possible values for a specific #stream_configs element in MLDepthCameraSettings.
The capabilities supported by the depth camera can be queried with MLDepthCameraGetCapabilities().
API Level:
- 29
MLDepthCameraCapability
typedef struct MLDepthCameraCapability MLDepthCameraCapability;
Structure to encapsulate a possible set of streams configuration. Such set describes a possible way of setting #stream_configs in MLDepthCameraSettings and may hold multiple MLDepthCameraStreamCapability for different streams.
Whole #stream_capabilities array describe a single configuration. Using only part of it is not recommended.
The capabilities supported by the depth camera can be queried with MLDepthCameraGetCapabilities().
API Level:
- 29
MLDepthCameraCapabilityList
typedef struct MLDepthCameraCapabilityList MLDepthCameraCapabilityList;
Structure to encapsulate a list of possible stream configurations.
User should be able to use any of the capabilities from the list. They are grouped in usable sets of configurations - one set may contain different MLDepthCameraStreamCapability for different streams. For more information see MLDepthCameraCapability.
API Level:
- 29
MLDepthCameraCapabilityFilter
typedef struct MLDepthCameraCapabilityFilter MLDepthCameraCapabilityFilter;
Structure to encapsulate camera capabilities filtering.
The capabilities supported by the depth camera can be queried with MLDepthCameraGetCapabilities().
API Level:
- 29
Functions Documentation
MLDepthCameraSettingsInit
static inline void MLDepthCameraSettingsInit(
MLDepthCameraSettings * inout_handle
)
Initialize the connect handle structure. Shall be called before calling MLDepthCameraConnect().
Parameters
MLDepthCameraSettings * | inout_handle | MLDepthCameraSettings structure to initialize. |
Required Permissions:
- None
API Level:
- 29
MLDepthCameraDataInit
static inline void MLDepthCameraDataInit(
MLDepthCameraData * inout_depth_camera_data
)
Initialize MLDepthCameraData with version.
Parameters
MLDepthCameraData * | inout_depth_camera_data | Set up the version for inout_depth_camera_data. |
API Level:
- 29
MLDepthCameraCapabilityFilterInit
static inline void MLDepthCameraCapabilityFilterInit(
MLDepthCameraCapabilityFilter * inout_handle
)
Initialize the capabilities filtering structure. Shall be called before calling MLDepthCameraGetCapabilities().
Parameters
MLDepthCameraCapabilityFilter * | inout_handle | MLDepthCameraCapabilityFilter structure to initialize. |
Required Permissions:
- None
API Level:
- 29
MLDepthCameraConnect
MLResult MLDepthCameraConnect(
const MLDepthCameraSettings * settings,
MLHandle * out_handle
)
Connect to depth camera.
Parameters
const MLDepthCameraSettings * | settings | A pointer to MLDepthCameraSettings structure. |
MLHandle * | out_handle | A pointer to camera handle to be used in later APIs. |
Returns
MLResult | MLResult_InvalidParam | One of the parameters is invalid. |
MLResult | MLResult_Ok | Connected to camera device(s) successfully. |
MLResult | MLResult_PermissionDenied | Necessary permission is missing. |
MLResult | MLResult_UnspecifiedFailure | The operation failed with an unspecified error. |
Required Permissions:
- com.magicleap.permission.DEPTH_CAMERA (protection level: dangerous)
API Level:
- 29
MLDepthCameraUpdateSettings
MLResult MLDepthCameraUpdateSettings(
MLHandle handle,
const MLDepthCameraSettings * settings
)
Update the depth camera settings.
Parameters
MLHandle | handle | Camera handle obtained from MLDepthCameraConnect. |
const MLDepthCameraSettings * | settings | Pointer to MLDepthCameraSettings. |
Returns
MLResult | MLResult_InvalidParam | Invalid handle. |
MLResult | MLResult_Ok | Settings updated successfully. |
MLResult | MLResult_UnspecifiedFailure | Failed due to internal error. |
Required Permissions:
- None
API Level:
- 29
MLDepthCameraGetCapabilities
MLResult MLDepthCameraGetCapabilities(
MLHandle handle,
const MLDepthCameraCapabilityFilter * filter,
MLDepthCameraCapabilityList * out_caps
)
Query the depth camera stream capabilities.
Parameters
MLHandle | handle | Camera handle obtained from MLDepthCameraConnect. |
const MLDepthCameraCapabilityFilter * | filter | Pointer to initialized MLDepthCameraCapabilityFilter structure. Used for filtering. |
MLDepthCameraCapabilityList * | out_caps | Pointer to initialized MLDepthCameraCapabilityList structure. |
Returns
MLResult | MLResult_InvalidParam | One of the parameters is invalid. |
MLResult | MLResult_Ok | API call completed successfully. |
Required Permissions:
- None
Retrieves the capabilities supported by the depth camera based on the provided filter. Overview of the capabilities can be seen in the [configuration table].
After each call, a corresponding call to MLDepthCameraReleaseCapabilities should be made to release allocated resources.
API Level:
- 29
MLDepthCameraReleaseCapabilities
MLResult MLDepthCameraReleaseCapabilities(
MLHandle handle,
MLDepthCameraCapabilityList * out_caps
)
Release resources allocated with MLDepthCameraGetCapabilities.
Parameters
MLHandle | handle | Camera handle obtained from MLDepthCameraConnect. |
MLDepthCameraCapabilityList * | out_caps | Pointer to MLDepthCameraCapabilityList filled by the call to MLDepthCameraGetCapabilities. |
Returns
MLResult | MLResult_InvalidParam | One of the parameters is invalid. |
MLResult | MLResult_Ok | API call completed successfully. |
Required Permissions:
- None
API Level:
- 29
MLDepthCameraGetLatestDepthData
MLResult MLDepthCameraGetLatestDepthData(
MLHandle handle,
uint64_t timeout_ms,
MLDepthCameraData * out_data
)
Poll for Frames.
Parameters
MLHandle | handle | Camera handle obtained from MLDepthCameraConnect. |
uint64_t | timeout_ms | Timeout in milliseconds. |
MLDepthCameraData * | out_data | Depth camera data. Should be an initialized MLDepthCameraData object. |
Returns
MLResult | MLResult_InvalidParam | Invalid handle. |
MLResult | MLResult_Ok | Depth camera data fetched successfully. |
MLResult | MLResult_Timeout | Returned because no new frame available at this time. |
MLResult | MLResult_UnspecifiedFailure | Failed due to internal error. |
Required Permissions:
- None
Returns MLDepthCameraData with this latest data when available. The memory is owned by the system. Application should copy the data it needs to cache it and then release the memory by calling #MLDepthCameraReleaseCameraData.
This is a blocking call. API is not thread safe.
If there are no new depth data frames within the timeout_ms duration then the API will return MLResult_Timeout.
API Level:
- 29
MLDepthCameraReleaseDepthData
MLResult MLDepthCameraReleaseDepthData(
MLHandle handle,
MLDepthCameraData * depth_camera_data
)
Releases specified MLDepthCameraData object.
Parameters
MLHandle | handle | Camera handle obtained from MLDepthCameraConnect. |
MLDepthCameraData * | depth_camera_data | Pointer to a valid MLDepthCameraData object. |
Returns
MLResult | MLResult_Ok | Successfully released depth camera data. |
MLResult | MLResult_InvalidParam | depth_camera_data parameter was not valid (NULL). |
MLResult | MLResult_UnspecifiedFailure | Failed due to internal error. |
Required Permissions:
- None
This function should be called exactly once for each successfull call to #MLDepthCameraGetLatestCameraData.
MLDepthCameraDisconnect
MLResult MLDepthCameraDisconnect(
MLHandle handle
)
Disconnect from depth camera.
Parameters
MLHandle | handle | Camera handle obtained from MLDepthCameraConnect. |
Returns
MLResult | MLResult_InvalidParam | Invalid handle. |
MLResult | MLResult_Ok | Disconnected camera successfully. |
MLResult | MLResult_UnspecifiedFailure | Failed to disconnect camera. |
Required Permissions:
- None
This will disconnect from the depth camera.
API Level:
- 29