MLCameraBase

NameSpace: MagicLeap

MLCamera class exposes static functions to query camera related functions. Most functions are currently a direct pass through functions to the native C-API functions and incur no overhead. More...

Inherits from:
MLAPIBase

Inherited by:
UnityEngine.XR.MagicLeap.MLCamera

Detailed Description

public class MLCameraBase : MLAPIBase 

Details*

APIs for accessing Camera Device and to do Camera Capture.

---

Public Fields

OnDeviceAvailable

Camera status callback, device available.

public static MLCamera.OnDeviceAvailabilityStatusDelegate OnDeviceAvailable { get; set; }

---

OnDeviceUnavailable

Camera status callback, device unavailable.

public static MLCamera.OnDeviceAvailabilityStatusDelegate OnDeviceUnavailable { get; set; }

---

Public Methods

MLResult CaptureImage

Capture still image. Result will be availble via the MLCamera.OnRawImageAvailable event.

public MLResult CaptureImage(
    uint numImages =1
)

Parameters

Type	Name	Description
uint	numImages	no of images to capture valid range is 1-10

---

MLResult CapturePreviewStart

Start preview provide raw frames through callback

public MLResult CapturePreviewStart()

---

MLResult CapturePreviewStop

Stop preview.

public MLResult CapturePreviewStop()

Returns:

---

MLResult CaptureVideoStart

Start video capture. capture either encoded video or YUV/RGBA frames. YUV/RGBA frames are provides through callback. For encoded video capture valid MediaRecorder object should be configured in the PrepareCapture() call. The captured video YUV/RGBA frames will be returned via the OnRawVideoFrameAvailable and OnRawVideoFrameAvailable_NativeCallbackThread events.

public MLResult CaptureVideoStart()

---

MLResult CaptureVideoStop

Stop video capture. Should only be called at least 500ms after a CaptureVideoStart().

public MLResult CaptureVideoStop()

---

void Cleanup

public void Cleanup()

---

void ClearPreviewTexture

public void ClearPreviewTexture()

---

MLResult Connect

public MLResult Connect(
    MLCamera.ConnectContext connectContext
)

Parameters

Type	Name	Description
MLCamera.ConnectContext	connectContext	Camera connection settings.

---

MLCameraBase Create

public static MLCameraBase Create()

---

MLResult Disconnect

Disconnect the current camera. Should call MLCamera.Connect() again on this instance or create a new MLCamera object using MLCamera.CreateAndConnect() function.

public MLResult Disconnect()

---

void FlipFrameVertically

public static void FlipFrameVertically(
    ref CameraOutput frame
)

Parameters

Type	Name	Description
ref CameraOutput	frame	Contains the information and data of each of the available buffers/planes in a capture Number of planes for JPEG and RGBA captures = 1 Number of planes for YUV captures = 3

---

MLResult GetCameraCharacteristics

Provides an MLCamera.Metadata object to configure the camera settings.

public MLResult GetCameraCharacteristics(
    out MLCamera.Metadata cameraMetadata
)

Parameters

Type	Name	Description
out MLCamera.Metadata	cameraMetadata	MLCameraMetadata Summary placeholder.

---

MLResult GetDeviceAvailabilityStatus

Check availablity of a given camera device. MLCamera instance should only be created & connected when the device is available.

public static MLResult GetDeviceAvailabilityStatus(
    MLCamera.Identifier camId,
    out bool deviceAvailable
)

Parameters

Type	Name	Description
MLCamera.Identifier	camId	Camera id to check availability for
out bool	deviceAvailable	True if device is available, false otherwise

Returns:

---

MLResult GetDeviceStatus

Poll camera device status. Use DeviceStatusFlag to view specific status bit. Call GetErrorCode(out ErrorType) to obtain the error code if DeviceStatusFlag.Error bit is set.

public MLResult GetDeviceStatus(
    out MLCamera.DeviceStatusFlag status
)

Parameters

Type	Name	Description
out MLCamera.DeviceStatusFlag	status

Returns:

---

MLResult GetErrorCode

Obtain device error code.

public MLResult GetErrorCode(
    out MLCamera.ErrorType error
)

Parameters

Type	Name	Description
out MLCamera.ErrorType	error	The ErrorType which triggered DeviceStatusFlag.Error

Returns:

---

StreamCapability [] GetImageStreamCapabilitiesForCamera

public static StreamCapability [] GetImageStreamCapabilitiesForCamera(
    MLCamera camera,
    params CaptureType [] captureTypes
)

Parameters

Type	Name	Description
MLCamera	camera	Provides a multithreadig implementation of the camera api. Handles pause behavior automatically.
params CaptureType []	captureTypes	Capture operation type

---

MLResult GetStreamCapabilities

Get the StreamCapabilitiesInfo for the connected camera, which contains the stream capabilities for each supported capture stream.

public MLResult GetStreamCapabilities(
    out MLCamera.StreamCapabilitiesInfo [] streamCapabilities
)

Parameters

Type	Name	Description
out MLCamera.StreamCapabilitiesInfo []	streamCapabilities	Each item in the array refers to a possible capture stream. Every index within [StreamCapabilitiesInfo.StreamCapabilities] represents a StreamCapability of that particular stream.

Details*

One camera can support multiple capture streams and each stream has it's own set of StreamCapabilities (capture types and resolutions).

---

bool IsCaptureTypeSupported

public static bool IsCaptureTypeSupported(
    MLCamera camera,
    CaptureType captureType
)

Parameters

Type	Name	Description
MLCamera	camera	Provides a multithreadig implementation of the camera api. Handles pause behavior automatically.
CaptureType	captureType	Capture operation type

---

delegate void OnCaptureAbortedDelegate

Delegate to notify the app when a capture request is aborted.

public delegate void OnCaptureAbortedDelegate()

---

delegate void OnCaptureCompletedDelegate

Delegate to notify the app when a capture request is completed.

public delegate void OnCaptureCompletedDelegate(
    Metadata metadataHandle,
    ResultExtras extra
)

Parameters

Type	Name	Description
Metadata	metadataHandle	Handle to metadata of captured frame.
ResultExtras	extra	Carries capture result information of current captured frame.

---

delegate void OnCaptureFailedDelegate

Delegate to notify the app when a capture request fails.

public delegate void OnCaptureFailedDelegate(
    ResultExtras extra
)

Parameters

Type	Name	Description
ResultExtras	extra	Carries capture result information of current captured frame.

---

delegate void OnCapturedFrameAvailableDelegate

Delegate to notify the app when the frame data of a capture is available.

public delegate void OnCapturedFrameAvailableDelegate(
    CameraOutput frameInfo,
    ResultExtras resultExtras,
    Metadata metadataHandle
)

Parameters

Type	Name	Description
CameraOutput	frameInfo	Frame data
ResultExtras	extra	Carries capture result information of current captured frame.
Metadata	metadataHandle	MLCameraMetadata Summary placeholder.

---

delegate void OnDeviceAvailabilityStatusDelegate

public delegate void OnDeviceAvailabilityStatusDelegate(
    Identifier camId
)

Parameters

Type	Name	Description
Identifier	camId	Logical Camera identifiers available for access.

---

delegate void OnDeviceDisconnectedDelegate

Delegate to notify the app when the camera device disconnects.

public delegate void OnDeviceDisconnectedDelegate(
    DisconnectReason reason
)

Parameters

Type	Name	Description
DisconnectReason	reason	Disconnect reason

---

delegate void OnDeviceErrorDelegate

Delegate to notify the app about camera device error

public delegate void OnDeviceErrorDelegate(
    ErrorType error
)

Parameters

Type	Name	Description
ErrorType	error	Error code

---

delegate void OnDeviceStatusDelegate

Delegate to notify the app about camera device status (idle or streaming)

public delegate void OnDeviceStatusDelegate()

---

delegate void OnPreviewBufferAvailableDelegate

Delegate to notify the app when the frame data of a preview is available.

public delegate void OnPreviewBufferAvailableDelegate(
    Metadata metadataHandle,
    ResultExtras extra
)

Parameters

Type	Name	Description
Metadata	metadataHandle	Handle to metadata of captured frame.
ResultExtras	extra	Carries capture result information of current captured frame.

---

delegate void OnPreviewCaptureCompletedDelegate

Delegate to notify the app when the result of a preview capture is available.

public delegate void OnPreviewCaptureCompletedDelegate(
    MLResult result
)

Parameters

Type	Name	Description
MLResult	result	Result reporting whether the preview capture completed or not.

---

MLResult PreCaptureAEAWB

Trigger AEAWB Convergence. This API triggers AEAWB sequence and returns when AEAWB convergence has been achieved. The users can either trigger and converge AEAWB metadata settings or use this API.

public MLResult PreCaptureAEAWB()

---

MLResult PrepareCapture

Prepare for capture.

public MLResult PrepareCapture(
    MLCamera.CaptureConfig captureConfig,
    out MLCamera.Metadata cameraMetadata
)

Parameters

Type	Name	Description
MLCamera.CaptureConfig	captureConfig	Capture configuration.
out MLCamera.Metadata	cameraMetadata	Camera metadata object to configure the camera for captures.

---

bool TryGetBestFitStreamCapabilityFromCollection

public static bool TryGetBestFitStreamCapabilityFromCollection(
    StreamCapability [] streamCapabilities,
    int width,
    int height,
    CaptureType captureType,
    out StreamCapability streamCapability
)

Parameters

Type	Name	Description
StreamCapability []	streamCapabilities
int	width
int	height
CaptureType	captureType	Capture operation type
out StreamCapability	streamCapability

---

MLResult Uninitialize

public static MLResult Uninitialize()

---

MLResult UpdateCaptureSettings

Update capture setting This API updates the capture settings for an active preview/video stream. The API can be used to update capture settings like exposure compensation manual exposure settings like exposure time, iso setting. The capture settings can be changed in MLCamera.Metadata object returned by MLCamera.PrepareCapture() function and then this API can be called to apply the capture settings.

public MLResult UpdateCaptureSettings()

---

Protected Methods

CreatePreviewTexture

Create a preview texture.

protected void CreatePreviewTexture()

---

GLPluginEvent

Issues an event when a render event has occurred.

protected void GLPluginEvent()

---

HandleOnRawVideoFrameAvailableInternal

Handles the event of a new image getting captured.

protected void HandleOnRawVideoFrameAvailableInternal(
    MLCamera.CameraOutput capturedFrame,
    MLCamera.ResultExtras resultExtras,
    MLCamera.Metadata metadataHandle
)

Parameters

Type	Name	Description
MLCamera.CameraOutput	imageData	The raw data of the image.
MLCamera.ResultExtras	resultExtras
MLCamera.Metadata	metadataHandle	MLCameraMetadata Summary placeholder.

---

InternalCheckCameraPermission

protected static MLResult.Code InternalCheckCameraPermission()

---

InternalConnect

protected MLResult.Code InternalConnect(
    MLCamera.ConnectContext cameraConnectContext
)

Parameters

Type	Name	Description
MLCamera.ConnectContext	cameraConnectContext	Camera connection settings.

---

InternalDisconnect

Disconnect the camera.

protected MLResult.Code InternalDisconnect(
    bool flagsOnly =false,
    bool isAsync =false
)

Parameters

Type	Name	Description
bool	flagsOnly	Set to true if you only need to disconect the camera and not stop any ongoing captuers. Useful when disconnecting due to internal errors. False by default.
bool	isAsync

Returns: MLResult.Result will be MLResult.Code.Ok if completed successfully. MLResult.Result will be MLResult.Code.UnspecifiedFailure if failed due to internal error. MLResult.Result will be MLResult.Code.PermissionDenied if a required permission is missing.

---

InternalGetStreamCapabilities

protected MLResult InternalGetStreamCapabilities(
    out MLCamera.StreamCapabilitiesInfo [] streamCapabilities
)

Parameters

Type	Name	Description
out MLCamera.StreamCapabilitiesInfo []	streamCapabilities

---

InternalInitialize

Establish a connection to the camera.

protected static MLResult InternalInitialize()

Returns: MLResult.Result will be MLResult.Code.Ok if connected to camera device successfully. MLResult.Result will be MLResult.Code.MediaGenericUnexpectedNull if failed to connect to camera device due to null pointer. MLResult.Result will be MLResult.Code.Timeout if failed connecting the camera due to timeout. MLResult.Result will be MLResult.Code.UnspecifiedFailure if failed due to internal error. MLResult.Result will be MLResult.Code.AllocFailed if failed to allocate memory. MLResult.Result will be MLResult.Code.PermissionDenied if a required permission is missing.

---

InternalPrepareCapture

protected MLResult.Code InternalPrepareCapture(
    MLCamera.CaptureConfig captureConfig,
    out MLCamera.Metadata cameraMetadata
)

Parameters

Type	Name	Description
MLCamera.CaptureConfig	captureConfig
out MLCamera.Metadata	cameraMetadata	MLCameraMetadata Summary placeholder.

---

InternalUninitialize

protected static MLResult InternalUninitialize()

---

MLCameraBase

protected MLCameraBase()

---

OnApplicationPause

Callback sent to all MagicLeap APIs on application pause.

protected virtual override void OnApplicationPause(
    bool pauseStatus
)

Parameters

Type	Name	Description
bool	pauseStatus	True if the application is paused, else False.

Reimplements: OnApplicationPause

---

Public Attributes

ConnectionContext

The connection context used to create this MLCamera object.

public MLCamera.ConnectContext ConnectionContext => cameraConnectContext;

Type	Description
MLCamera.ConnectContext	Camera connection settings.

---

ConnectionEstablished

Connection status of the camera.

public bool ConnectionEstablished => cameraConnectionEstablished;

---

CurrentFPS

public float CurrentFPS => currentFPS;

---

PreviewTexture

A reference to Camera Preview texture.

public RenderTexture PreviewTexture => previewTexture;

---

Protected Attributes

byteArrays

protected byte [][] byteArrays;

---

cameraCaptureConfig

Last used camera Capture Config

protected MLCamera.CaptureConfig cameraCaptureConfig;

---

cameraConnectContext

camera Connect Context

protected MLCamera.ConnectContext cameraConnectContext;

Type	Description
MLCamera.ConnectContext	Camera connection settings.

---

cameraConnectionEstablished

Connection status of the camera.

protected bool cameraConnectionEstablished = false;

---

cameraInited

protected static bool cameraInited = false;

---

currentFPS

Calculated FPS based on recevied frames interval.

protected float currentFPS;

---

gcHandle

GC Handle.

protected GCHandle gcHandle;

---

internalOnDeviceAvailable

protected static MLCamera.OnDeviceAvailabilityStatusDelegate internalOnDeviceAvailable;

---

internalOnDeviceUnavailable

protected static MLCamera.OnDeviceAvailabilityStatusDelegate internalOnDeviceUnavailable;

---

isCapturingPreview

Gets a value indicating whether the camera preview is enabled and textures are being rendered.

protected bool isCapturingPreview = false;

---

isCapturingVideo

Capture status of the camera.

protected bool isCapturingVideo = false;

---

previewRenderer

Preview Renderer;

protected MLCamera.Renderer previewRenderer;

---

previewTexture

A reference to Camera Preview texture.

protected RenderTexture previewTexture;

---

previewTexture2D

The preview texture for the camera captures.

protected Texture2D previewTexture2D = null;

---

previousCaptureTimestamp

Timestamp of previously captured frame. needed for FPS calculations.

protected long previousCaptureTimestamp;

---

rawVideoFrameQueueLockObject

A raw video queue lock object.

protected static object rawVideoFrameQueueLockObject = new object();

---

resumeConnect

The resume connect state of the camera.

protected bool resumeConnect = false;

---

resumePreviewCapture

The resume preview state of the camera.

protected bool resumePreviewCapture = false;

---

resumeVideoCapture

The resume raw video capture state of the camera.

protected bool resumeVideoCapture = false;

---

Public Events

OnCaptureAborted

Callback is invoked when an ongoing video or preview capture or both are aborteddue to an error.

public MLCamera.OnCaptureAbortedDelegate OnCaptureAborted()

---

OnCaptureCompleted

Callback is invoked when capturing single frame is completed and result is available.

public MLCamera.OnCaptureCompletedDelegate OnCaptureCompleted()

---

OnCaptureFailed

Callback is invoked when a capture has failed when the camera device failed to produce a capture result for the request.

public MLCamera.OnCaptureFailedDelegate OnCaptureFailed()

---

OnDeviceDisconnected

Callback is invoked when the camera is disconnected.

public MLCamera.OnDeviceDisconnectedDelegate OnDeviceDisconnected()

---

OnDeviceError

Camera status callback, device error.

public MLCamera.OnDeviceErrorDelegate OnDeviceError()

---

OnDeviceIdle

Callback is invoked when the camera stops streaming.

public MLCamera.OnDeviceStatusDelegate OnDeviceIdle()

---

OnDeviceStreaming

Callback is invoked when the camera is streaming.

public MLCamera.OnDeviceStatusDelegate OnDeviceStreaming()

---

OnPreviewBufferAvailable

Callback is invoked when a captured preview frame buffer is available, invoked on the main thread.

public MLCamera.OnPreviewBufferAvailableDelegate OnPreviewBufferAvailable()

---

OnPreviewCaptureCompleted

Callback is invoked when a preview video frame buffer is available with MLCamera.CaptureType.Preview. Not valid for MR/VR Capture since it does not have preview support.

public MLCamera.OnPreviewCaptureCompletedDelegate OnPreviewCaptureCompleted()

---

OnRawImageAvailable

Callback is invoked when a captured image buffer is available.

public MLCamera.OnCapturedFrameAvailableDelegate OnRawImageAvailable()

---

OnRawVideoFrameAvailable

Callback is invoked when a captured raw/compressed video frame buffer is available, invoked on the main thread.

public MLCamera.OnCapturedFrameAvailableDelegate OnRawVideoFrameAvailable()

---

OnRawVideoFrameAvailable_NativeCallbackThread

Camera capture callback, capture raw video frame, invoked on the same thread as the native callback, allowing the use of the unmanaged native pointer to the frame data memory.

public MLCamera.OnCapturedFrameAvailableDelegate OnRawVideoFrameAvailable_NativeCallbackThread()

---

Protected Events

OnRawVideoFrameAvailableInternal

Callback is invoked when a captured raw/compressed video frame buffer is available, invoked on the main thread.

protected MLCamera.OnCapturedFrameAvailableDelegate OnRawVideoFrameAvailableInternal()

---

Public Enums

CaptureFrameRate

Capture Frame Rate Call MLCameraPrepareCapture to configure frame rate use FrameRate_None when configuring only Image capture FrameRate_60fps only supported when resolution of captures <= 1080P.

Enumerator	Value	Description
None		None Still Capture
_15FPS		Specified 15FPS
_30FPS		Specified 30FPS
_60FPS		Specified 60FPS

---

CaptureType

Capture operation type

Enumerator	Value	Description
Image		To capture an image
Video		To capture a video
Preview		To capture a video and and access the raw buffer of the frames.

---

ConnectFlag

Flags to describe various modules in camera pipeline.

Enumerator	Value	Description
CamOnly		Camera only frame capture
VirtualOnly		virtual only capture
MR		Mixed Reality capture

---

DeviceStatusFlag

Client can implement polling mechanism to retrieve device status and use these masks to view device status.

Enumerator	Value	Description
Connected	1 << 0	Connected
Idle	1 << 1	Idle
Streaming	1 << 2	Opened.
Disconnected	1 << 3	Disconnected.
Error	1 << 4	Error. Call MLCameraGetErrorCode() to obtain the error code.

---

DisconnectReason

Camera Disconnect Reason.

Enumerator	Value	Description
DeviceLost		Device Lost.
PriorityLost		Priority Lost.

---

ErrorType

Camera errors

Enumerator	Value	Description
None	0	No error
Invalid		Invalid state
Disabled		Camera disabled
DeviceFailed		Camera device failed
ServiceFailed		Camera service failed
CaptureFailed		Capture failed

---

Identifier

Logical Camera identifiers available for access.

Enumerator	Value	Description
Main	0	x86 logical camera
CV		CV logical camera

---

MRBlendType

Comment Needed!

Enumerator	Value	Description
Additive	1	Additive Blend Type. It simply adds pixel values of real world and virtual layer.

---

MRQuality

MR Video Quality enumeration

Enumerator	Value	Description
_648x720	1	Specifies 648 x 720 resolution. Aspect ratio: 9x10.
_972x1080	2	Specifies 972 x 1080 resolution. Aspect ratio: 9x10.
_1944x2160	3	Specifies 1944 x 2160 resolution. Aspect ratio: 9x10. CaptureFrameRate._60FPS is not supported for this quality mode.
_960x720	4	Specifies 960 x 720 resolution. Aspect ratio: 4x3.
_1440x1080	5	Specifies 1440 x 1080 resolution. Aspect ratio: 4x3.
_2880x2160	6	Specifies 2880 x 2160 resolution. Aspect ratio: 4x3. CaptureFrameRate._60FPS is not supported for this quality mode.

---

MetadataColorCorrectionAberrationMode

The metadata for the color correction aberration mode.

Enumerator	Value	Description
Off	0	The color correction aberration mode: Off.
Fast		The color correction aberration mode: Fast.
HighQuality		The color correction aberration mode: High Quality.

---

MetadataColorCorrectionMode

The metadata for the color correction mode.

Enumerator	Value	Description
TransformMatrix	0	The color correction mode: Transform Matrix
Fast		The color correction mode: Fast
HighQuality		The color correction mode: High Quality

---

MetadataControlAEAntibandingMode

The metadata for the control AE anti banding mode.

Enumerator	Value	Description
Off	0	The control AE anti banding mode: Off
FiftyHz		The control AE anti banding mode: 50hz
SixtyHz		The control AE anti banding mode: 60hz
Auto		The control AE anti banding mode: Auto

---

MetadataControlAELock

The metadata for the control AE lock.

Enumerator	Value	Description
Off	0	The control AE lock: Off
On		The control AE lock: On

---

MetadataControlAEMode

The metadata for the control AE mode.

Enumerator	Value	Description
Off	0	The control AE mode: Off.
On		The control AE mode: On.

---

MetadataControlAEState

The metadata for the control AE state.

Enumerator	Value	Description
Inactive	0	The control AE state: Inactive
Searching		The control AE state: Searching
Converged		The control AE state: Converged
Locked		The control AE state: Locked
FlashRequired		The control AE state: Flash Required
PreCapture		The control AE state: Pre-capture

---

MetadataControlAWBLock

The metadata for the control AWB lock.

Enumerator	Value	Description
Off	0	The control AWB lock: Off
On		The control AWB lock: On

---

MetadataControlAWBMode

The metadata for the control AWB mode.

Enumerator	Value	Description
Off	0	The control AWB mode: Off
Auto		The control AWB mode: Auto
Incandescent		The control AWB mode: Incandescent
Fluorescent		The control AWB mode: Fluorescent
WarmFluorescent		The control AWB mode: Warm Fluorescent
Daylight		The control AWB mode: Daylight
CloudyDaylight		The control AWB mode: Cloudy Day Light
Twilight		The control AWB mode: Twilight
Shade		The control AWB mode: Shade

---

MetadataControlAWBState

The metadata for the control AWB state.

Enumerator	Value	Description
Inactive	0	The control AWB state: Inactive
Searching		The control AWB state: Searching
Converged		The control AWB state: Converged
Locked		The control AWB state: Locked

---

MetadataScalerAvailableFormats

The metadata for scaler available formats.

Enumerator	Value	Description
RAW16	0x20	RAW16 Format
RAW_OPAQUE	0x24	RAW OPAQUE Format
YV12	0x32315659	YV12 Format
YCrCb_420_SP	0x11	YCrCb 420 SP Format
IMPLEMENTATION_DEFINED	0x22	Implementation Defined
YCbCr_420_888	0x23	YCbCr 420 888 Format
BLOB	0x21	BLOB Format

---

MetadataScalerAvailableStreamConfigurations

The metadata for scaler available stream configurations.

Enumerator	Value	Description
OUTPUT	0	The scaler available stream configuration: Output
INPUT		The scaler available stream configuration: Input

---

OutputFormat

Captured output format

Enumerator	Value	Description
YUV_420_888	1	YUV planar format.
JPEG		Compressed output stream.
RGBA_8888		RGB32 format.

---