Version: 20 Mar 2024

ml_native_surface.h

Classes

	Name
struct	MLNativeSurfaceFrameAvailableInfo NativeSurface/NativeSurfaceTexture Output Frame Availability information.
struct	MLNativeSurfaceConfig Configuration values for creating the Native Surface.
struct	MLNativeSurfaceOnFrameAvailableCallback Callbacks for notifying when an output frame is available for consumption.

Types

	Name
typedef struct MLNativeSurfaceFrameAvailableInfo	MLNativeSurfaceFrameAvailableInfo NativeSurface/NativeSurfaceTexture Output Frame Availability information.
typedef struct MLNativeSurfaceConfig	MLNativeSurfaceConfig Configuration values for creating the Native Surface.
typedef struct MLNativeSurfaceOnFrameAvailableCallback	MLNativeSurfaceOnFrameAvailableCallback Callbacks for notifying when an output frame is available for consumption.

Enums

	Name
enum	MLNativeSurfaceAcquiredBufferCount { MLNativeSurfaceAcquiredBufferCount_Min = 1, MLNativeSurfaceAcquiredBufferCount_Max = 16, MLNativeSurfaceAcquiredBufferCount_Ensure32Bits = 0x7FFFFFFF } Number of acquired buffers to be requested in functions MLNativeSurfaceCreate and MLNativeSurfaceTextureCreate.
enum	MLNativeSurfacePixelFormat { MLNativeSurfacePixelFormat_Custom = -4, MLNativeSurfacePixelFormat_Translucent = -3, MLNativeSurfacePixelFormat_Transparent = -2, MLNativeSurfacePixelFormat_Opaque = -1, MLNativeSurfacePixelFormat_None = 0, MLNativeSurfacePixelFormat_Rgba8888 = 1, MLNativeSurfacePixelFormat_Rgbx8888 = 2, MLNativeSurfacePixelFormat_Rgb888 = 3, MLNativeSurfacePixelFormat_Rgb565 = 4, MLNativeSurfacePixelFormat_Bgra8888 = 5, MLNativeSurfacePixelFormat_RgbaFp16 = 22, MLNativeSurfacePixelFormat_Rgba1010102 = 43, MLNativeSurfacePixelFormat_Ensure32Bits = 0x7FFFFFFF } Pixel format to use when creating the native surface. Used in functions MLNativeSurfaceCreate and MLNativeSurfaceTextureCreate.
enum	MLNativeSurfaceTextureBackend { MLNativeSurfaceTextureBackend_OpenGL = 0, MLNativeSurfaceTextureBackend_Ensure32Bits = 0x7FFFFFFF }

Functions

	Name
void	MLNativeSurfaceConfigInit(MLNativeSurfaceConfig * inout_config_values) Initializes MLNativeSurfaceConfig with default values.
void	MLNativeSurfaceOnFrameAvailableCallbackInit(MLNativeSurfaceOnFrameAvailableCallback * inout_callbacks) Initializes MLNativeSurfaceOnFrameAvailableCallback with default values.
MLResult	MLNativeSurfaceCreate(const MLNativeSurfaceConfig config_values, MLHandle out_handle) Create a new NativeSurface.
MLResult	MLNativeSurfaceRelease(MLHandle handle) Release a NativeSurface.
MLResult	MLNativeSurfaceGetNativeWindow(MLHandle handle, MLHandle out_native_window, int32_t out_format) Gets an implementation of NativeWindow that feeds graphic buffers into MLNativeSurface.
MLResult	MLNativeSurfaceReleaseNativeWindow(MLHandle handle, MLHandle native_window) Releases NativeWindow previously acquired by MLNativeSurfaceGetNativeWindow.
MLResult	MLNativeSurfaceAcquireNextAvailableFrame(MLHandle handle, MLHandle * out_native_buffer) Consume next available buffer.
MLResult	MLNativeSurfaceAcquireHardwareBufferFromNativeBuffer(MLHandle handle, MLHandle native_buffer, struct AHardwareBuffer * out_hardware_buffer, uint32_t out_width, uint32_t * out_height) Acquire an AHardwareBuffer* from a native buffer handle.
MLResult	MLNativeSurfaceReleaseFrame(MLHandle handle, MLHandle native_buffer) Release a native buffer previously acquired.
MLResult	MLNativeSurfaceGetFrameTransformationMatrix(MLHandle handle, float out_mtx[16]) Retrieves the 4x4 texture coordinate transform matrix associated with the last acquired Frame.
MLResult	MLNativeSurfaceGetFrameTimestamp(MLHandle handle, int64_t * out_timestamp_ns) Retrieves the timestamp associated with the last acquired frame. The timestamp is in nanoseconds, and is monotonically increasing from beginning of the playback. Its other semantics (zero point, etc) are codec-dependent and should be documented by the codec vendor.
MLResult	MLNativeSurfaceGetFrameQueueBufferTimestamp(MLHandle handle, int64_t * out_timestamp_ns) Retrieves the QueueBuffer timestamp associated with the last acquired Frame This timestamp is for profiling purposes only and corresponds to the time when this frame was queued by internal Codec's producer code. The timestamp is in nanoseconds, and is monotonically increasing from beginning of playback. Its other semantics (zero point, etc) are codec-dependent and should by documented by the codec vendor.
MLResult	MLNativeSurfaceGetFrameNumber(MLHandle handle, uint64_t * out_number) Retrieves the last acquired frame number. The frame number is an incrementing counter set to 0 at Codec configuration time.
MLResult	MLNativeSurfaceGetFrameCropRect(MLHandle handle, MLRecti * out_crop_rect) Retrieves the cropping rectangle of the current frame/buffer.
MLResult	MLNativeSurfaceSetOnFrameAvailableCallback(MLHandle handle, const MLNativeSurfaceOnFrameAvailableCallback callback, void user_data) Set callback to notify client when a frame is available.
MLResult	MLNativeSurfaceTextureCreate(const MLNativeSurfaceConfig config_values, MLNativeSurfaceTextureBackend backend, MLHandle out_handle) Create a new NativeSurfaceTexture.
MLResult	MLNativeSurfaceTextureRelease(MLHandle handle) Release a NativeSurfaceTexture.
MLResult	MLNativeSurfaceTextureUpdateTexImage(MLHandle handle) Update the texture image to the most recent frame from the image stream.
MLResult	MLNativeSurfaceTextureAttachToGLContext(MLHandle handle, uint32_t texture_id) Attach the NativeSurfaceTexture to the graphic context that is current on the calling thread.
MLResult	MLNativeSurfaceTextureDetachFromGLContext(MLHandle handle) Detach the NativeSurfaceTexture from the graphic context that owns the graphic texture object.
MLResult	MLNativeSurfaceTextureGetTransformationMatrix(MLHandle handle, float out_matrix[16]) Retrieve the 4x4 texture coordinate transform matrix associated with the texture image set by the most recent call to MLNativeSurfaceTextureUpdateTexImage.
MLResult	MLNativeSurfaceTextureGetTimestamp(MLHandle handle, int64_t * out_timestamp_ns) Retrieve the timestamp associated with the texture image set by the most recent call to MLNativeSurfaceTextureUpdateTexImage.
MLResult	MLNativeSurfaceTextureSetOnFrameAvailableCallback(MLHandle handle, const MLNativeSurfaceOnFrameAvailableCallback callback, void user_data) Set callback to notify client when a frame is available.

Enums Documentation

MLNativeSurfaceAcquiredBufferCount

Enumerator	Value	Description
MLNativeSurfaceAcquiredBufferCount_Min	1	Min number of buffer count.
MLNativeSurfaceAcquiredBufferCount_Max	16	Max number of buffer count.
MLNativeSurfaceAcquiredBufferCount_Ensure32Bits	0x7FFFFFFF	Ensure enum is represented as 32 bits.

Number of acquired buffers to be requested in functions MLNativeSurfaceCreate and MLNativeSurfaceTextureCreate.

MLNativeSurfacePixelFormat

Enumerator	Value	Description
MLNativeSurfacePixelFormat_Custom	-4	Custom format.
MLNativeSurfacePixelFormat_Translucent	-3	Translucent format.
MLNativeSurfacePixelFormat_Transparent	-2	Transparent format.
MLNativeSurfacePixelFormat_Opaque	-1	Opaque format.
MLNativeSurfacePixelFormat_None	0	None.
MLNativeSurfacePixelFormat_Rgba8888	1	Below enums should match android_pixel_format_t in graphics-base-v1.0.h header.4x8-bit RGBA format.
MLNativeSurfacePixelFormat_Rgbx8888	2	4x8-bit RGBX format.
MLNativeSurfacePixelFormat_Rgb888	3	3x8-bit RGB format.
MLNativeSurfacePixelFormat_Rgb565	4	16-bit RGB format.
MLNativeSurfacePixelFormat_Bgra8888	5	4x8-bit BGRA format.
MLNativeSurfacePixelFormat_RgbaFp16	22	64-bit RGBA format.
MLNativeSurfacePixelFormat_Rgba1010102	43	32-bit RGBA format.
MLNativeSurfacePixelFormat_Ensure32Bits	0x7FFFFFFF	Ensure enum is represented as 32 bits.

Pixel format to use when creating the native surface. Used in functions MLNativeSurfaceCreate and MLNativeSurfaceTextureCreate.

MLNativeSurfaceTextureBackend

Enumerator	Value	Description
MLNativeSurfaceTextureBackend_OpenGL	0	OpenGL texture.
MLNativeSurfaceTextureBackend_Ensure32Bits	0x7FFFFFFF	Ensure enum is represented as 32 bits.

Graphic backend to use.

API Level:

Types Documentation

MLNativeSurfaceFrameAvailableInfo

typedef struct MLNativeSurfaceFrameAvailableInfo MLNativeSurfaceFrameAvailableInfo;

NativeSurface/NativeSurfaceTexture Output Frame Availability information.

More Info

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceConfig

typedef struct MLNativeSurfaceConfig MLNativeSurfaceConfig;

Configuration values for creating the Native Surface.

More Info

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceOnFrameAvailableCallback

typedef struct MLNativeSurfaceOnFrameAvailableCallback MLNativeSurfaceOnFrameAvailableCallback;

Callbacks for notifying when an output frame is available for consumption.

More Info

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

Functions Documentation

MLNativeSurfaceConfigInit

static inline void MLNativeSurfaceConfigInit(
    MLNativeSurfaceConfig * inout_config_values
)

Initializes MLNativeSurfaceConfig with default values.

Parameters


MLNativeSurfaceConfig *	inout_config_values	MLNativeSurfaceConfig structure defined by MLNativeSurfaceConfig that needs to be initialized.

MLNativeSurfaceOnFrameAvailableCallbackInit

static inline void MLNativeSurfaceOnFrameAvailableCallbackInit(
    MLNativeSurfaceOnFrameAvailableCallback * inout_callbacks
)

Initializes MLNativeSurfaceOnFrameAvailableCallback with default values.

Parameters


MLNativeSurfaceOnFrameAvailableCallback *	inout_callbacks	MLNativeSurfaceOnFrameAvailableCallback Callback structure defined by MLNativeSurfaceOnFrameAvailableCallback that needs to be initialized.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

MLNativeSurfaceCreate

MLResult MLNativeSurfaceCreate(
    const MLNativeSurfaceConfig * config_values,
    MLHandle * out_handle
)

Create a new NativeSurface.

Parameters


const MLNativeSurfaceConfig *	config_values	Configuration values for creating the native surface, refer MLNativeSurfaceConfig for more details.
MLHandle *	out_handle	Upon successful return will point to handle to the created NativeSurface. Or else, it will point to ML_INVALID_HANDLE.

Returns


MLResult	MLResult_AllocFailed	Failed to allocate memory for surface.
MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	NativeSurface object was created successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

The NativeSurface created through this API should be released using MLNativeSurfaceRelease once done using it.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceRelease

MLResult MLNativeSurfaceRelease(
    MLHandle handle
)

Release a NativeSurface.

Parameters


MLHandle	handle	MLHandle to the NativeSurface.

Returns


MLResult	MLResult_InvalidParam	NativeSurface object is invalid.
MLResult	MLResult_Ok	NativeSurface object was successfully released.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceGetNativeWindow

MLResult MLNativeSurfaceGetNativeWindow(
    MLHandle handle,
    MLHandle * out_native_window,
    int32_t * out_format
)

Gets an implementation of NativeWindow that feeds graphic buffers into MLNativeSurface.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance. This handle can be:

* Surface created on an Encoder using MLMediaCodecCreateInputSurface. * The output surface created using MLNativeSurfaceCreate. * The output surface created using MLNativeSurfaceTextureCreate. | | MLHandle |out_native_window|Native window as MLHandle on success. | | int32_t |out_format|Native window format on success and for valid pointer, which will be used as EGL_NATIVE_VISUAL_ID.|

Returns


MLResult	MLResult_InvalidParam	NativeSurface object is invalid or out_native_window is NULL.
MLResult	MLResult_Ok	Native window is successfully retrieved.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

This function returns the platform-dependent native window typically used by Graphics API or HW decoders to render frames.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceReleaseNativeWindow

MLResult MLNativeSurfaceReleaseNativeWindow(
    MLHandle handle,
    MLHandle native_window
)

Releases NativeWindow previously acquired by MLNativeSurfaceGetNativeWindow.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
MLHandle	native_window	Native window handle acquired through MLNativeSurfaceGetNativeWindow.

Returns


MLResult	MLResult_InvalidParam	NativeSurface or NativeWindow object is invalid.
MLResult	MLResult_Ok	Native window is successfully released.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

MLNativeSurfaceAcquireNextAvailableFrame

MLResult MLNativeSurfaceAcquireNextAvailableFrame(
    MLHandle handle,
    MLHandle * out_native_buffer
)

Consume next available buffer.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
MLHandle *	out_native_buffer	A pointer to the acquired native buffer of the frame. When buffer is not required anymore, caller must call MLNativeSurfaceReleaseFrame() with this nativeBuffer handle to free up queuing space for incoming buffers.

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	Next available frame is acquired successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.
MLResult	MLMediaGenericResult_NotEnoughData	Native buffer is not available yet to acquire.

Required Permissions:

None

Note: This function is non-blocking and should only be called from the rendering thread after #on_frame_available callback has been fired.

Note: The returned buffer's color format is multi-planar YUV420. Since our underlying hardware interops do not support multiplanar formats, advanced texture features like mipmaping, anisotropic filtering and MSAA will not work on those returned buffers. If such advanced texturing features are required, one has to acquire the YUV buffers as usual and render to an RGB texture and only then, will one be able to enable hardware optimization for such advanced texture features.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceAcquireHardwareBufferFromNativeBuffer

MLResult MLNativeSurfaceAcquireHardwareBufferFromNativeBuffer(
    MLHandle handle,
    MLHandle native_buffer,
    struct AHardwareBuffer ** out_hardware_buffer,
    uint32_t * out_width,
    uint32_t * out_height
)

Acquire an AHardwareBuffer* from a native buffer handle.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance. Pass ML_INVALID_HANDLE when using with native buffers acquired without a MLNativeSurface, like in the case of ml_webview api.
MLHandle	native_buffer	The native buffer handle acquired from a MLNativeSurfaceAcquireNextAvailableFrame call.
struct AHardwareBuffer **	out_hardware_buffer	The AHardwareBuffer pointer to the imported surface.
uint32_t *	out_width	Width of the imported surface.
uint32_t *	out_height	Height of the imported surface.

Returns


MLResult	MLResult_InvalidParam	Import was not performed due to an invalid parameter.
MLResult	MLResult_Ok	Surface was acquired successfully.
MLResult	MLResult_UnspecifiedFailure	Import was not performed due to an unknown error.

Required Permissions:

None

The AHardwareBuffer* can be used with the Vulkan extension VK_ANDROID_external_memory_android_hardware_buffer to get the hardware buffer format properties, which can then be used to bind this hardware buffer to a VkImage. Caller is responsible for releasing the AHardwareBuffer by calling AHardwareBuffer_release() after its usage.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceReleaseFrame

MLResult MLNativeSurfaceReleaseFrame(
    MLHandle handle,
    MLHandle native_buffer
)

Release a native buffer previously acquired.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
MLHandle	native_buffer	The native buffer for the frame to release.

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	Frame was released successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceGetFrameTransformationMatrix

MLResult MLNativeSurfaceGetFrameTransformationMatrix(
    MLHandle handle,
    float out_mtx[16]
)

Retrieves the 4x4 texture coordinate transform matrix associated with the last acquired Frame.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
float	out_mtx	A pointer to an array of 16 floats that will receive retrieved transformation matrix.

Required Permissions:

None

This transform matrix maps 2D homogeneous texture coordinates of the form (s, t, 0, 1) with s and t in the inclusive range [0, 1] to the texture coordinate that should be used to sample that location from the texture. Sampling the texture outside of the range of this transform is undefined.

This transform is necessary to compensate for transforms that the stream content producer may implicitly apply to the content. By forcing users of a GLConsumer to apply this transform we avoid performing an extra copy of the data that would be needed to hide the transform from the user.

The matrix is stored in column-major order.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

Return:

MLResult_InvalidParam One of the parameters is invalid.
MLResult_Ok Transformation matrix was returned.

API Level:

MLNativeSurfaceGetFrameTimestamp

MLResult MLNativeSurfaceGetFrameTimestamp(
    MLHandle handle,
    int64_t * out_timestamp_ns
)

Retrieves the timestamp associated with the last acquired frame. The timestamp is in nanoseconds, and is monotonically increasing from beginning of the playback. Its other semantics (zero point, etc) are codec-dependent and should be documented by the codec vendor.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
int64_t *	out_timestamp_ns	A pointer to user variable for storing retrieved timestamp.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

Return:

MLResult_InvalidParam One of the parameters is invalid.
MLResult_Ok Timestamp was returned.

API Level:

MLNativeSurfaceGetFrameQueueBufferTimestamp

MLResult MLNativeSurfaceGetFrameQueueBufferTimestamp(
    MLHandle handle,
    int64_t * out_timestamp_ns
)

Retrieves the QueueBuffer timestamp associated with the last acquired Frame This timestamp is for profiling purposes only and corresponds to the time when this frame was queued by internal Codec's producer code. The timestamp is in nanoseconds, and is monotonically increasing from beginning of playback. Its other semantics (zero point, etc) are codec-dependent and should by documented by the codec vendor.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
int64_t *	out_timestamp_ns	A pointer to user variable for storing retrieved timestamp.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

Return:

MLResult_InvalidParam One of the parameters is invalid.
MLResult_Ok Timestamp was returned.

API Level:

MLNativeSurfaceGetFrameNumber

MLResult MLNativeSurfaceGetFrameNumber(
    MLHandle handle,
    uint64_t * out_number
)

Retrieves the last acquired frame number. The frame number is an incrementing counter set to 0 at Codec configuration time.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
uint64_t *	out_number	A pointer to user variable for storing retrieved frame number.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

Return:

MLResult_InvalidParam One of the parameters is invalid.
MLResult_Ok Frame number was returned.

API Level:

MLNativeSurfaceGetFrameCropRect

MLResult MLNativeSurfaceGetFrameCropRect(
    MLHandle handle,
    MLRecti * out_crop_rect
)

Retrieves the cropping rectangle of the current frame/buffer.

Parameters


MLHandle	handle	MLHandle to the MLNativeSurface instance.
MLRecti *	out_crop_rect	A pointer to user variable for storing retrieved Crop.

Required Permissions:

None

The Frame crop is returned as {left, bottom, width, height}.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

Return:

MLResult_InvalidParam One of the parameters is invalid.
MLResult_Ok Frame Crop was returned.

API Level:

MLNativeSurfaceSetOnFrameAvailableCallback

MLResult MLNativeSurfaceSetOnFrameAvailableCallback(
    MLHandle handle,
    const MLNativeSurfaceOnFrameAvailableCallback * callback,
    void * user_data
)

Set callback to notify client when a frame is available.

Parameters


MLHandle	handle	MLHandle to the NativeSurface.
const MLNativeSurfaceOnFrameAvailableCallback *	callback	A MLNativeSurfaceOnFrameAvailableCallback function.
void *	user_data	Pointer to user payload data.

Returns


MLResult	MLResult_InvalidParam	MLNativeSurface object is invalid.
MLResult	MLResult_Ok	Callback was set successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureCreate

MLResult MLNativeSurfaceTextureCreate(
    const MLNativeSurfaceConfig * config_values,
    MLNativeSurfaceTextureBackend backend,
    MLHandle * out_handle
)

Create a new NativeSurfaceTexture.

Parameters


const MLNativeSurfaceConfig *	config_values	Configuration values for creating the native surface, refer MLNativeSurfaceConfig for more details.
MLNativeSurfaceTextureBackend	backend	The graphic backend to use.
MLHandle *	out_handle	Upon successful return will point to handle to the created NativeSurfaceTexture. Or else, it will point to ML_INVALID_HANDLE.

Returns


MLResult	MLResult_AllocFailed	Failed to allocate memory for surface.
MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	NativeSurfaceTexture object was created successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureRelease

MLResult MLNativeSurfaceTextureRelease(
    MLHandle handle
)

Release a NativeSurfaceTexture.

Parameters


MLHandle	handle	MLHandle to the NativeSurfaceTexture.

Returns


MLResult	MLResult_InvalidParam	NativeSurfaceTexture object is invalid.
MLResult	MLResult_Ok	NativeSurfaceTexture object was successfully released.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureUpdateTexImage

MLResult MLNativeSurfaceTextureUpdateTexImage(
    MLHandle handle
)

Update the texture image to the most recent frame from the image stream.

Parameters


MLHandle	handle	MLHandle to the NativeSurfaceTexture.

Returns


MLResult	MLResult_InvalidParam	NativeSurfaceTexture object is invalid.
MLResult	MLResult_Ok	the texture was updated successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

May only be called on the thread with the graphic context that contains the texture object.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureAttachToGLContext

MLResult MLNativeSurfaceTextureAttachToGLContext(
    MLHandle handle,
    uint32_t texture_id
)

Attach the NativeSurfaceTexture to the graphic context that is current on the calling thread.

Parameters


MLHandle	handle	MLHandle to the NativeSurfaceTexture.
uint32_t	texture_id	The graphic texture object id.

Returns


MLResult	MLResult_InvalidParam	NativeSurfaceTexture object is invalid.
MLResult	MLResult_Ok	the texture was detached successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureDetachFromGLContext

MLResult MLNativeSurfaceTextureDetachFromGLContext(
    MLHandle handle
)

Detach the NativeSurfaceTexture from the graphic context that owns the graphic texture object.

Parameters


MLHandle	handle	MLHandle to the NativeSurfaceTexture.

Returns


MLResult	MLResult_InvalidParam	NativeSurfaceTexture object is invalid.
MLResult	MLResult_Ok	the texture was detached successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

If called on the thread with graphic context that contains the attached texture object, said texture will be deleted.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureGetTransformationMatrix

MLResult MLNativeSurfaceTextureGetTransformationMatrix(
    MLHandle handle,
    float out_matrix[16]
)

Retrieve the 4x4 texture coordinate transform matrix associated with the texture image set by the most recent call to MLNativeSurfaceTextureUpdateTexImage.

Parameters


MLHandle	handle	MLHandle to the NativeSurfaceTexture.
float	out_matrix	A 4x4 matrix stored in column-major order as 16 consecutive values.

Returns


MLResult	MLResult_InvalidParam	NativeSurfaceTexture object is invalid.
MLResult	MLResult_Ok	Operation was successful.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureGetTimestamp

MLResult MLNativeSurfaceTextureGetTimestamp(
    MLHandle handle,
    int64_t * out_timestamp_ns
)

Retrieve the timestamp associated with the texture image set by the most recent call to MLNativeSurfaceTextureUpdateTexImage.

Parameters


MLHandle	handle	MLHandle to the NativeSurfaceTexture.
int64_t *	out_timestamp_ns	The timestamp in nanoseconds.

Returns


MLResult	MLResult_InvalidParam	NativeSurfaceTexture object is invalid.
MLResult	MLResult_Ok	Operation was successful.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLNativeSurfaceTextureSetOnFrameAvailableCallback

MLResult MLNativeSurfaceTextureSetOnFrameAvailableCallback(
    MLHandle handle,
    const MLNativeSurfaceOnFrameAvailableCallback * callback,
    void * user_data
)

Set callback to notify client when a frame is available.

Parameters


MLHandle	handle	MLHandle to the NativeSurfaceTexture.
const MLNativeSurfaceOnFrameAvailableCallback *	callback	A MLNativeSurfaceTextureOnFrameAvailableCallback function.
void *	user_data	Pointer to user payload data.

Returns


MLResult	MLResult_InvalidParam	NativeSurfaceTexture object is invalid.
MLResult	MLResult_Ok	Callback was set successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

Source code

// %BANNER_BEGIN%
// ---------------------------------------------------------------------
// %COPYRIGHT_BEGIN%
// Copyright (c) 2021 Magic Leap, Inc. All Rights Reserved.
// Use of this file is governed by the Software License Agreement,
// located here: https://www.magicleap.com/software-license-agreement-ml2
// Terms and conditions applicable to third-party materials accompanying
// this distribution may also be found in the top-level NOTICE file
// appearing herein.
// %COPYRIGHT_END%
// ---------------------------------------------------------------------
// %BANNER_END%

#pragma once

#include "ml_api.h"
#include "ml_types.h"

ML_EXTERN_C_BEGIN

typedef enum MLNativeSurfaceAcquiredBufferCount{
  MLNativeSurfaceAcquiredBufferCount_Min = 1,
  MLNativeSurfaceAcquiredBufferCount_Max = 16,
  MLNativeSurfaceAcquiredBufferCount_Ensure32Bits = 0x7FFFFFFF
}MLNativeSurfaceAcquiredBufferCount;

typedef enum MLNativeSurfacePixelFormat {
  MLNativeSurfacePixelFormat_Custom        = -4,
  MLNativeSurfacePixelFormat_Translucent   = -3,
  MLNativeSurfacePixelFormat_Transparent   = -2,
  MLNativeSurfacePixelFormat_Opaque        = -1,
  MLNativeSurfacePixelFormat_None          = 0,
  MLNativeSurfacePixelFormat_Rgba8888      = 1,
  MLNativeSurfacePixelFormat_Rgbx8888      = 2,
  MLNativeSurfacePixelFormat_Rgb888        = 3,
  MLNativeSurfacePixelFormat_Rgb565        = 4,
  MLNativeSurfacePixelFormat_Bgra8888      = 5,
  MLNativeSurfacePixelFormat_RgbaFp16      = 22,
  MLNativeSurfacePixelFormat_Rgba1010102   = 43,
  MLNativeSurfacePixelFormat_Ensure32Bits = 0x7FFFFFFF
} MLNativeSurfacePixelFormat;

typedef ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.") struct MLNativeSurfaceFrameAvailableInfo {
  void *user_data;
} MLNativeSurfaceFrameAvailableInfo;

typedef ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.") struct MLNativeSurfaceConfig {
  uint32_t version;

  MLNativeSurfacePixelFormat pixel_format;

  uint16_t buffer_count;

  uint32_t width;

  uint32_t height;

} MLNativeSurfaceConfig;

ML_STATIC_INLINE void MLNativeSurfaceConfigInit(MLNativeSurfaceConfig *inout_config_values)
{
  if (inout_config_values) {
    inout_config_values->version = 1;
    inout_config_values->width = 0;
    inout_config_values->height = 0;
    inout_config_values->buffer_count = 1;
  }
}

typedef ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.") struct MLNativeSurfaceOnFrameAvailableCallback {
  uint32_t version;

  void (*on_frame_available)(MLHandle handle, const MLNativeSurfaceFrameAvailableInfo *info);
} MLNativeSurfaceOnFrameAvailableCallback;

ML_STATIC_INLINE void MLNativeSurfaceOnFrameAvailableCallbackInit(MLNativeSurfaceOnFrameAvailableCallback *inout_callbacks) {
  if (inout_callbacks) {
    inout_callbacks->version = 1;
    inout_callbacks->on_frame_available = NULL;
  }
}

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceCreate(const MLNativeSurfaceConfig *config_values, MLHandle* out_handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceRelease(MLHandle handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceGetNativeWindow(MLHandle handle, MLHandle *out_native_window, int32_t *out_format);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceReleaseNativeWindow(MLHandle handle, MLHandle native_window);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceAcquireNextAvailableFrame(MLHandle handle, MLHandle *out_native_buffer);

struct AHardwareBuffer;

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceAcquireHardwareBufferFromNativeBuffer(MLHandle handle, MLHandle native_buffer, struct AHardwareBuffer **out_hardware_buffer, uint32_t *out_width, uint32_t *out_height);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceReleaseFrame(MLHandle handle, MLHandle native_buffer);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceGetFrameTransformationMatrix(MLHandle handle, float out_mtx[16]);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceGetFrameTimestamp(MLHandle handle, int64_t *out_timestamp_ns);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceGetFrameQueueBufferTimestamp(MLHandle handle, int64_t *out_timestamp_ns);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceGetFrameNumber(MLHandle handle, uint64_t *out_number);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceGetFrameCropRect(MLHandle handle, MLRecti *out_crop_rect);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceSetOnFrameAvailableCallback(MLHandle handle, const MLNativeSurfaceOnFrameAvailableCallback *callback, void *user_data);

typedef enum MLNativeSurfaceTextureBackend {
  MLNativeSurfaceTextureBackend_OpenGL = 0,
  MLNativeSurfaceTextureBackend_Ensure32Bits = 0x7FFFFFFF
} MLNativeSurfaceTextureBackend;

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureCreate(const MLNativeSurfaceConfig *config_values, MLNativeSurfaceTextureBackend backend,
    MLHandle *out_handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureRelease(MLHandle handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureUpdateTexImage(MLHandle handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureAttachToGLContext(MLHandle handle, uint32_t texture_id);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureDetachFromGLContext(MLHandle handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureGetTransformationMatrix(MLHandle handle, float out_matrix[16]);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureGetTimestamp(MLHandle handle, int64_t *out_timestamp_ns);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLNativeSurfaceTextureSetOnFrameAvailableCallback(MLHandle handle,
    const MLNativeSurfaceOnFrameAvailableCallback *callback, void *user_data);

ML_EXTERN_C_END

ml_native_surface.h

Classes​

Types​

Enums​

Functions​

Enums Documentation​

MLNativeSurfaceAcquiredBufferCount​

MLNativeSurfacePixelFormat​

MLNativeSurfaceTextureBackend​

Types Documentation​

MLNativeSurfaceFrameAvailableInfo​

MLNativeSurfaceConfig​

MLNativeSurfaceOnFrameAvailableCallback​

Functions Documentation​

MLNativeSurfaceConfigInit​

MLNativeSurfaceOnFrameAvailableCallbackInit​

MLNativeSurfaceCreate​

MLNativeSurfaceRelease​

MLNativeSurfaceGetNativeWindow​

MLNativeSurfaceReleaseNativeWindow​

MLNativeSurfaceAcquireNextAvailableFrame​

MLNativeSurfaceAcquireHardwareBufferFromNativeBuffer​

MLNativeSurfaceReleaseFrame​

MLNativeSurfaceGetFrameTransformationMatrix​

MLNativeSurfaceGetFrameTimestamp​

MLNativeSurfaceGetFrameQueueBufferTimestamp​

MLNativeSurfaceGetFrameNumber​

MLNativeSurfaceGetFrameCropRect​

MLNativeSurfaceSetOnFrameAvailableCallback​

MLNativeSurfaceTextureCreate​

MLNativeSurfaceTextureRelease​

MLNativeSurfaceTextureUpdateTexImage​

MLNativeSurfaceTextureAttachToGLContext​

MLNativeSurfaceTextureDetachFromGLContext​

MLNativeSurfaceTextureGetTransformationMatrix​

MLNativeSurfaceTextureGetTimestamp​

MLNativeSurfaceTextureSetOnFrameAvailableCallback​

Source code​

Classes

Types

Enums

Functions

Enums Documentation

MLNativeSurfaceAcquiredBufferCount

MLNativeSurfacePixelFormat

MLNativeSurfaceTextureBackend

Types Documentation

MLNativeSurfaceFrameAvailableInfo

MLNativeSurfaceConfig

MLNativeSurfaceOnFrameAvailableCallback

Functions Documentation

MLNativeSurfaceConfigInit

MLNativeSurfaceOnFrameAvailableCallbackInit

MLNativeSurfaceCreate

MLNativeSurfaceRelease

MLNativeSurfaceGetNativeWindow

MLNativeSurfaceReleaseNativeWindow

MLNativeSurfaceAcquireNextAvailableFrame

MLNativeSurfaceAcquireHardwareBufferFromNativeBuffer

MLNativeSurfaceReleaseFrame

MLNativeSurfaceGetFrameTransformationMatrix

MLNativeSurfaceGetFrameTimestamp

MLNativeSurfaceGetFrameQueueBufferTimestamp

MLNativeSurfaceGetFrameNumber

MLNativeSurfaceGetFrameCropRect

MLNativeSurfaceSetOnFrameAvailableCallback

MLNativeSurfaceTextureCreate

MLNativeSurfaceTextureRelease

MLNativeSurfaceTextureUpdateTexImage

MLNativeSurfaceTextureAttachToGLContext

MLNativeSurfaceTextureDetachFromGLContext

MLNativeSurfaceTextureGetTransformationMatrix

MLNativeSurfaceTextureGetTimestamp

MLNativeSurfaceTextureSetOnFrameAvailableCallback

Source code