Version: 20 Mar 2024

ml_media_muxer.h

Classes

	Name
struct	MLMediaMuxerSampleData Data type that encapsulates sample data to be written in to MLMediaMuxer.

Types

	Name
typedef struct MLMediaMuxerSampleData	MLMediaMuxerSampleData Data type that encapsulates sample data to be written in to MLMediaMuxer.

Enums

	Name
enum	MLMediaMuxerOutputFormat { MLMediaMuxerOutputFormat_MPEG4 = 0, MLMediaMuxerOutputFormat_WEBM = 1, MLMediaMuxerOutputFormat_3GPP = 2, MLMediaMuxerOutputFormat_HEIF = 3, MLMediaMuxerOutputFormat_OGG = 4, MLMediaMuxerOutputFormat_Ensure32Bits = 0x7FFFFFFF } Output formats for MediaMuxer.

Functions

	Name
void	MLMediaMuxerSampleDataInit(MLMediaMuxerSampleData * inout_sampledata) Initializes MLMediaMuxerSampleData with default values.
MLResult	MLMediaMuxerCreate(MLHandle * out_handle) Create a MLMediaMuxer instance.
MLResult	MLMediaMuxerRelease(MLHandle media_muxer) Release the MLMediaMuxer instance.
MLResult	MLMediaMuxerGetSupportedOutputFormats(MLHandle media_muxer, const MLMediaMuxerOutputFormat out_format_list[], size_t out_format_list_size) Get a list of all the supported Media Muxer Output Formats on the Platform.
MLResult	MLMediaMuxerGetSupportedMimes(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char const out_mime_list[], size_t * out_mime_list_size) Get a list of all the supported mime-types for a given Media Muxer Output Format on the Platform.
MLResult	MLMediaMuxerConfigure(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char * path) Configure the MLMediaMuxer for a given Output format with output path.
MLResult	MLMediaMuxerAddTrack(MLHandle media_muxer, MLHandle media_format, size_t * out_track_index) Add a track with given format information.
MLResult	MLMediaMuxerSetOrientationHint(MLHandle media_muxer, int32_t degrees) Set the orientation hint for output video playback.
MLResult	MLMediaMuxerSetLocation(MLHandle media_muxer, float latitude, float longitude) Set and store the geodata (latitude and longitude) in the output file.
MLResult	MLMediaMuxerStart(MLHandle media_muxer) Start muxing.
MLResult	MLMediaMuxerWriteSampleData(MLHandle media_muxer, const MLMediaMuxerSampleData * data) Writes an encoded sample into the muxer.
MLResult	MLMediaMuxerStop(MLHandle media_muxer) Stop muxing.

Enums Documentation

MLMediaMuxerOutputFormat

Enumerator	Value	Description
MLMediaMuxerOutputFormat_MPEG4	0	MP4 format.
MLMediaMuxerOutputFormat_WEBM	1	VP8/VORBIS data in a WEBM container.
MLMediaMuxerOutputFormat_3GPP	2	3gpp format.
MLMediaMuxerOutputFormat_HEIF	3	HEIC data in a HEIF container.
MLMediaMuxerOutputFormat_OGG	4	Opus data in a OGG container.
MLMediaMuxerOutputFormat_Ensure32Bits	0x7FFFFFFF	Ensure enum is represented as 32 bits.

Output formats for MediaMuxer.

This enumarates list of all the possibly supported output formats for the muxer across all the releases/Platform.

Use MLMediaMuxerGetSupportedOutputFormats to query for all supported Output formats on the Platform.

API Level:

Types Documentation

MLMediaMuxerSampleData

typedef struct MLMediaMuxerSampleData MLMediaMuxerSampleData;

Data type that encapsulates sample data to be written in to MLMediaMuxer.

More Info

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

Functions Documentation

MLMediaMuxerSampleDataInit

static inline void MLMediaMuxerSampleDataInit(
    MLMediaMuxerSampleData * inout_sampledata
)

Initializes MLMediaMuxerSampleData with default values.

Parameters


MLMediaMuxerSampleData *	inout_sampledata	SampleData defined by MLMediaMuxerSampleData that needs to be initialized.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

MLMediaMuxerCreate

MLResult MLMediaMuxerCreate(
    MLHandle * out_handle
)

Create a MLMediaMuxer instance.

Parameters


MLHandle *	out_handle	Upon successful return will point to handle to the created MLMediaMuxer. Or else, it will point to ML_INVALID_HANDLE.

Returns


MLResult	MLResult_AllocFailed	MLMediaMuxer object creation failed with resource allocation failure.
MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	MLMediaMuxer object was created successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerRelease

MLResult MLMediaMuxerRelease(
    MLHandle media_muxer
)

Release the MLMediaMuxer instance.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	MLMediaMuxer object was released successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerGetSupportedOutputFormats

MLResult MLMediaMuxerGetSupportedOutputFormats(
    MLHandle media_muxer,
    const MLMediaMuxerOutputFormat * out_format_list[],
    size_t * out_format_list_size
)

Get a list of all the supported Media Muxer Output Formats on the Platform.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.
const MLMediaMuxerOutputFormat *	out_format_list	Pointer to MLMediaMuxerOutputFormat. This should not be NULL.
size_t *	out_format_list_size	The size of the returned out_format_list. This should not be NULL.

Returns


MLResult	MLResult_InvalidParam	If one of the parameters are invalid.
MLResult	MLResult_Ok	Successfully retrieved all the supported Output Formats.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

User doesn't own the memory/buffer returned from this API and MUST NOT be freeing or releasing the out_format_list.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerGetSupportedMimes

MLResult MLMediaMuxerGetSupportedMimes(
    MLHandle media_muxer,
    MLMediaMuxerOutputFormat format,
    const char *const * out_mime_list[],
    size_t * out_mime_list_size
)

Get a list of all the supported mime-types for a given Media Muxer Output Format on the Platform.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.
MLMediaMuxerOutputFormat	format	The Output format, to which we would like to know all the supported mime-types. The format should be one returned from MLMediaMuxerGetSupportedOutputFormats.
const char const	out_mime_list	Pointer to array of mime-type (char array). This should not be NULL.
size_t *	out_mime_list_size	The size of the returned out_mime_list. This should not be NULL.

Returns


MLResult	MLResult_InvalidParam	If one of the parameters is invalid.
MLResult	MLResult_Ok	Successfully retrieved all the supported Output Formats.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.

Required Permissions:

None

User doesn't own the memory/buffer returned from this API and MUST NOT be freeing or releasing the out_mime_list.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerConfigure

MLResult MLMediaMuxerConfigure(
    MLHandle media_muxer,
    MLMediaMuxerOutputFormat format,
    const char * path
)

Configure the MLMediaMuxer for a given Output format with output path.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.
MLMediaMuxerOutputFormat	format	The format of the muxed output media file. The format should be one returned from MLMediaMuxerGetSupportedOutputFormats.
const char *	path	The linux-style path for the output media file. This value cannot be empty or NULL.

Returns


MLResult	MLResult_AllocFailed	Failed to configure the MLMediaMuxer object with resource allocation failure.
MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	MLMediaMuxer object was configured successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.
MLResult	MLMediaResult_Unsupported	If the format is not supported.

Required Permissions:

None

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerAddTrack

MLResult MLMediaMuxerAddTrack(
    MLHandle media_muxer,
    MLHandle media_format,
    size_t * out_track_index
)

Add a track with given format information.

Parameters


MLHandle	media_muxer	Integer handle type used to reference many things returned by the API, instead of returning pointers directly. Use MLHandleIsValid() to check if a handle is valid or not.
MLHandle	media_format	Integer handle type used to reference many things returned by the API, instead of returning pointers directly. Use MLHandleIsValid() to check if a handle is valid or not.
size_t *	out_track_index

This should be called before MLMediaMuxerStart() and after MLMediaMuxerConfigure.

The following media format keys are supported for Audio/Video and Metadata tracks.

| Audio Track | Video Track | Metadata Track |

| | MLMediaFormat_Key_CSD1 | |

MLMediaMuxerGetSupportedMimes can be used to query for all the supported MLMediaFormat_Key_Mime for a given Output Format.

While MLMediaFormat_Key_Mime is must for all tracks, some of the keys aren't mandatory. For Audio track, MLMediaFormat_Key_SAMPLE_RATE and MLMediaFormat_Key_CHANNEL_COUNT is must. For Video track, MLMediaFormat_Key_WIDTH anb MLMediaFormat_Key_HEIGHT are must.

Requierment of providing csd (Codec Specific Data) through MediaFormat is optional for some audio/video mimes where as it is mandatory for some other ones (for example vorbis).

With mp4 container following audio/video and metadata mime types can be supported.

| Audio Mimes | Video Mimes | Metadata Mimes |

| | video/hevc | |

With Webm container following audio/video mime types can be supported.

| Audio Mimes | Video Mimes |

| audio/vorbis | video/x-vnd.on2.vp8 |

| audio/opus | video/x-vnd.on2.vp9 |

With Ogg container, only supported mime is "audio/opus".

If a mime passed through the MLMediaFormat is not supported, it will return MLMediaResult_Unsupported.

API Level:\n 20

media_muxerHandle to the MLMediaMuxer instance.

media_formatMLHandle to the MLMediaFormat that contains the track's format.

out_track_indexUpon successful return, this will conatin the index of the track added.

MLResult_InvalidParamOne of the parameters is invalid.

MLResult_OkIf the operation completes successfully.

MLResult_UnspecifiedFailureThe operation failed with an unspecified error.

MLMediaGenericResult_InvalidOperationThe operation is invalid.

MLMediaResult_UnsupportedIf one of the keys in the format is not supported.

Required PermissionsNone

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

MLMediaMuxerSetOrientationHint

MLResult MLMediaMuxerSetOrientationHint(
    MLHandle media_muxer,
    int32_t degrees
)

Set the orientation hint for output video playback.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.
int32_t	degrees	The rotation degrees. It has to be either 0, 90, 180 or 270.

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	If the operation completes successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.
MLResult	MLMediaGenericResult_InvalidOperation	The operation is invalid.

Required Permissions:

None

This should be called before MLMediaMuxerStart() and after MLMediaMuxerConfigure.

Calling this method will not rotate the video frame when muxer is generating the file, but add a composition matrix containing the rotation angle in the output video if the output format is MLMediaMuxerOutputFormat_MPEG4 so that a video player can choose the proper orientation for playback.

Note that some video players may choose to ignore the composition matrix in a video during playback. By default, the rotation degree is 0.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerSetLocation

MLResult MLMediaMuxerSetLocation(
    MLHandle media_muxer,
    float latitude,
    float longitude
)

Set and store the geodata (latitude and longitude) in the output file.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.
float	latitude	The latitude in degree. Its value must be in the range [-90, 90].
float	longitude	The longitude in degree. Its value must be in the range [-180, 180].

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	If the operation completes successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.
MLResult	MLMediaGenericResult_InvalidOperation	The operation is invalid.

Required Permissions:

None

This should be called before MLMediaMuxerStart() and after MLMediaMuxerConfigure.

The geodata is stored in udta box if the output format is MLMediaMuxerOutputFormat_MPEG4, and is ignored for other output formats. The geodata is stored according to ISO-6709 standard.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerStart

MLResult MLMediaMuxerStart(
    MLHandle media_muxer
)

Start muxing.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	If the operation completes successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.
MLResult	MLMediaGenericResult_InvalidOperation	The operation is invalid.

Required Permissions:

None

Make sure all the tracks have been added (MLMediaMuxerAddTrack) before calling this.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerWriteSampleData

MLResult MLMediaMuxerWriteSampleData(
    MLHandle media_muxer,
    const MLMediaMuxerSampleData * data
)

Writes an encoded sample into the muxer.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.
const MLMediaMuxerSampleData *	data	The sample data to be written encapsulated in MLMediaMuxerSampleData.

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	If the operation completes successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.
MLResult	MLMediaGenericResult_InvalidOperation	The operation is invalid.

Required Permissions:

None

The application needs to make sure that the samples are written into the right tracks. Also, it needs to make sure the samples for each track are written in chronological order (e.g. in the order they are provided by the encoder).

For MPEG4 media format, the duration of the last sample in a track can be set by passing an additional empty buffer) with MLMediaCodecBufferFlag_EOS flag and a suitable presentation timestamp set in time_us (of MLMediaMuxerSampleData structure) as the last sample of that track. This last sample's presentation timestamp shall be a sum of the presentation timestamp and the duration preferred for the original last sample. If no explicit END_OF_STREAM sample was passed, then the duration of the last sample would be the same as that of the sample before that.

The buffer can be reused once this method returns.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

MLMediaMuxerStop

MLResult MLMediaMuxerStop(
    MLHandle media_muxer
)

Stop muxing.

Parameters


MLHandle	media_muxer	Handle to the MLMediaMuxer instance.

Returns


MLResult	MLResult_InvalidParam	One of the parameters is invalid.
MLResult	MLResult_Ok	If the operation completes successfully.
MLResult	MLResult_UnspecifiedFailure	The operation failed with an unspecified error.
MLResult	MLMediaGenericResult_InvalidOperation	The operation is invalid.

Required Permissions:

None

Once the muxer stops, it can not be restarted.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

Source code

// %BANNER_BEGIN%
// ---------------------------------------------------------------------
// %COPYRIGHT_BEGIN%
// Copyright (c) 2021 Magic Leap, Inc. All Rights Reserved.
// Use of this file is governed by the Software License Agreement,
// located here: https://www.magicleap.com/software-license-agreement-ml2
// Terms and conditions applicable to third-party materials accompanying
// this distribution may also be found in the top-level NOTICE file
// appearing herein.
// %COPYRIGHT_END%
// ---------------------------------------------------------------------
// %BANNER_END%

#pragma once

#include "ml_api.h"
#include "ml_types.h"

ML_EXTERN_C_BEGIN

typedef enum MLMediaMuxerOutputFormat {
  MLMediaMuxerOutputFormat_MPEG4 = 0,
  MLMediaMuxerOutputFormat_WEBM  = 1,
  MLMediaMuxerOutputFormat_3GPP  = 2,
  MLMediaMuxerOutputFormat_HEIF  = 3,
  MLMediaMuxerOutputFormat_OGG   = 4,
  MLMediaMuxerOutputFormat_Ensure32Bits = 0x7FFFFFFF
} MLMediaMuxerOutputFormat;

typedef ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.") struct MLMediaMuxerSampleData {
  uint32_t version;
  size_t track_index;
  const uint8_t *buffer;
  size_t size;
  int64_t time_us;
  uint32_t flags;
} MLMediaMuxerSampleData;

ML_STATIC_INLINE void MLMediaMuxerSampleDataInit(MLMediaMuxerSampleData *inout_sampledata) {
  if (inout_sampledata) {
    inout_sampledata->version = 1;
    inout_sampledata->track_index = 0;
    inout_sampledata->buffer = NULL;
    inout_sampledata->size = 0;
    inout_sampledata->time_us = 0;
    inout_sampledata->flags = 0;
  }
}

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerCreate(MLHandle *out_handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerRelease(MLHandle media_muxer);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerGetSupportedOutputFormats(MLHandle media_muxer, const MLMediaMuxerOutputFormat *out_format_list[], size_t *out_format_list_size);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerGetSupportedMimes(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char *const *out_mime_list[], size_t *out_mime_list_size);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerConfigure(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char *path);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerAddTrack(MLHandle media_muxer, MLHandle media_format, size_t *out_track_index);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerSetOrientationHint(MLHandle media_muxer, int32_t degrees);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerSetLocation(MLHandle media_muxer, float latitude, float longitude);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerStart(MLHandle media_muxer);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerWriteSampleData(MLHandle media_muxer, const MLMediaMuxerSampleData *data);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerStop(MLHandle media_muxer);

ML_EXTERN_C_END

ml_media_muxer.h

Classes​

Types​

Enums​

Functions​

Enums Documentation​

MLMediaMuxerOutputFormat​

Types Documentation​

MLMediaMuxerSampleData​

Functions Documentation​

MLMediaMuxerSampleDataInit​

MLMediaMuxerCreate​

MLMediaMuxerRelease​

MLMediaMuxerGetSupportedOutputFormats​

MLMediaMuxerGetSupportedMimes​

MLMediaMuxerConfigure​

MLMediaMuxerAddTrack​

| Audio Track | Video Track | Metadata Track |

| | MLMediaFormat_Key_CSD1 | |

| Audio Mimes | Video Mimes | Metadata Mimes |

| | video/hevc | |

| Audio Mimes | Video Mimes |

| audio/opus | video/x-vnd.on2.vp9 |

MLMediaMuxerSetOrientationHint​

MLMediaMuxerSetLocation​

MLMediaMuxerStart​

MLMediaMuxerWriteSampleData​

MLMediaMuxerStop​

Source code​

Classes

Types

Enums

Functions

Enums Documentation

MLMediaMuxerOutputFormat

Types Documentation

MLMediaMuxerSampleData

Functions Documentation

MLMediaMuxerSampleDataInit

MLMediaMuxerCreate

MLMediaMuxerRelease

MLMediaMuxerGetSupportedOutputFormats

MLMediaMuxerGetSupportedMimes

MLMediaMuxerConfigure

MLMediaMuxerAddTrack

MLMediaMuxerSetOrientationHint

MLMediaMuxerSetLocation

MLMediaMuxerStart

MLMediaMuxerWriteSampleData

MLMediaMuxerStop

Source code