Skip to main content
Version: 21 Aug 2024

ml_media_muxer.h

Classes

Name
structMLMediaMuxerSampleData
Data type that encapsulates sample data to be written in to MLMediaMuxer.

Types

Name
typedef struct MLMediaMuxerSampleDataMLMediaMuxerSampleData
Data type that encapsulates sample data to be written in to MLMediaMuxer.

Enums

Name
enumMLMediaMuxerOutputFormat
{
MLMediaMuxerOutputFormat_MPEG4 = 0,
MLMediaMuxerOutputFormat_WEBM = 1,
MLMediaMuxerOutputFormat_3GPP = 2,
MLMediaMuxerOutputFormat_HEIF = 3,
MLMediaMuxerOutputFormat_OGG = 4,
MLMediaMuxerOutputFormat_Ensure32Bits = 0x7FFFFFFF
}
Output formats for MediaMuxer.

Functions

Name
voidMLMediaMuxerSampleDataInit(MLMediaMuxerSampleData * inout_sampledata)
Initializes MLMediaMuxerSampleData with default values.
MLResultMLMediaMuxerCreate(MLHandle * out_handle)
Create a MLMediaMuxer instance.
MLResultMLMediaMuxerRelease(MLHandle media_muxer)
Release the MLMediaMuxer instance.
MLResultMLMediaMuxerGetSupportedOutputFormats(MLHandle media_muxer, const MLMediaMuxerOutputFormat out_format_list[], size_t out_format_list_size)
Get a list of all the supported Media Muxer Output Formats on the Platform.
MLResultMLMediaMuxerGetSupportedMimes(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char const out_mime_list[], size_t * out_mime_list_size)
Get a list of all the supported mime-types for a given Media Muxer Output Format on the Platform.
MLResultMLMediaMuxerConfigure(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char * path)
Configure the MLMediaMuxer for a given Output format with output path.
MLResultMLMediaMuxerAddTrack(MLHandle media_muxer, MLHandle media_format, size_t * out_track_index)
Add a track with given format information.
MLResultMLMediaMuxerSetOrientationHint(MLHandle media_muxer, int32_t degrees)
Set the orientation hint for output video playback.
MLResultMLMediaMuxerSetLocation(MLHandle media_muxer, float latitude, float longitude)
Set and store the geodata (latitude and longitude) in the output file.
MLResultMLMediaMuxerStart(MLHandle media_muxer)
Start muxing.
MLResultMLMediaMuxerWriteSampleData(MLHandle media_muxer, const MLMediaMuxerSampleData * data)
Writes an encoded sample into the muxer.
MLResultMLMediaMuxerStop(MLHandle media_muxer)
Stop muxing.

Enums Documentation

MLMediaMuxerOutputFormat

EnumeratorValueDescription
MLMediaMuxerOutputFormat_MPEG40MP4 format.
MLMediaMuxerOutputFormat_WEBM1VP8/VORBIS data in a WEBM container.
MLMediaMuxerOutputFormat_3GPP23gpp format.
MLMediaMuxerOutputFormat_HEIF3HEIC data in a HEIF container.
MLMediaMuxerOutputFormat_OGG4Opus data in a OGG container.
MLMediaMuxerOutputFormat_Ensure32Bits0x7FFFFFFFEnsure enum is represented as 32 bits.

Output formats for MediaMuxer.

This enumarates list of all the possibly supported output formats for the muxer across all the releases/Platform.

Use MLMediaMuxerGetSupportedOutputFormats to query for all supported Output formats on the Platform.

API Level:

  • 20

Types Documentation

MLMediaMuxerSampleData

typedef struct MLMediaMuxerSampleData MLMediaMuxerSampleData;

Data type that encapsulates sample data to be written in to MLMediaMuxer.

More Info

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

Functions Documentation

MLMediaMuxerSampleDataInit

static inline void MLMediaMuxerSampleDataInit(
MLMediaMuxerSampleData * inout_sampledata
)

Initializes MLMediaMuxerSampleData with default values.

Parameters

MLMediaMuxerSampleData *inout_sampledataSampleData defined by MLMediaMuxerSampleData that needs to be initialized.
Deprecated

Deprecated since 1.4.0. Scheduled for removal.


MLMediaMuxerCreate

MLResult MLMediaMuxerCreate(
MLHandle * out_handle
)

Create a MLMediaMuxer instance.

Parameters

MLHandle *out_handleUpon successful return will point to handle to the created MLMediaMuxer. Or else, it will point to ML_INVALID_HANDLE.

Returns

MLResultMLResult_AllocFailedMLMediaMuxer object creation failed with resource allocation failure.
MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkMLMediaMuxer object was created successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.

Required Permissions:

  • None
Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerRelease

MLResult MLMediaMuxerRelease(
MLHandle media_muxer
)

Release the MLMediaMuxer instance.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.

Returns

MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkMLMediaMuxer object was released successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.

Required Permissions:

  • None
Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerGetSupportedOutputFormats

MLResult MLMediaMuxerGetSupportedOutputFormats(
MLHandle media_muxer,
const MLMediaMuxerOutputFormat * out_format_list[],
size_t * out_format_list_size
)

Get a list of all the supported Media Muxer Output Formats on the Platform.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.
const MLMediaMuxerOutputFormat *out_format_listPointer to MLMediaMuxerOutputFormat. This should not be NULL.
size_t *out_format_list_sizeThe size of the returned out_format_list. This should not be NULL.

Returns

MLResultMLResult_InvalidParamIf one of the parameters are invalid.
MLResultMLResult_OkSuccessfully retrieved all the supported Output Formats.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.

Required Permissions:

  • None

User doesn't own the memory/buffer returned from this API and MUST NOT be freeing or releasing the out_format_list.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerGetSupportedMimes

MLResult MLMediaMuxerGetSupportedMimes(
MLHandle media_muxer,
MLMediaMuxerOutputFormat format,
const char *const * out_mime_list[],
size_t * out_mime_list_size
)

Get a list of all the supported mime-types for a given Media Muxer Output Format on the Platform.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.
MLMediaMuxerOutputFormatformatThe Output format, to which we would like to know all the supported mime-types. The format should be one returned from MLMediaMuxerGetSupportedOutputFormats.
const char const out_mime_listPointer to array of mime-type (char array). This should not be NULL.
size_t *out_mime_list_sizeThe size of the returned out_mime_list. This should not be NULL.

Returns

MLResultMLResult_InvalidParamIf one of the parameters is invalid.
MLResultMLResult_OkSuccessfully retrieved all the supported Output Formats.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.

Required Permissions:

  • None

User doesn't own the memory/buffer returned from this API and MUST NOT be freeing or releasing the out_mime_list.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerConfigure

MLResult MLMediaMuxerConfigure(
MLHandle media_muxer,
MLMediaMuxerOutputFormat format,
const char * path
)

Configure the MLMediaMuxer for a given Output format with output path.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.
MLMediaMuxerOutputFormatformatThe format of the muxed output media file. The format should be one returned from MLMediaMuxerGetSupportedOutputFormats.
const char *pathThe linux-style path for the output media file. This value cannot be empty or NULL.

Returns

MLResultMLResult_AllocFailedFailed to configure the MLMediaMuxer object with resource allocation failure.
MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkMLMediaMuxer object was configured successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.
MLResultMLMediaResult_UnsupportedIf the format is not supported.

Required Permissions:

  • None
Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerAddTrack

MLResult MLMediaMuxerAddTrack(
MLHandle media_muxer,
MLHandle media_format,
size_t * out_track_index
)

Add a track with given format information.

Parameters

MLHandlemedia_muxerInteger handle type used to reference many things returned by the API, instead of returning pointers directly. Use MLHandleIsValid() to check if a handle is valid or not.
MLHandlemedia_formatInteger handle type used to reference many things returned by the API, instead of returning pointers directly. Use MLHandleIsValid() to check if a handle is valid or not.
size_t *out_track_index

This should be called before MLMediaMuxerStart() and after MLMediaMuxerConfigure.

The following media format keys are supported for Audio/Video and Metadata tracks.


| Audio Track | Video Track | Metadata Track |

| MLMediaFormat_Key_Mime | MLMediaFormat_Key_Mime | MLMediaFormat_Key_Mime | | MLMediaFormat_Key_Sample_Rate | MLMediaFormat_Key_Width | | | MLMediaFormat_Key_Channel_Count | MLMediaFormat_Key_Height | | | MLMediaFormat_Key_CSD0 | MLMediaFormat_Key_Bt_Rate | | | MLMediaFormat_Key_CSD1 | MLMediaFormat_Key_Frame_Rate | | | | MLMediaFormat_Key_Color_Range | | | | MLMediaFormat_Key_Color_Standard | | | | MLMediaFormat_Key_Color_Transfer | | | | MLMediaFormat_Key_HDR_Static_Info | | | | MLMediaFormat_Key_CSD0 | |

| | MLMediaFormat_Key_CSD1 | |

MLMediaMuxerGetSupportedMimes can be used to query for all the supported MLMediaFormat_Key_Mime for a given Output Format.

While MLMediaFormat_Key_Mime is must for all tracks, some of the keys aren't mandatory. For Audio track, MLMediaFormat_Key_SAMPLE_RATE and MLMediaFormat_Key_CHANNEL_COUNT is must. For Video track, MLMediaFormat_Key_WIDTH anb MLMediaFormat_Key_HEIGHT are must.

Requierment of providing csd (Codec Specific Data) through MediaFormat is optional for some audio/video mimes where as it is mandatory for some other ones (for example vorbis).

With mp4 container following audio/video and metadata mime types can be supported.


| Audio Mimes | Video Mimes | Metadata Mimes |

| audio/3gpp | video/3gpp | application/ | | audio/mp4a-latm | video/mp4v-es | image/vnd.android.heic | | audio/amr-wb | video/avc | |

| | video/hevc | |

With Webm container following audio/video mime types can be supported.


| Audio Mimes | Video Mimes |

| audio/vorbis | video/x-vnd.on2.vp8 |

| audio/opus | video/x-vnd.on2.vp9 |

With Ogg container, only supported mime is "audio/opus".

If a mime passed through the MLMediaFormat is not supported, it will return MLMediaResult_Unsupported.

API Level:\n 20

media_muxerHandle to the MLMediaMuxer instance.

media_formatMLHandle to the MLMediaFormat that contains the track's format.

out_track_indexUpon successful return, this will conatin the index of the track added.

MLResult_InvalidParamOne of the parameters is invalid.

MLResult_OkIf the operation completes successfully.

MLResult_UnspecifiedFailureThe operation failed with an unspecified error.

MLMediaGenericResult_InvalidOperationThe operation is invalid.

MLMediaResult_UnsupportedIf one of the keys in the format is not supported.

Required PermissionsNone

Deprecated

Deprecated since 1.4.0. Scheduled for removal.


MLMediaMuxerSetOrientationHint

MLResult MLMediaMuxerSetOrientationHint(
MLHandle media_muxer,
int32_t degrees
)

Set the orientation hint for output video playback.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.
int32_tdegreesThe rotation degrees. It has to be either 0, 90, 180 or 270.

Returns

MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkIf the operation completes successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.
MLResultMLMediaGenericResult_InvalidOperationThe operation is invalid.

Required Permissions:

  • None

This should be called before MLMediaMuxerStart() and after MLMediaMuxerConfigure.

Calling this method will not rotate the video frame when muxer is generating the file, but add a composition matrix containing the rotation angle in the output video if the output format is MLMediaMuxerOutputFormat_MPEG4 so that a video player can choose the proper orientation for playback.

Note that some video players may choose to ignore the composition matrix in a video during playback. By default, the rotation degree is 0.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerSetLocation

MLResult MLMediaMuxerSetLocation(
MLHandle media_muxer,
float latitude,
float longitude
)

Set and store the geodata (latitude and longitude) in the output file.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.
floatlatitudeThe latitude in degree. Its value must be in the range [-90, 90].
floatlongitudeThe longitude in degree. Its value must be in the range [-180, 180].

Returns

MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkIf the operation completes successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.
MLResultMLMediaGenericResult_InvalidOperationThe operation is invalid.

Required Permissions:

  • None

This should be called before MLMediaMuxerStart() and after MLMediaMuxerConfigure.

The geodata is stored in udta box if the output format is MLMediaMuxerOutputFormat_MPEG4, and is ignored for other output formats. The geodata is stored according to ISO-6709 standard.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerStart

MLResult MLMediaMuxerStart(
MLHandle media_muxer
)

Start muxing.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.

Returns

MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkIf the operation completes successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.
MLResultMLMediaGenericResult_InvalidOperationThe operation is invalid.

Required Permissions:

  • None

Make sure all the tracks have been added (MLMediaMuxerAddTrack) before calling this.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerWriteSampleData

MLResult MLMediaMuxerWriteSampleData(
MLHandle media_muxer,
const MLMediaMuxerSampleData * data
)

Writes an encoded sample into the muxer.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.
const MLMediaMuxerSampleData *dataThe sample data to be written encapsulated in MLMediaMuxerSampleData.

Returns

MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkIf the operation completes successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.
MLResultMLMediaGenericResult_InvalidOperationThe operation is invalid.

Required Permissions:

  • None

The application needs to make sure that the samples are written into the right tracks. Also, it needs to make sure the samples for each track are written in chronological order (e.g. in the order they are provided by the encoder).

For MPEG4 media format, the duration of the last sample in a track can be set by passing an additional empty buffer) with MLMediaCodecBufferFlag_EOS flag and a suitable presentation timestamp set in time_us (of MLMediaMuxerSampleData structure) as the last sample of that track. This last sample's presentation timestamp shall be a sum of the presentation timestamp and the duration preferred for the original last sample. If no explicit END_OF_STREAM sample was passed, then the duration of the last sample would be the same as that of the sample before that.

The buffer can be reused once this method returns.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

MLMediaMuxerStop

MLResult MLMediaMuxerStop(
MLHandle media_muxer
)

Stop muxing.

Parameters

MLHandlemedia_muxerHandle to the MLMediaMuxer instance.

Returns

MLResultMLResult_InvalidParamOne of the parameters is invalid.
MLResultMLResult_OkIf the operation completes successfully.
MLResultMLResult_UnspecifiedFailureThe operation failed with an unspecified error.
MLResultMLMediaGenericResult_InvalidOperationThe operation is invalid.

Required Permissions:

  • None

Once the muxer stops, it can not be restarted.

Deprecated

Deprecated since 1.4.0. Scheduled for removal.

API Level:

  • 20

Source code

// %BANNER_BEGIN%
// ---------------------------------------------------------------------
// %COPYRIGHT_BEGIN%
// Copyright (c) 2021 Magic Leap, Inc. All Rights Reserved.
// Use of this file is governed by the Software License Agreement,
// located here: https://www.magicleap.com/software-license-agreement-ml2
// Terms and conditions applicable to third-party materials accompanying
// this distribution may also be found in the top-level NOTICE file
// appearing herein.
// %COPYRIGHT_END%
// ---------------------------------------------------------------------
// %BANNER_END%

#pragma once

#include "ml_api.h"
#include "ml_types.h"

ML_EXTERN_C_BEGIN

typedef enum MLMediaMuxerOutputFormat {
MLMediaMuxerOutputFormat_MPEG4 = 0,
MLMediaMuxerOutputFormat_WEBM = 1,
MLMediaMuxerOutputFormat_3GPP = 2,
MLMediaMuxerOutputFormat_HEIF = 3,
MLMediaMuxerOutputFormat_OGG = 4,
MLMediaMuxerOutputFormat_Ensure32Bits = 0x7FFFFFFF
} MLMediaMuxerOutputFormat;

typedef ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.") struct MLMediaMuxerSampleData {
uint32_t version;
size_t track_index;
const uint8_t *buffer;
size_t size;
int64_t time_us;
uint32_t flags;
} MLMediaMuxerSampleData;

ML_STATIC_INLINE void MLMediaMuxerSampleDataInit(MLMediaMuxerSampleData *inout_sampledata) {
if (inout_sampledata) {
inout_sampledata->version = 1;
inout_sampledata->track_index = 0;
inout_sampledata->buffer = NULL;
inout_sampledata->size = 0;
inout_sampledata->time_us = 0;
inout_sampledata->flags = 0;
}
}

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerCreate(MLHandle *out_handle);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerRelease(MLHandle media_muxer);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerGetSupportedOutputFormats(MLHandle media_muxer, const MLMediaMuxerOutputFormat *out_format_list[], size_t *out_format_list_size);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerGetSupportedMimes(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char *const *out_mime_list[], size_t *out_mime_list_size);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerConfigure(MLHandle media_muxer, MLMediaMuxerOutputFormat format, const char *path);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerAddTrack(MLHandle media_muxer, MLHandle media_format, size_t *out_track_index);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerSetOrientationHint(MLHandle media_muxer, int32_t degrees);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerSetLocation(MLHandle media_muxer, float latitude, float longitude);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerStart(MLHandle media_muxer);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerWriteSampleData(MLHandle media_muxer, const MLMediaMuxerSampleData *data);

ML_DEPRECATED_MSG("Use standard Android SDK and NDK APIs.")
ML_API MLResult ML_CALL MLMediaMuxerStop(MLHandle media_muxer);

ML_EXTERN_C_END