185 docs tagged with "Unity"

Overview

Using Agora with Magic Leap 2 in Unity

An overview of the Anchors example scene which utilizes the Spaces application.

Prerequisites

This guide provides a comprehensive overview of the MLSpace API, specifically focusing on the key features and functions used to manage MLSpaces in your applications. It covers topics such as subscribing to the OnLocalizationEvent to monitor localization status changes, fetching available spaces, understanding and getting localization results, localizing into a specific space, and the export and import of spaces. Each topic is explored with detailed explanations and illustrative code snippets, aiming to provide a clear understanding of how to leverage the LocalizationMap effectively. The guide offers insights into the usage of key structs, values, and enums, enhancing your ability to create more interactive and responsive applications using MLSpaces.

This guide provides a walk through of creating a custom camera capture script. The sections in this guide were written in a linear flow although each section can also be referenced individually.

Magic Leap's Eye Tracking data is retrieved in two ways.

This section provides information on the Gesture Classification API and how to enable it inside your application.

Namespace

This guide provides detailed instructions and examples on how to use the MagicLeap's MLPowerManager API for managing power states and properties of components in Unity. The guide first explains the concept of power states, showing how to get all available states and retrieve a specific state for a device. It then delves into the process of setting a power state with thorough examples. The guide proceeds to explain how to get and update the properties of a component, focusing on different aspects like battery info, battery level, charging state, and connection state. Lastly, it touches on how to handle power management events such as errors, state changes, and property changes through callbacks. By following this guide, developers can efficiently control and manage Magic Leap 2's power states and properties inside their Unity Application.

This guide provides a comprehensive overview of the MLSpace API, specifically focusing on the key features and functions used to manage MLSpaces in your applications. It covers topics such as subscribing to the OnLocalizationEvent to monitor localization status changes, fetching available spaces, understanding and getting localization results, localizing into a specific space, and the export and import of spaces. Each topic is explored with detailed explanations and illustrative code snippets, aiming to provide a clear understanding of how to leverage the MLSpace API effectively. The guide offers insights into the usage of key structs, values, and enums, enhancing your ability to create more interactive and responsive applications using MLSpaces.

This document demonstrates how to use the MagicLeapXrProvider to check if the application is using the Magic Leap App Simulator.

This section describes how to configure your Unity Project settings so that you can test your application inside the Magic Leap App simulator plug-in for the Unity Editor.

This section provides information on using Magic Leap 2's Audio API for audio input and output.

An overview of the audio capture demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

This section provides an example script that demonstrates how to subscribe to Magic Leap's Audio change events.

This section provides examples of how to use Magic Leap's APIs to record audio.

This section demonstrates how to collect battery information from the Magic Leap 2's Compute Pack and Controller in Unity. See the Android Intents Overview guide for information regarding the native implementation.

This document describes how to build a Magic Leap 2 application using Unity.

This document describes how to build a Magic Leap 2 application using Unity.

Overview

Overview

An overview of the camera capture demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

The Magic Leap 2 supports the Android Camera2 NDK APIs. With Android Camera allows developers to capture real and virtual content inside their applications. While the Magic Leap 2 has only one physical, it is presented as 3 separate devices in the API. The Main Camera (Camera Id 0) and Mixed Reality Camera (Camera Id 3) are mutually exclusive, however they can be accessed even while the CV Camera (Camera Id 1) is being used.

This section provides detailed steps on how to manually configure your Unity Project so that you can build and deploy OpenXR applications for Magic Leap 2.

This topic describes how to configure the Unity Build Settings to utilize Magic Leap features.

An overview of the controller demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

This section demonstrates how to use Unity's Input System to access input events from the Magic Leap 2 controller.

Unity developers can use the OpenXR Magic Leap 2 Controller Interaction Profile to access the controller's input using Unity's Input system. This profile can be enabled inside your project's OpenXR Settings Window > XR Plugin Manager > OpenXR Settings

This section provides instructions on creating a new Unity with a version that is compatible with Magic Leap 2 development.

This section provides instructions on creating a new Unity with a version that is compatible with Magic Leap 2 development.

Instructions on how to create a new MRTK 3 project compatible with Magic Leap 2

Learn how to headlock content in Unity applications.

An overview of the Anchors example scene which utilizes the Spaces application.

An overview of the cv camera demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

The Magic Leap 2 Voice Input framework supports App Specific Voice Intents which are custom voice intents you can develop to use within your app. You can develop a full set of voice intents to incorporate in your applications with the assistance of the Voice Intent Development ToolKit (VIDTK).

An overview of the Depth Camera example scene.

Introduction

Instructions on how to download the Magic Leap MRTK Template Project

Learn various tools to enhance the way Magic Leap 2 displays content.

Instructions for enabling Developer Mode on device

If you are experiencing issues with the Magic Leap App Simulator, let us know so we can help you get up and running. The App Simulator package makes it easy for users to save all of the diagnostics logs.

Magic Leap Unity (OpenXR) Examples

This section demonstrates how to use the Gesture Classification API.

Explicit intents specify which application will satisfy the intent, by supplying either the target app's package name or a fully-qualified component class name. Explicit Intents can also be used to pass data to other activities.

Unity developers can use the OpenXR Eye Gaze Interaction Profile to determine what a user is looking at, allowing a hands-free method of interacting with their application. To access the Eye Gaze data, the interaction profile needs to be enabled in your project's OpenXR Settings (Window > XR Plugin Manager > OpenXR Settings).

An overview of the eye tracking demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

If your Application collects, stores, transfers or otherwise uses data off the Magic Leap 2 device that is received via this API, then you must comply with the Magic Leap 2 Eye Tracking Data Transparency Policy.

If your Application collects, stores, transfers or otherwise uses data off the Magic Leap 2 device that is received via this API, then you must comply with the Magic Leap 2 Eye Tracking Data Transparency Policy.

If your Application collects, stores, transfers or otherwise uses data off the Magic Leap 2 device that is received via this API, then you must comply with the Magic Leap 2 Eye Tracking Data Transparency Policy.

Learn how make content appear more stable.

If your Application collects, stores, transfers or otherwise uses data off the Magic Leap 2 device that is received via this API, then you must comply with the Magic Leap 2 Eye Tracking Data Transparency Policy.

An overview of the gesture classification demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

Overview

These guides serve as a starting point for Magic Leap app development in Unity using OpenXR. If this is your first time developing for Magic Leap, we recommend that you follow the guides sequentially. If you are an experienced Magic Leap developer, these guides can be used as standalone references.

This section includes information on how to detect if an application is running on a Magic Leap 2 in addition to checking if a developer is most likely building for Magic Leap 2.

These guides serve as a starting point for Magic Leap app development in Unity. If this is your first time developing for Magic Leap, we recommend that you follow the guides sequentially. If you are an experienced Magic Leap developer, these guides can be used as standalone references.

An overview of the Global Dimmer example scene which showcases global dimming range.

Learn how to use the Magic Leap 2's unique global dimming technology.

Learn how to use the Magic Leap 2's unique global dimming technology.

An overview of the hand tracking demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

The Magic Leap 2 OpenXR Unity SDK supports hand tracking via the Hand Interaction Profile. The interaction profile can be enabled by selecting Edit > Project Settings > XR Plug-in Management > OpenXR, then adding the interaction profile into the Enabled Interaction Profiles section.

This section provides information on the core Hand Tracking API and how to enable it inside your applications.

Overview

If the Magic Leap can't locate its position in an environment, it experiences "tracking loss". The Magic Leap 2 lets developers manage their own tracking loss behavior -- some developers may want to pause the update loop and display a splash image, while others may want the app to continue playing.

If the Magic Leap can't locate its position in an environment, it experiences "tracking loss". The Magic Leap 2 lets developers manage their own tracking loss behavior -- some developers may want to pause the update loop and display a splash image, while others may want the app to continue playing.

An overview of the Haptics example scene.

An overview of the hand tracking demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

Using Immersal SDK with Magic Leap 2 in Unity

Implicit Intents do not directly specify the Android components which should be called, it only specifies an action to be performed. This allows a component from another app to handle the action request. For example, you may want to prompt the user to perform eye-tracking calibration.

Unity's Input System can locate Controls using paths. Bindings on Input Actions utilize this feature to identify the Control(s) they receive input from. You can also use paths to directly look up Controls and Devices, or to have the Input System search for Controls among all devices using the InputSystem.FindControls method.

This guide provides information on the input paths that are supported on Magic Leap 2 when using the OpenXR Eye Gaze Interaction Profile. For general information about OpenXR input in Unity, see the Unity OpenXR Plugin Input Manual.

This guide provides information the Input Control Paths and Interaction Profiles that are supported on Magic Leap 2. For general information about the OpenXR Hand Interaction Profile in Unity. See the Unity OpenXR Hand Interaction Input Manual.

Input Binding Paths

It is recommended that developers read the controller input using Unity's Input System. However, developers can obtain the controller's input directly from the InputDevice. This section provides an example of reading input using the TryGetFeatureValue method and XRCommonUsages features.

It is recommended that developers use the Unity Input System to obtain the Gaze Input. However, developers can also obtain eye tracking input directly from the InputDevice. This section provides an example of how to read input using the TryGetFeatureValue method and EyeTrackingUsages features.

It is recommended that developers read the controller input using Unity's Input System. However, developers can obtain the controller's input directly from the InputDevice. This section provides an example of reading input using the TryGetFeatureValue method and XRCommonUsages features.

This section describes how to read Magic Leap 2's controller input using Unity's Input System and Input Actions. The samples in this category assume that you are using the MagicLeapOpenXRInput.inputactions asset provided in the Magic Leap Samples. However, they can be easily modified to support custom input actions.

This section describes how to download and configure the Magic Leap Unity Example project.

This section provides information on how to install and configure the Unity Editor required to develop for Magic Leap 2.

This section provides information on how to install and configure the Unity Editor required to develop for Magic Leap 2.

Unity developers can use intents inside their applications to open external activities and services. For example, you can use Intents to open the Eye Calibration application after detecting poor accuracy when using the Eye Tracking feature, or if you want the user to localize into a map before loading the content from your application. This section provides examples of how to call both implicit and explicit intents.

This section includes details on reading the Intrinsic and Extrinsic parameters from the Magic Leap camera. These values can be queried using the MLCamera.ResultExtras value provided in the Camera Capture callbacks.

Accessing Iris Unlock for developers

These scripts demonstrate various functionalities of the OpenXR Localization Map Feature, which is used for managing spaces in a mixed reality environment. The scripts are broken down into specific functionalities: requesting permissions, retrieving available spaces, localizing into a space, exporting and saving spaces, and importing previously saved spaces. Each script serves as an isolated example of a particular functionality, making it easier to understand and implement these functions in your own projects. They are designed to be used with the Unity game engine and the Magic Leap platform. Note that they should be used in a real device environment for proper functioning.

The Magic Leap OpenXR Localization Map Feature allows developers to manage Spaces inside their application. This includes functionality such as querying a list of spaces, requesting localization into a specific Space and importing/exporting on device Spaces.

To obtain Magic Leap device specific features such as checking the eye tracking FixationConfidence status or if the user is blinking use Magic Leap's InputSubsystem.Extensions

Information about MRTK 3 settings for Magic Leap

Magic Leap Spectator (ML Spectator) is a mobile application that makes it easy for anyone to capture and share high quality mixed reality content streamed from a Magic Leap 2 device. This powerful application supports a wide variety of use cases, for example:

An overview and walkthrough guide for using the Magic Leap XR Keyboard.

This section provides an overview of the marker tracker and references to create a custom Marker Tracking script.

The Magic Leap 2's marker tracker API is light weight and can be extended depending on your application's needs. This section provides an example of extending the API to broadcast an event when a marker is found, lost or updated based on the amount of time that has passed since the marker was update.

This section includes an example of detecting Fiducial Markers on the Magic Leap 2 headset.

An overview of the marker tracking demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

This section provides an overview and API references for the the Magic Leap 2 Marker Understanding OpenXR Feature.

This section includes an example of detecting Fiducial Markers on the Magic Leap 2 headset.

An overview of the media player demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

Learn how to create meshes from surfaces detected by the Magic Leap.

An overview of the meshing demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

This section details how to use Magic Leap 2's Meshing Subsystem Support feature in Unity. This feature allows applications to access a mesh that represents real-world geometry. Unity's mesh manager. Developers can use Unity's AR Mesh Manager or AR Point Cloud Manager component to visualize the mesh. Magic Leap specific settings, such as MeshingQuerySettings, can be configured via the MagicLeapMeshingFeature class.

Learn how to adjust the mesh queries location and bounds.

Learn how the Meshing Subsystem component works and it's features.

Learn to adjust the appearance of the mesh generated by the Magic Leap 2.

This section includes useful information about API changes when migrating Magic Leap 1 applications to Magic Leap 2.

Upgrade to Magic Leap's new Eye Tracking APIs

In order for an existing Magic Leap 2 Unity application to continue working under OpenXR, here are the minimum steps required:

The Unity Mixed Reality Template allows developers to start their projects configured for multi-platform mixed reality development. This guide will walk you through updating the Mixed Reality Template project to work with the Magic Leap 2's Unity OpenXR SDK.

Using MRTK with Magic Leap 2 in Unity

Instructions on how to get started with Magic Leap MRTK3

Overview of Microsoft's Mixed Reality ToolKit

The MLSpace API allows developers to manage Spaces inside their application. This includes functionality such as querying a list of spaces, requesting localization into a specific Space and importing/exporting on device Spaces.

The Magic Leap 2 MLCamera API allows developers to capture real and virtual content inside their applications. While the Magic Leap 2 has only one camera for capturing content, two separate streams can be access from the camera at the same time. This allows you to create applications that can stream the user's point of view, while using the camera to perform computer vision tasks.

This section includes a simple camera example. A detailed explanation of each function can be found in the Camera Overview guide.

The Magic Leap 2 MLCamera API allows developers to capture real and virtual content inside their applications.

These scripts demonstrate various functionalities of the MLSpace API, which is used for managing spaces in a mixed reality environment. The scripts are broken down into specific functionalities: requesting permissions, retrieving available spaces, localizing into a space, exporting and saving spaces, and importing previously saved spaces. Each script serves as an isolated example of a particular functionality, making it easier to understand and implement these functions in your own projects. They are designed to be used with the Unity game engine and the Magic Leap platform. Note that they should be used in a real device environment for proper functioning.

Use the following chart to see if your project is compatible with MRTK 2.8 or MRTK 3.0.

MRTK3 Samples Installation and Overview

Object Alignment is a sample application that demonstrates how to attach large scale 3D content, such as digital twin meshes, to a spatial localization map such that the 3D content remains stuck to the world as the user moves around a large space.

The transition to OpenXR involves the phasing out of MLSDK in favor of OpenXR extensions. To align with this change, Magic Leap’s Unity SDK is shifting from Unity's platform specific XR Plugin (ie: com.unity.xr.magicleap) to Unity's OpenXR Plugin (com.unity.xr.openxr). This shift is accompanied by the addition of new OpenXR Features and API for accessing the functions previously defined by MLSDK C-APIs. This document will outline the rationale behind the move to OpenXR, detail the steps involved in the migration process, and provide insights into the changes and enhancements introduced by this transition.

Learn how to use the Magic Leap 2 passthrough feature for capture and dimming.

Guide on using the ML2 Photon Fusion example project

Using Ready Player Me with Magic Leap 2

This example uses the OpenXR Unity API. For a previous version using the Unity MLSDK, see this page.

Learn how the Magic Leap classifies surfaces.

Learn how to detect surfaces using the Magic Leap SDK.

An overview of the planes demo scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

Magic Leap 2 tries to understand its environment using feature points and planes.

Note: This example uses the OpenXR version of the Magic Leap 2 Unity SDK. For a previous version using the Unity MLSDK, see here.

This section includes details on detecting if an application is running on a Magic Leap 2 and if a developer is targeting Magic Leap in Unity's XR Manager.

This section includes details on detecting if an application is running on a Magic Leap 2 and if a developer is targeting Magic Leap in Unity's XR Manager.

This example demonstrates how to manage power states and properties on a device using the Magic Leap platform's Power Manager API. It includes initialization and setup of available power states and properties, reading current power state and properties, and setting a new power state. The code also handles different events that could occur during these operations, such as changes in power state or properties, and potential errors. The detailed logging provides real-time insight into the power management process.

Power Manager provides a set of power management APIs that allow applications to receive callbacks when the power state changes, APIs to get device components power state and properties, and to set the controller’s power states.

Learn how to query planes Magic Leap that were detected by the Magic Leap.

This section covers how to use the Magic Leap 2 Controller Interaction Profile with Input System. For more information about obtaining OpenXR Input see Unity's OpenXR Input Documentation.

This section includes information on how to get started using Hand Tracking via Unity's XR Hands Package and the OpenXR Hand Interaction Profile. See the Unity Manual for more information about the XRHands Package and OpenXR Input.

Before adding the sample scripts to your scene, make sure to complete the following:

This section assumes you created a new project using the URP template. The URP template project includes a collection of various URP assets, each pre-configured with different settings to represent different levels of quality. However, for Android apps running on our platform, the default configuration does not allow for a smooth 60 FPS experience in even the simplest scene. Follow these steps to configure the URP settings in your project to improve performance.

This section assumes you created a new project using the URP template. The URP template project includes a collection of various URP assets, each pre-configured with different settings to represent different levels of quality. However, for Android apps running on our platform, the default configuration does not allow for a smooth 60 FPS experience in even the simplest scene. Follow these steps to configure the URP settings in your project to improve performance.

This section includes information about the Magic Leap 2 OpenXR Rendering Extensions Feature that will make your content appear more robust.

This guide walks through how to run Unity apps in the Application Simulator.

This section provides details on how developers can create and register voice commands dynamically at runtime. This feature can be helpful when loading content dynamically.

This section provides details on how developers can create voice commands at runtime. This feature can be helpful when loading content dynamically.

MRTK3 Samples Installation and Overview

The Magic Leap 2 Unity Example Project showcases the Magic Leap's Unity SDK features. This section includes walkthroughs of each example and can be used as a reference point for development or troubleshooting.

An overview of the Segmented Dimmer example scene which showcases the segmented dimming range.

Learn how to dim sections of the display to add contrast to certain objects.

Setting Up MRTK with Magic Leap 2 for Unity

This section includes code examples for developers to reference when implementing voice input in their applications.

This document provides step-by-step instructions on how to create a cube that can be selected, grabbed and manipulated using Unity's XR Interaction Toolkit and the sample scene created in the Building a Simple App guide.

In this tutorial, you'll learn how to set up the Spatial Mapping component with Magic Leap 2 in Unity. By the end of this guide, you'll have a scene capable of meshing the world around you, providing a foundation for immersive mixed reality experiences. The Spatial Mapping in Magic Leap 2 is similar to AR Foundation's Spatial Mapping component.

Soundfield Plugin Overview

Detailed descriptions and screenshots of all available components of Soundfield Audio

Learn how to link objects to Spatial Anchors and use them to create persistent content.

Learn the core API calls required to implement Magic Leap 2's Spatial Anchors API.

Learn how to get notified when spatial anchors are created, added or removed.

Contains code that can be used as a reference or demo Magic Leap 2's Spatial Anchors functionality.

Learn how to link objects to Spatial anchors and use them to create persistent content.

Learn how make content appear more stable.

This section walks through the options you have for starting a session in Application Simulator that you can use to run your app. You can use ML C API or Unity apps.

This section provides details on how developers can start and stop tracking voice intents inside their applications.

This section reviews different steps you can take to troubleshoot problems you may encounter when using Application Simulator. Topics covered include:

This guide only covers Unity C# script debugging. For C/C++ debugging, i.e. debugging custom or third party libraries, see Unity Native Debugging via Android Studio.

This section includes information about developing Unity applications for Magic Leap 2.

Unity does not contain a specific Platform Defines for Magic Leap 2. Instead, developers are required to create and edit custom defines symbols if they wish to use them when developing their applications. This topic describes how to create a custom directive to identify code you wish to include or exclude from your application. For example, you can choose code that will only be compiled when deploying to the Magic Leap 2.

Unity Power Optimization

This section provides various samples, tools, and external libraries that you can reference or integrate into your Magic Leap 2 Unity projects.

Sensors are devices that measure environmental characteristics of the device that the content is running on, such as light, pressure, temperature, orientation, and motion. Unity’s Input System allows you to access sensors on Android devices using the standard Android Sensor API. In this guide, we will show you how to enable, disable, and read data from sensors on Android using the Input System.

Sensors are devices that measure environmental characteristics of the device that the content is running on, such as light, pressure, temperature, orientation, and motion. Unity’s Input System allows you to access sensors on Android devices using the standard Android Sensor API. In this guide, we will show you how to enable, disable, and read data from sensors on Android using the Input System.

VisionLib Tracking on Magic Leap 2 in Unity

This section includes details on rendering the Magic Leap's camera output on a Raw Image UI Component. When receiving camera output developers can query the format using the MLCamera.CameraOutput.Format property.

The Voice Intent API will only register voice commands if Voice Input is enabled inside the Magic Leap 2's System Settings (Settings > Magic Leap Inputs > Voice). This section demonstrates how to check if a user has enabled Voice Input.

An overview of the Voice Intents scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

Magic Leap recommends using the Voice Intent Development Toolkit (VIDTK) to create and deploy custom voice commands. The toolkit provides validation and guidance within the toolkit UI that are not available in the Unity Editor. For more information, see Voice Commands.

A Slot is a placeholder string for a set of potential values. The utterance will use one of the values and the developer can have different logic based on which value was spoken. To indicate use of a slot, put the slot name within { } for the command.

Overview of PTC's Vuforia Engine

An overview of the Wacom Tablet example scene.

Information on how to create your own Web View with the Magic Leap Unity SDK.

Learn how to interact with Magic Leap's Web View API.

Learn about Magic Leap's WebView Feature.

An overview of the WebRTC scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

An overview of the WebView scene included in the Magic Leap 2 Examples Project, which uses Unity's XR Interaction Toolkit.

|Projects||

An overview of the World Camera example.

This guide will demonstrate how to configure the Unity XRI Hands Demo Scene to work with the Magic Leap 2.

This section provides information on how to create a Player Rig that is compatible with Magic Leap 2 and is driven by the user's head and controller movement controller. This can be done by either: