OptiTrack Unity Plugin

Main pageOptiTrack Unity Plugin

Streaming tracking data into Unity

Plugin Overview

The OptiTrack Unity3D Plugin allows you to stream real-time rigid body, skeleton, and HMD tracking data from Motive into Unity. Using the streamed data, objects and characters in the scene can be animated. The plugin contents are distributed in unitypackage format, and you can simply load this file into Unity projects to import its contents. Once imported, included C# scripts can be used for instantiating a client origin and receiving the tracking data. This article focuses on how to set up and use the plugin.

Requirements

  • Unity Version: 2017.2 / 2017.1 / 5.4 or above.
  • Visual Studio 2019 or lastest Visual C++ Redistributable

Info2.png

Notes on HMD Integration

Warning2.png

Deprecated support for Oculus HMDs:

  • Support for Oculus Integration have been deprecated starting from UE plugin version 1.23; Plugin version 1.22 or below must be used for Oculus HMDs.
  • Vive and Valve Index HMDs are supported through the OpenVR driver.

Motive Setup (Server)


Streaming Setup

Data Streaming settings in Motive.

From Motive, the tracking data can be streamed in real-time either from a live capture (Live Mode) or recorded capture (Edit Mode). The streaming settings are configured in the Data Streaming pane, and in this pane, the Broadcast Frame Data must be enabled to start streaming the data to a local network interface designated by the IP address in the Local Interface section.

For the best performance, disable streaming of unnecessary data types is recommended. This will reduce the size of data packets and simplify the parsing process. Please make sure only necessary data types are enabled for streaming:

Streaming in Motive

Open the Data Streaming pane in Motive and configure the settings below:

  • Enable Broadcast Frame Data setting at the top
  • Local Interface - Choose the desired IP network address from this dropdown to stream data over.
    • Loopback
      • This is the local computer IP address (127.0.0.1 or Localhost).
      • Used for streaming data locally on the PC you are running Motive on that does not interact with the LAN.
      • Good option for testing network issues.
    • 192.168.0.1x (typical, but may be different depending on which interface is used to establish a LAN connection)
      • This IP address is the interface of the LAN either by Wi-Fi or Ethernet.
      • This will be the same address the Client application will use to connect to Motive.
  • Transmission Type - Typically you will want to set this to Unicast since it subscribes only to the data you wish to use and normally uses less network bandwidth. This is especially advised if streaming data over Wi-Fi.
    • (Optional) If using Multicast, then enable/disable the desired data types. For tracking HMDs, disabling the Marker streaming is advised.
  • Labeled, Unlabeled, Asset Markers - For tracking HMDs, disabling these is advised.
  • Rigid Bodies - This should always be enabled.
  • Skeletons - This is optionally enabled if you are streaming skeletons for skeleton tracking.
  • Skeleton Coordinates - Set the desired option to either Local or Global.
  • Bone Naming Convention - When streaming skeletons, set the appropriate bone naming convention for the client application. (If the character uses the FBX naming convention, this will need to be set to FBX.)

Info2.png

Additional Tips

  • In order to stream data from Edit mode, a captured recording must be playing back in Motive.
  • For best results, it is advised to run Motive and Unreal Engine separately on different computers, so that they are not competing for processing resources.
  • When streaming the data over a Wi-Fi network, Unicast transmission must be used.

Unity Setup (Client)


Import Plugin Package

After you have created a new Unity project or loaded an existing one, double-click on the plugin unitypackage file from the folder you had it saved to after you downloaded it from the OptiTrack website. This should open an Import Unity Package window. Click Import. When the package has been successfully imported, the following contents will be available within the project:

  • Unity plugin files.
  • Import Unity Package window
Plugin Contents
Folder Content Description
Assets/OptiTrack All of the Unity plugin contents are included in this folder.
Assets/OptiTrack/Scripts This is the folder that you will mainly use. It contains plugin C# script components that can be imported into Unity objects for receiving streamed data.
Assets/OptiTrack/Plugins This folder contains the plugin libraries and header files.
Assets/OptiTrack/Prefabs This folder contains prefabs of sample client object, skeleton object, and HMD object.
Assets/OptiTrack/Scenes This folder contains sample Unity scene that includes pre-configured client, rigid body, skeleton, and HMD objects.

Setting Up the Client Object

In order to receive tracking data from a server application (e.g. Motive), a client object must be set up. A client object can be any object within a scene in Unity, and it can be assigned by attaching OptitrackStreamingClient.cs script. This script receives the tracking data from the connected server application (e.g. Motive) and makes the data available within the scene. You can either attach the client script onto an existing object or an empty object, or you can just import the sample from the Assets/Optitrack/Prefabs folder.

  • [Motive] In the Data Streaming pane, configure the desired connection settings and check the Broadcast Frame Data box.
  • [Unity] Under the Prefabs folder, import the "Client - OptiTrack" prefab object into the scene, or attach OptitrackStreamingClient.cs script onto an empty object.
  • [Unity] In the streaming Client object, configure the connection settings to match the streaming settings in Motive.
    • Server Address - IP address of the PC that the server application (Motive) is running on.
    • Local Address - Local IP Address of the PC that the client application (Unity) is running on. (Typically, this looks similar to the Server Address except maybe the last digits.)
    • Connection Type - Must match Motive. Unicast is recommended.
    • Skeleton Coordinates - Choose either Local or Global. Required to match whatever option was chosen in Motive.
    • Bone Naming Conventions - Again, this setting must match what was selected in Motive.
  • [Unity] If you wish to receive tracking data from more than one server instances, you may create multiple objects with the client script attached.
  • Client object configuration in Unity
  • Motive streaming settings

Info2.png

Position Data in Unity

Position component of the streamed data will be received as reference positions in Unity. In other words, the streamed position will reflect the object's relative displacement from its root object or the global origin if the object is placed at the root level of the hierarchy in Unity. Although it is not strictly necessary, you may find it helpful to organize your tracked objects as children of the streaming Client object. This will allow you to adjust the position of the Client object to adjust the position of all streamed objects relative to the Client object.


[Unity Hierarchy Pane] Components set to children of the root Client object.

Animating Rigid Body

  1. [Unity] On an object that you wish to animate, attach the OpitrackRigidBody.cs script.
  2. [Unity] In the Streaming Client entry, link the Client object which the OptitrackStreamingClient.cs is attached to. By default, it searches for an existing client instance, however, this must be specified when there are more than one streaming client objects in the scene.
  3. [Unity] For the Rigid Body ID entry, input the streaming ID of corresponding rigid body asset in Motive. The streaming ID can be found, and changed, under the rigid body properties.
  4. [Motive] Make sure Motive is tracking and streaming the data.
  5. [Unity] Play the scene. The linked object will be animated according to the associated rigid body movement in Motive.
  • Sample configuration of the rigid body component in Unity.
  • Streaming ID assigned for a rigid body in Motive.

Animating Skeleton

By integrating with Unity's animation system, Mecanim, the Unity3D plugin allows Motive to stream full body skeleton data. The skeleton tracking data from Motive is streamed out as hierarchical bone segment orientations, and this data is fed into the Unity's Mecanim system which allows animating characters with different proportions.

Warning2.png

Note: At the time of writing, Mecanim does not support explicit goals for inverse kinematics end-effectors when using real-time retargeting. In addition, you may observe a difference in the overall scale of the position data between the retargeted skeletal animations and streamed rigid bodies. These two limitations may lead to inconsistencies with actors interacting with rigid body props, and will hopefully be addressed in a future version of the integration.

Steps

  1. [Unity] On Unity characters, attach OptitrackSkeletonAnimator.cs script as one of its components.
  2. [Unity] For the Streaming Client entry, link the object which the client script is attached to. By default, it searches for an existing client instance, but this must be specified when there are more than one streaming client objects.
  3. [Unity] Enter Skeleton Asset Name which is Assigned in Motive
  4. [Unity] For the Destination Avatar entry, link to the character that the script is loaded to.
  5. [Motive] From the Data Streaming pane, make sure Motive is streaming.
  6. [Unity] Play the scene. When everything is set up properly, the linked avatar in Unity will be animated according to the streamed skeleton in Motive. The position of the actor will be in its reference position as explained above.
Optitrack Skeleton Animator script configuration from a character in Unity.

Streaming Marker Positions

  1. [Unity] On the OptiTrack Streaming instance, enable Draw Markers setting.
  2. [Motive] In the Data Streaming pane, enable streaming of the markers. Here, you can choose to stream labeled markers, unlabeled markers, or both. When the marker data is being streamed, the passive markers will appear as white sphere and the active markers will appear as cyan in the scene.
  3. [Unity] Make sure the streaming setting is configured correctly, and play the scene.
  4. [Unity] Each marker will be drawn as sphere objects in the scene, as shown in the screenshot below.
Skeleton labeled markers drawn in Unity scene.

Integrating HMDs


OptiTrack motion capture systems can be used to track head-mounted displays (HMD) and integrate the tracking data into Unity for unique VR applications. For instructions on integrating HMD tracking data into Unity, please refer to the corresponding page Unity: HMD Setup.

Info2.png

Supported HMDs

At the time of writing, the following HMDs are supported:

  • HTC VIVE
  • HTC VIVE Pro
  • HTC VIVE Pro 2
  • Valve Index
  • HP Reverb


Warning2.png

Deprecated support for Oculus HMDs:

  • Support for Oculus Integration will be deprecated.
  • Vive, Vive Pro, and Valve Index HMDs are supported through the OptiTrack OpenVR driver.

Live Streaming


Once you have connected the client object and configured the script components on the assets from the scene. Play the scene, and if the properties are correctly configured, the assets in the game scene will be animated according to the position and orientation of the rigid bodies and the HMD in Motive. When playing the scene, the HMD and the assigned actor objects will be placed in respect to their location in Motive’s coordinate system in regards to their position in the editor.

Wireless Multiplayer Setup


When setting up multiplayer games with wireless clients, it is more beneficial for each client to make direct connection to both the tracking-server (Motive) and the game-server, rather than rebroadcasting the streamed tracking data through the game-server. Then, any of the game related actions that interacts with the tracking data can be processed on the game-server, and this server can send out the corresponding updates to the wireless clients. This allows the wireless clients to only receive both the tracking data or updates without having to send back any information; in other words, minimizing the number of data transfers needed. If wireless clients are sending data there will be a minimum of two transfers on the wireless network, and each transfer of data through wireless network is at risk of latency or lost packets.

WirelessClientSetup.png


Related Pages