Difference between revisions of "NatNet: Data Types"

 
(43 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  [[Main Page|Back to the Main page]] → [[NatNet SDK|Back to the NatNet page]]
+
  [[Main Page|Main page]] → [[NatNet SDK 3.0|NatNet SDK]] → [[NatNet: Data Types]]
 
----
 
----
  
Line 13: Line 13:
  
 
For every asset (e.g. reconstructed markers, rigid bodies, skeletons, force plates) included within streamed capture sessions, their descriptions and tracking data are stored separately. This format allows frame-independent parameters (e.g. name, size, and number) to be stored within instances of the description ''structs'', and frame-dependent values (e.g. position and orientation) to be stored within instances of the frame data ''structs''. When needed, two different packets of an asset can be correlated by referencing to its unique identifier values.
 
For every asset (e.g. reconstructed markers, rigid bodies, skeletons, force plates) included within streamed capture sessions, their descriptions and tracking data are stored separately. This format allows frame-independent parameters (e.g. name, size, and number) to be stored within instances of the description ''structs'', and frame-dependent values (e.g. position and orientation) to be stored within instances of the frame data ''structs''. When needed, two different packets of an asset can be correlated by referencing to its unique identifier values.
 
When streaming from Motive, received NatNet data will contain only the assets that are enabled in the [[Project pane]] and the asset types that are set to ''true'' under Streaming Settings in the [[Data Streaming pane]].
 
  
 
{{Indent|
 
{{Indent|
Line 21: Line 19:
 
}}
 
}}
  
 +
 +
{{Info|When streaming from Motive, received NatNet data will contain only the assets that are enabled in the [[Project pane]] and the asset types that are set to ''true'' under Streaming Settings in the [[Data Streaming pane]].}}
  
 
=Dataset Descriptions=
 
=Dataset Descriptions=
Line 27: Line 27:
 
To receive data descriptions from a connected server, use the [[NatNet: Class/Function Reference#NatNetClient::GetDataDescriptionList|NatNetClient::GetDataDescriptionList]] method. Calling this function saves a list of available descriptions in an instance of sDataSetDescriptions.
 
To receive data descriptions from a connected server, use the [[NatNet: Class/Function Reference#NatNetClient::GetDataDescriptionList|NatNetClient::GetDataDescriptionList]] method. Calling this function saves a list of available descriptions in an instance of sDataSetDescriptions.
  
The '''sDataSetDescriptions''' structure stores an array of multiple descriptions for each of assets (MarkerSets, RigidBodies, Skeletons, and Force Plates) involved in a capture and necessary information can be parsed from it. The following table lists out the main data description ''structs'' that are available through the SDK. For a complete list of structs and their members,  are listed in the [[NatNet SDK#File List|NatNetTypes.h]] header file.
+
The '''sDataSetDescriptions''' structure stores an array of multiple descriptions for each of assets (MarkerSets, RigidBodies, Skeletons, and Force Plates) involved in a capture and necessary information can be parsed from it. The following table lists out the main data description ''structs'' that are available through the SDK.  
  
{| class = "wikitable" style = "width:95%; margin: auto"
+
{{Info|Refer to the [[NatNet SDK#File List|NatNetTypes.h]] header file for more information on each data type and members of each description struct. }}
 +
 
 +
{| class = "wikitable" style = "width:95%; margin: auto;"
 
|+'''Description Struct'''
 
|+'''Description Struct'''
 
|-
 
|-
Line 42: Line 44:
 
|style = "padding: 0.5em;"|''sServerDescription''
 
|style = "padding: 0.5em;"|''sServerDescription''
 
|style = "padding: 0.5em;"|''ServerDescription''
 
|style = "padding: 0.5em;"|''ServerDescription''
|style = "padding: 1em;"| Contains basic network information of the connected server application and the host computer that it is running on. Server descriptions are obtained by calling the GetServerDescription method.
+
|style = "padding: 1em;"| Contains basic network information of the connected server application and the host computer that it is running on. Server descriptions are obtained by calling the GetServerDescription method from the NatNetClient class.
|-
+
 
 +
* Host connection status
 +
* Host information (computer name, IP, server app name)
 +
* NatNet version
 +
* Host's high resolution clock frequency. Used for calculating the latency
 +
* Connection status
 +
 
 +
|- style = "background:white;"
 
|style = "padding: 1em;"|Data Descriptions
 
|style = "padding: 1em;"|Data Descriptions
 
|style = "padding: 0.5em;"|''sDataDescriptions''
 
|style = "padding: 0.5em;"|''sDataDescriptions''
 
|style = "padding: 0.5em;"|''List<DataDescriptor>''
 
|style = "padding: 0.5em;"|''List<DataDescriptor>''
|style = "padding: 1em;"| Contains a list of multiple DataDescriptions each storing description objects for corresponding assets.
+
|style = "padding: 1em;"|Contains an array of data descriptions for each active asset in a capture, and basic information about corresponding asset is stored in each description packet. Data descriptions are obtained by calling the GetDataDescriptions method from the NatNetClient class. Descriptions of each asset type is explained below.
 
|-
 
|-
 
|style = "padding: 1em;"|MarkerSet Description
 
|style = "padding: 1em;"|MarkerSet Description
Line 53: Line 62:
 
|style = "padding: 0.5em;"|''MarkerSet''
 
|style = "padding: 0.5em;"|''MarkerSet''
 
|style = "padding: 1em;"| MarkerSet description contains a total number of markers in a [[Assets|MarkerSet]] and each of their labels. Note that rigid body and skeleton assets are included in the MarkerSet as well. Also, for every mocap session, there is a spcial MarkerSet named ''all'', which contains a list of all of the labeld markers from the capture.
 
|style = "padding: 1em;"| MarkerSet description contains a total number of markers in a [[Assets|MarkerSet]] and each of their labels. Note that rigid body and skeleton assets are included in the MarkerSet as well. Also, for every mocap session, there is a spcial MarkerSet named ''all'', which contains a list of all of the labeld markers from the capture.
|-
+
 
 +
* Name of the markerset
 +
* Number of markers in the set
 +
* Marker names
 +
 
 +
|- style = "background:white;"
 
|style = "padding: 1em;"|Rigid Body Description
 
|style = "padding: 1em;"|Rigid Body Description
 
|style = "padding: 0.5em;"|''sRigidBodyDescription''
 
|style = "padding: 0.5em;"|''sRigidBodyDescription''
 
|style = "padding: 0.5em;"|''RigidBody''
 
|style = "padding: 0.5em;"|''RigidBody''
 
|style = "padding: 1em;"| Rigid body description contains corresponding rigid body names. Skeleton bones are also considered as rigid bodies, and in this case, the description also contains hierarchial relationship for parent/chile rigid bodies.  
 
|style = "padding: 1em;"| Rigid body description contains corresponding rigid body names. Skeleton bones are also considered as rigid bodies, and in this case, the description also contains hierarchial relationship for parent/chile rigid bodies.  
 +
 +
* Rigid body name
 +
* Rigid body streaming ID
 +
* Rigid body parent ID (when streaming skeleton as rigid bodies)
 +
* Offset displacement from the parent rigid body
 +
* Array of marker locations that represent the expected marker locations of the rigid body asset.
 
|-
 
|-
 
|style = "padding: 1em;"|Skeleton Description
 
|style = "padding: 1em;"|Skeleton Description
Line 63: Line 83:
 
|style = "padding: 0.5em;"|''Skeleton''
 
|style = "padding: 0.5em;"|''Skeleton''
 
|style = "padding: 1em;"| Skeleton description contains corresponding skeleton asset name, skeleton ID, and total number of rigid bodies (bones) involved in the asset. The skeleton desciption also contains an array of rigid body descriptions which relates to individual bones of the corresponding skeleton.
 
|style = "padding: 1em;"| Skeleton description contains corresponding skeleton asset name, skeleton ID, and total number of rigid bodies (bones) involved in the asset. The skeleton desciption also contains an array of rigid body descriptions which relates to individual bones of the corresponding skeleton.
|-
+
 
 +
* Name of the skeleton
 +
* Skeleton ID: Unique identifier
 +
* Number of rigid bodies (bones)
 +
* Array of bone descriptions
 +
 
 +
 
 +
{{Info|'''Update Note:''' In NatNet 3.0, Skeleton bone data description packet has been changed from left-handed convention to right-handed convention to be consistent with the convention used in all other data packets. For older versions of NatNet clients, the server, Motive, will detect the client version and stream out skeleton data in the matching convention. This change will only affect direct-depacketization clients as well as clients that have the NatNet library upgraded to 3.0 from previous versions; for those clients, corresponding changes must be made to work with Motive 2.0.}}
 +
|-  style = "background:white;"
 
|style = "padding: 1em;"|Force Plate Description
 
|style = "padding: 1em;"|Force Plate Description
 
|style = "padding: 0.5em;"|''sForcePlateDescription''
 
|style = "padding: 0.5em;"|''sForcePlateDescription''
 
|style = "padding: 0.5em;"|''ForcePlate''
 
|style = "padding: 0.5em;"|''ForcePlate''
 
|style = "padding: 1em;"| Force plate description contains names and IDs of the plate and its channels as well as other hardware parameter settings. Please refer to the [[NatNet SDK#File List|NatNetTypes.h]] header file for specific details.
 
|style = "padding: 1em;"| Force plate description contains names and IDs of the plate and its channels as well as other hardware parameter settings. Please refer to the [[NatNet SDK#File List|NatNetTypes.h]] header file for specific details.
 +
 +
* Force plate ID and serial number
 +
* Force plate dimensions
 +
* Electrical offset
 +
* Number of channels
 +
* Channel info
 +
* More. See ''NatNetTypes.h'' file for more information
 
|-
 
|-
 
|style = "padding: 1em;"|Device Description
 
|style = "padding: 1em;"|Device Description
Line 73: Line 108:
 
|style = "padding: 0.5em;"|''Device''
 
|style = "padding: 0.5em;"|''Device''
 
|style = "padding: 1em;"|An instance of the sDeviceDescription contains information of the data acquisition (NI-DAQ) devices. It includes information on both the DAQ device (ID, name , serial number) as well as its corresponding channels (channel count, channel data type, channel names). Please refer to the [[NatNet SDK#File List|NatNetTypes.h]] header file for specific details.
 
|style = "padding: 1em;"|An instance of the sDeviceDescription contains information of the data acquisition (NI-DAQ) devices. It includes information on both the DAQ device (ID, name , serial number) as well as its corresponding channels (channel count, channel data type, channel names). Please refer to the [[NatNet SDK#File List|NatNetTypes.h]] header file for specific details.
 +
 +
* Device ID. Used only for identification of devices in the stream.
 +
* Device Name
 +
* Device serial number
 +
* Device Type
 +
* Channel count
 +
* Channel Names
 
|}
 
|}
  
Line 80: Line 122:
 
As mentioned in the beginning, frame-specific tracking data are stored separately from the DataDescription instances as this cannot be known ahead of time or out of band but only by per frame basis. These data gets saved into instances of '''sFrameOfMocapData''' for corresponding frames, and they will contain arrays of frame-specific data ''structs'' (e.g.sRigidBodyData, sSkeletonData) for each types of assets included in the capture. Respective frame number, timecode, and streaming latency values are also saved in these packets.
 
As mentioned in the beginning, frame-specific tracking data are stored separately from the DataDescription instances as this cannot be known ahead of time or out of band but only by per frame basis. These data gets saved into instances of '''sFrameOfMocapData''' for corresponding frames, and they will contain arrays of frame-specific data ''structs'' (e.g.sRigidBodyData, sSkeletonData) for each types of assets included in the capture. Respective frame number, timecode, and streaming latency values are also saved in these packets.
  
The sFrameOfMocapData can be obtained by setting up a frame handler function using the [[NatNet: Class/Function Reference#NatNetClient::SetFrameReceivedCallback|NatNetClient::SetFrameReceivedCallback]] method or by calling the GetLastFrameOfData() method. In most cases, a frame handler function must be assigned in order to make sure every frames are promptly processed. Refer to the provided [[NatNet: Sample Projects#NatNet Sample Projects|''SampleClient'']] project for an exemplary setup.
+
The sFrameOfMocapData can be obtained by setting up a frame handler function using the [[NatNet: Class/Function Reference#NatNetClient::SetFrameReceivedCallback|NatNetClient::SetFrameReceivedCallback]] method. In most cases, a frame handler function must be assigned in order to make sure every frames are promptly processed. Refer to the provided [[NatNet: Sample Projects#NatNet Sample Projects|''SampleClient'']] project for an exemplary setup.
  
  
 
{| class = "wikitable" style = "width:95%; margin: auto"
 
{| class = "wikitable" style = "width:95%; margin: auto"
 
|+'''FrameOfMocapData'''
 
|+'''FrameOfMocapData'''
!rowspan = 2; style = "padding: 1em; width:10%"|Data Type
+
!rowspan = 2; style = "padding: 0.5em; text-align: center"|Data
!colspan = 2; style = "padding: 1em; width:10%"|Saved ''struct'' Type
+
!colspan = 2; style = "padding: 1em; width:10%"|Variable name
 
!rowspan = 2; style = "padding: 1em;"|Description
 
!rowspan = 2; style = "padding: 1em;"|Description
 
|-
 
|-
!style = "padding: 1em; width:5%;"|Native Library
+
!style = "padding: 1em; width:5%;"|Native
!style = "padding: 1em; width:5%"|Managed Assembly
+
!style = "padding: 1em; width:5%"|Managed
 
|-
 
|-
|style = "padding: 1em;"|MarkerSet Data
+
|style = "padding: 0.5em; text-align: center"|Frame Count
|style = "padding: 0.5em;"|''sMarkerSetData''
+
| colspan = 2; style = "padding: 0.5em; text-align: center"|''iFrame''
|style = "padding: 0.5em;"|''MarkerSetData''
+
|style = "padding: 1em;"|Host (server) defined frame number.
|style = "padding: 1em;"|A named collection of identified markers and the marker positions (X, Y, Z). This list is ordered, padded, point cloud solved, model filled (where occluded). This list also contains special MarkerSet named "all" which is a list of all labeled markers.
+
|- style = "background:white;"
 +
|rowspan = 2; style = "padding: 0.5em;text-align: center"|Labeled Markers
 +
|style = "padding: 0.5em; text-align: center"|''nLabeledMarkers''
 +
|style = "padding: 0.5em; text-align: center"|''nMarkers''
 +
|style = "padding: 1em;"|A total number of labeled markers in the frame.
 +
|- style = "background:white;"
 +
|colspan = 2; style = "padding: 0.5em;text-align: center"|''LabeledMarkers''
 +
|style = "padding: 1em;"|A list of ordered, padded, point cloud solved, labeled marker data. The data includes the unique ID, x/y/z positions, marker size, and its residual value from reconstruction. 
 +
 
 +
* The labeled marker data is solved by the point cloud engine only.
 +
* '''Unique ID:''' For Active markers, this value represents the Active ID. For passive markers labeled through an asset, this represents both Asset ID (High-bit) and Member ID (Low-bit). For unlabeled markers, this is just arbitrary value assigned by the Point Cloud reconstruction engine.
 +
* '''Active or Passive''': Within the host defined 16-bit integer ''param'' value, the sixth bit represents whether the marker is active or passive. This is demonstrated in SampleClient application. e.g.) <code>bActiveMarker = ((data->LabeledMarkers[i].params & 0x20) == 0);</code>
 +
|-
 +
|rowspan = 2; style = "padding: 0.5em;text-align: center"|Unlabeled Markers
 +
|colspan = 2; style = "padding: 0.5em;text-align: center"|''nOtherMarkers''
 +
|style = "padding: 1em;"|A total number of unlabeled markers in the frame.
 
|-
 
|-
|style = "padding: 1em;"|Unlabeled Markers (Other markers)
+
|colspan = 2; style = "padding: 0.5em;text-align: center"|''OtherMarkers''
|style = "padding: 0.5em;"|''MarkerData''
+
|style = "padding: 1em;"|A list of point cloud solved 3D positions (X, Y, Z) for all ''unlabeled'' markers in the frame.
|style = "padding: 0.5em;"|''Marker''
+
 
|style = "padding: 1em;"|List of point cloud solved 3D positions (X, Y, Z) for all markers in the frame that are not labeled.
+
* The unlabeled marker data is solved by the point cloud engine only.
 +
|- style = "background:white;"
 +
|rowspan = 2; style = "padding: 0.5em; text-align: center"|MarkerSet Data
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''nMarkerSets''
 +
|style = "padding: 1em;"|A total number of markersets.
 +
|- style = "background:white;"
 +
|style = "padding: 0.5em; text-align: center"|''MocapData''
 +
|style = "padding: 0.5em; text-align: center"|''MarkerSets''
 +
|style = "padding: 1em;"|
 +
A collection of MarkerSets (MarkerSet, Rigid Body, or Skeletons) in the frame. The struct includes name, number of involved markers, and their corresponding X/Y/Z locations.
 +
 
 +
* In this data, the corresponding 3D marker locations are point cloud solved and model-filled on occluded frames.  
 
|-
 
|-
|style = "padding: 1em;"|Labeled Markers
+
|rowspan = 2; style = "padding: 0.5em;text-align: center"|Rigid Body Data
|style = "padding: 0.5em;"|''sMarker''
+
|colspan = 2; style = "padding: 0.5em;text-align: center"|''nRigidBodies''
|style = "padding: 0.5em;"|''Marker''
+
|style = "padding: 1em;"|A total number of rigid body assets, both tracked and untracked, in the frame.
|style = "padding: 1em;"|Ordered, padded, point cloud solved, model filled (where occluded) labeled Marker data (labeled markers not associated with a "MarkerSet").
 
 
|-
 
|-
|style = "padding: 1em;"|RigidBody Data
+
|style = "padding: 0.5em; text-align: center"|''sRigidBodyData''
|style = "padding: 0.5em;"|''sRigidBodyData''
+
|style = "padding: 0.5em; text-align: center"|''RigidBodyData''
|style = "padding: 0.5em;"|''RigidBodyData''
+
|style = "padding: 1em;"|A named segment with a unique ID, position, and orientation data. For skeletons rigid bodies, this will represent one of the segments on a skeleton asset.
|style = "padding: 1em;"|A named segment with a unique ID, position, and orientation data, and the collection of identified markers used to define it. Its marker data indicates model-solved positions.
+
 
 +
* '''Unique ID''': For rigid body assets, Rigid body ID is the streaming ID assigned under the rigid body properties in Motive. For skeleton assets, this ID is both the Skeleton ID (High-bit) and the Bone index ID (Low-bit).
 +
|- style = "background:white;"
 +
|rowspan = 2; style = "padding: 0.5em;text-align: center"|Skeleton Data
 +
|colspan = 2; style = "padding: 0.5em;text-align: center"|''nSkeletons''
 +
|style = "padding: 1em;"|A total number of skeleton assets, both tracked and untracked, in the frame.
 +
|- style = "background:white;"
 +
|style = "padding: 0.5em; text-align: center"|sSkeletonData
 +
|style = "padding: 0.5em; text-align: center"|SkeletonData
 +
|style = "padding: 1em;"|A named, hierarchical collection of RigidBody data in sRigidBodyData struct.
 +
 
 +
*'''Unique ID:''' A unqiue ID is assigned to each skeleton so that it could be referenced.
 
|-
 
|-
|style = "padding: 1em;"|Skeleton Data
+
|rowspan = 2; style = "padding: 0.5em; text-align: center"|Force Plate Data
|style = "padding: 0.5em;"|''sSkeletonData''
+
|colspan = 2; style = "padding: 0.5em; text-align: center"|''nForcePlates''
|style = "padding: 0.5em;"|''SkeletonData''
+
|style = "padding: 1em;"|A total number of force plates.
|style = "padding: 1em;"|A named, hierarchical collection of RigidBodies.  Marker data is model-solved positions.
 
 
|-
 
|-
|style = "padding: 1em;"|Force Plate Data
+
|colspan=2 ; style = "padding: 0.5em; text-align: center"|''ForcePlates''
|style = "padding: 0.5em;"|''sForcePlateData''
+
|style = "padding: 1em;"|Force plate channel data (Fx, Fy, Fz, Mx, My, Mz). Each channel data is saved as an instance of the sAnalogchannelData which contains values measured from corresponding channel as well as the total number of analog subframes contained per mocap frame.  Force plate data will contain multiple samples per mocap frame, depending upon the force plate acquisition rate. The total number of subframes per mocap frame can be quiried from  a AnalogChannelData instance of each channel.
|style = "padding: 0.5em;"|''ForcePlateData''
+
|- style = "background:white;"
|style = "padding: 1em;"|Force plate channel data (Fx, Fy, Fz, Mx, My, Mz).  Force plate data will contain multiple samples per mocap frame, depending upon the force plate acquisition rate.
+
|rowspan = 2; style = "padding: 0.5em; text-align: center"|Device Data
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''nDevices''
 +
|style = "padding: 1em;"|A total number of analog devices in the capture. (e.g. NI-DAQ)
 +
|- style = "background:white;"
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''Devices''
 +
|style = "padding: 1em;"|An array containing data from each of analog device channels (e.g. NI-DAQ). Each channel data will be saved as an instance of the sAnalogchannelData which contains values measured from corresponding channel as well as the total number of analog subframes contained per mocap frame.
 
|-
 
|-
|style = "padding: 1em;"|Frame Number
+
|rowspan = 2; style = "padding: 0.5em; text-align: center"|Latency
|style = "padding: 0.5em;"|''int''
+
|colspan = 2; style = "padding: 0.5em; text-align: center"|''fSystemLatency''
|style = "padding: 0.5em;"|''int''
+
|style = "padding: 1em;"|<div style="color:#ff0000">''(Deprecated)''</div> Now, more accurate system latency values can be derived from the reported timestamp values. For more information, read through the [[Latency Measurements]] page.
|style = "padding: 1em;"|Host (server) defined frame number.
 
 
|-
 
|-
|style = "padding: 1em;"|Latency
+
|colspan = 2; style = "padding: 0.5em; text-align: center"|''fLatency''
|style = "padding: 0.5em;"|''float''
+
|style = "padding: 1em;"|<div style="color:#ff0000">''(Deprecated)''</div> Now, more accurate software latency values can be derived from the reported timestamp values. For more information, read through the [[Latency Measurements]] page.
|style = "padding: 0.5em;"|''float''
+
|- style = "background:white;"
|style = "padding: 1em;"|For every sFrameOfMocapData received through NatNet, a latency value is reported:
+
|rowspan = 6; style = "padding: 0.5em; text-align: center"|Time Information
 
+
|colspan = 2; style = "padding: 0.5em; text-align: center"|''Timecode''
*(For NatNet versions 2.11 and above) the latency value represents the system latency, which is the calculated time between the camera hardware exposure and when Motive is ready to stream the data out; in other words, combining both the hardware and software latency.
+
|style = "padding: 1em;"|Timing information for the frame. If SMPTE timecode is detected in the system, this time information is also included.  See: [[OptiTrack Timecode]]
 
 
*(For NatNet 2.10) the latency value represents the software processing latency, which is the calculated time between when Motive receives frame data from all of the connected cameras to the time when it is ready to stream the data out.
 
 
 
*(For NatNet versions below 2.10) the latency was used to report the capture computer’s hardware timestamp for the given frame. This was kept in for compatibility reasons.
 
|-
 
|style = "padding: 1em;"|Timestamp data
 
|style = "padding: 0.5em;"|''unsigned int''
 
|style = "padding: 0.5em;"|''uint''
 
|style = "padding: 1em;"|Timing information for the frame. If SMPTE timecode is detected in the system, this time code is also included.  See: [[OptiTrack Timecode]]
 
 
* Frame ID
 
* Frame ID
 
* Frame Timestamp
 
* Frame Timestamp
 
* SMPTE Timecode (If timecode is present)
 
* SMPTE Timecode (If timecode is present)
|-
+
|- style = "background:white;"
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''TimecodeSubframe''
 +
|style = "padding: 1em;"|The subframe value of the timecode. See: [[OptiTrack Timecode]]
 +
|- style = "background:white;"
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''fTimestamp''
 +
|style = "padding: 1em;"|Software timestamp value. Reports the time since software start.
 +
|- style = "background:white;"
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''CameraMidExposureTimestamp''
 +
|style = "padding: 1em;"|Given in host's high resolution ticks, this stores a timestamp value of when the cameras expose. The timestamp precisely indicates the center of the exposure window. For more information, refer to the [[Latency Measurements]] article.
 +
|- style = "background:white;"
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''CameraDataReceivedTimestamp''
 +
|style = "padding: 1em;"|Given in host's high resolution ticks, this  stores a timestamp value of when Motive receives the camera data. For more information, refer to the [[Latency Measurements]] article.
 +
|- style = "background:white;"
 +
|colspan = 2; style = "padding: 0.5em; text-align: center"|''TransmitTimestamp''
 +
|style = "padding: 1em;"|Given in host's high resolution ticks, this stores a timestamp value of when tracking data is fully processed and ready to be streamed out. For more information, refer to the [[Latency Measurements]] article.
 
|}
 
|}
 
  
 
<div class="padded">
 
<div class="padded">
Line 157: Line 242:
  
  
{{Info|Refer to the NatNetTypes.h header file or the NatNetML.dll assembly for the most up to date descriptions of the types.}}
+
{{Indent|{{Info|Refer to the NatNetTypes.h header file or the NatNetML.dll assembly for the most up to date descriptions of the types.}}}}
 
</div>
 
</div>
 
  
 
=Unique ID=
 
=Unique ID=
Line 165: Line 249:
  
 
Most of the NatNet SDK data packets contain ID values. This value is assigned uniquely to individual markers as well as each of assets within a capture. These values can be used to figure out which asset a given data packet is associated with. One common use is for correlating data descriptions and frame data packets of an asset.
 
Most of the NatNet SDK data packets contain ID values. This value is assigned uniquely to individual markers as well as each of assets within a capture. These values can be used to figure out which asset a given data packet is associated with. One common use is for correlating data descriptions and frame data packets of an asset.
 
  
 
<div class="padded">
 
<div class="padded">
'''Decoding Member IDs'''
+
====Decoding Member IDs====
 
<div class="padded">
 
<div class="padded">
 
For each member object that is included within a parental model, its unique ID value points to both its parental model and the member itself. Thus, the ID value of a member object needs to be decoded in order to parse which objects and the parent models they are referencing to.  
 
For each member object that is included within a parental model, its unique ID value points to both its parental model and the member itself. Thus, the ID value of a member object needs to be decoded in order to parse which objects and the parent models they are referencing to.  

Latest revision as of 22:09, 28 May 2021

Main pageNatNet SDKNatNet: Data Types


This page provides an overview of the general data structure used in the NatNet software development kit (SDK) and how the library is used to parse received tracking information.

Info2.png

For specific details on each of the data types, please refer to the NatNetTypes.h header file.

General Structure


When receiving streamed data using the NatNet SDK library, its data descriptions should be received before receiving the tracking data. NatNet data is packaged mainly into two different formats: data descriptions and frame-specific tracking data. Utilizing this format, the client application can discover which data are streamed out from the server application in advance to accessing the actual tracking data.

For every asset (e.g. reconstructed markers, rigid bodies, skeletons, force plates) included within streamed capture sessions, their descriptions and tracking data are stored separately. This format allows frame-independent parameters (e.g. name, size, and number) to be stored within instances of the description structs, and frame-dependent values (e.g. position and orientation) to be stored within instances of the frame data structs. When needed, two different packets of an asset can be correlated by referencing to its unique identifier values.

  • Dataset Descriptions contains descriptions of the motion capture data sets for which a frame of motion capture data will be generated. (e.g. sSkeletonDescription, sRigidBodyDescription)
  • Frame of Mocap Data contains a single frame of motion capture data for all the datasets described from the Dataset Descriptions. (e.g. sSkeletonData, sRigidBodyData)


Info2.png

When streaming from Motive, received NatNet data will contain only the assets that are enabled in the Project pane and the asset types that are set to true under Streaming Settings in the Data Streaming pane.

Dataset Descriptions


To receive data descriptions from a connected server, use the NatNetClient::GetDataDescriptionList method. Calling this function saves a list of available descriptions in an instance of sDataSetDescriptions.

The sDataSetDescriptions structure stores an array of multiple descriptions for each of assets (MarkerSets, RigidBodies, Skeletons, and Force Plates) involved in a capture and necessary information can be parsed from it. The following table lists out the main data description structs that are available through the SDK.

Info2.png

Refer to the NatNetTypes.h header file for more information on each data type and members of each description struct.

Description Struct
Data Type Saved struct Type Description
Native Library Managed Assembly
Server Description sServerDescription ServerDescription Contains basic network information of the connected server application and the host computer that it is running on. Server descriptions are obtained by calling the GetServerDescription method from the NatNetClient class.
  • Host connection status
  • Host information (computer name, IP, server app name)
  • NatNet version
  • Host's high resolution clock frequency. Used for calculating the latency
  • Connection status
Data Descriptions sDataDescriptions List<DataDescriptor> Contains an array of data descriptions for each active asset in a capture, and basic information about corresponding asset is stored in each description packet. Data descriptions are obtained by calling the GetDataDescriptions method from the NatNetClient class. Descriptions of each asset type is explained below.
MarkerSet Description sMarkerSetDescription MarkerSet MarkerSet description contains a total number of markers in a MarkerSet and each of their labels. Note that rigid body and skeleton assets are included in the MarkerSet as well. Also, for every mocap session, there is a spcial MarkerSet named all, which contains a list of all of the labeld markers from the capture.
  • Name of the markerset
  • Number of markers in the set
  • Marker names
Rigid Body Description sRigidBodyDescription RigidBody Rigid body description contains corresponding rigid body names. Skeleton bones are also considered as rigid bodies, and in this case, the description also contains hierarchial relationship for parent/chile rigid bodies.
  • Rigid body name
  • Rigid body streaming ID
  • Rigid body parent ID (when streaming skeleton as rigid bodies)
  • Offset displacement from the parent rigid body
  • Array of marker locations that represent the expected marker locations of the rigid body asset.
Skeleton Description sSkeletonDescription Skeleton Skeleton description contains corresponding skeleton asset name, skeleton ID, and total number of rigid bodies (bones) involved in the asset. The skeleton desciption also contains an array of rigid body descriptions which relates to individual bones of the corresponding skeleton.
  • Name of the skeleton
  • Skeleton ID: Unique identifier
  • Number of rigid bodies (bones)
  • Array of bone descriptions


Info2.png

Update Note: In NatNet 3.0, Skeleton bone data description packet has been changed from left-handed convention to right-handed convention to be consistent with the convention used in all other data packets. For older versions of NatNet clients, the server, Motive, will detect the client version and stream out skeleton data in the matching convention. This change will only affect direct-depacketization clients as well as clients that have the NatNet library upgraded to 3.0 from previous versions; for those clients, corresponding changes must be made to work with Motive 2.0.

Force Plate Description sForcePlateDescription ForcePlate Force plate description contains names and IDs of the plate and its channels as well as other hardware parameter settings. Please refer to the NatNetTypes.h header file for specific details.
  • Force plate ID and serial number
  • Force plate dimensions
  • Electrical offset
  • Number of channels
  • Channel info
  • More. See NatNetTypes.h file for more information
Device Description sDeviceDescription Device An instance of the sDeviceDescription contains information of the data acquisition (NI-DAQ) devices. It includes information on both the DAQ device (ID, name , serial number) as well as its corresponding channels (channel count, channel data type, channel names). Please refer to the NatNetTypes.h header file for specific details.
  • Device ID. Used only for identification of devices in the stream.
  • Device Name
  • Device serial number
  • Device Type
  • Channel count
  • Channel Names

Frame of Mocap Data


As mentioned in the beginning, frame-specific tracking data are stored separately from the DataDescription instances as this cannot be known ahead of time or out of band but only by per frame basis. These data gets saved into instances of sFrameOfMocapData for corresponding frames, and they will contain arrays of frame-specific data structs (e.g.sRigidBodyData, sSkeletonData) for each types of assets included in the capture. Respective frame number, timecode, and streaming latency values are also saved in these packets.

The sFrameOfMocapData can be obtained by setting up a frame handler function using the NatNetClient::SetFrameReceivedCallback method. In most cases, a frame handler function must be assigned in order to make sure every frames are promptly processed. Refer to the provided SampleClient project for an exemplary setup.


FrameOfMocapData
Data Variable name Description
Native Managed
Frame Count iFrame Host (server) defined frame number.
Labeled Markers nLabeledMarkers nMarkers A total number of labeled markers in the frame.
LabeledMarkers A list of ordered, padded, point cloud solved, labeled marker data. The data includes the unique ID, x/y/z positions, marker size, and its residual value from reconstruction.
  • The labeled marker data is solved by the point cloud engine only.
  • Unique ID: For Active markers, this value represents the Active ID. For passive markers labeled through an asset, this represents both Asset ID (High-bit) and Member ID (Low-bit). For unlabeled markers, this is just arbitrary value assigned by the Point Cloud reconstruction engine.
  • Active or Passive: Within the host defined 16-bit integer param value, the sixth bit represents whether the marker is active or passive. This is demonstrated in SampleClient application. e.g.) bActiveMarker = ((data->LabeledMarkers[i].params & 0x20) == 0);
Unlabeled Markers nOtherMarkers A total number of unlabeled markers in the frame.
OtherMarkers A list of point cloud solved 3D positions (X, Y, Z) for all unlabeled markers in the frame.
  • The unlabeled marker data is solved by the point cloud engine only.
MarkerSet Data nMarkerSets A total number of markersets.
MocapData MarkerSets

A collection of MarkerSets (MarkerSet, Rigid Body, or Skeletons) in the frame. The struct includes name, number of involved markers, and their corresponding X/Y/Z locations.

  • In this data, the corresponding 3D marker locations are point cloud solved and model-filled on occluded frames.
Rigid Body Data nRigidBodies A total number of rigid body assets, both tracked and untracked, in the frame.
sRigidBodyData RigidBodyData A named segment with a unique ID, position, and orientation data. For skeletons rigid bodies, this will represent one of the segments on a skeleton asset.
  • Unique ID: For rigid body assets, Rigid body ID is the streaming ID assigned under the rigid body properties in Motive. For skeleton assets, this ID is both the Skeleton ID (High-bit) and the Bone index ID (Low-bit).
Skeleton Data nSkeletons A total number of skeleton assets, both tracked and untracked, in the frame.
sSkeletonData SkeletonData A named, hierarchical collection of RigidBody data in sRigidBodyData struct.
  • Unique ID: A unqiue ID is assigned to each skeleton so that it could be referenced.
Force Plate Data nForcePlates A total number of force plates.
ForcePlates Force plate channel data (Fx, Fy, Fz, Mx, My, Mz). Each channel data is saved as an instance of the sAnalogchannelData which contains values measured from corresponding channel as well as the total number of analog subframes contained per mocap frame. Force plate data will contain multiple samples per mocap frame, depending upon the force plate acquisition rate. The total number of subframes per mocap frame can be quiried from a AnalogChannelData instance of each channel.
Device Data nDevices A total number of analog devices in the capture. (e.g. NI-DAQ)
Devices An array containing data from each of analog device channels (e.g. NI-DAQ). Each channel data will be saved as an instance of the sAnalogchannelData which contains values measured from corresponding channel as well as the total number of analog subframes contained per mocap frame.
Latency fSystemLatency
(Deprecated)
Now, more accurate system latency values can be derived from the reported timestamp values. For more information, read through the Latency Measurements page.
fLatency
(Deprecated)
Now, more accurate software latency values can be derived from the reported timestamp values. For more information, read through the Latency Measurements page.
Time Information Timecode Timing information for the frame. If SMPTE timecode is detected in the system, this time information is also included. See: OptiTrack Timecode
  • Frame ID
  • Frame Timestamp
  • SMPTE Timecode (If timecode is present)
TimecodeSubframe The subframe value of the timecode. See: OptiTrack Timecode
fTimestamp Software timestamp value. Reports the time since software start.
CameraMidExposureTimestamp Given in host's high resolution ticks, this stores a timestamp value of when the cameras expose. The timestamp precisely indicates the center of the exposure window. For more information, refer to the Latency Measurements article.
CameraDataReceivedTimestamp Given in host's high resolution ticks, this stores a timestamp value of when Motive receives the camera data. For more information, refer to the Latency Measurements article.
TransmitTimestamp Given in host's high resolution ticks, this stores a timestamp value of when tracking data is fully processed and ready to be streamed out. For more information, refer to the Latency Measurements article.

Additional Notes

  • One reconstructed 3D marker can be stored in two different places (e.g. in LabeledMarkers and in RigidBody) within a frame of mocap data. In those cases, unique identifier values of the marker can be used to correlate them in the client application if necessary.
  • Declarations for these data types are listed in the NatNetTypes.h header files within the SDK. The SampleClient project included in the \NatNet SDK\Sample folder illustrates how to retrieve and interpret the data descriptions and frame data.


Info2.png

Refer to the NatNetTypes.h header file or the NatNetML.dll assembly for the most up to date descriptions of the types.

Unique ID


Most of the NatNet SDK data packets contain ID values. This value is assigned uniquely to individual markers as well as each of assets within a capture. These values can be used to figure out which asset a given data packet is associated with. One common use is for correlating data descriptions and frame data packets of an asset.

Decoding Member IDs

For each member object that is included within a parental model, its unique ID value points to both its parental model and the member itself. Thus, the ID value of a member object needs to be decoded in order to parse which objects and the parent models they are referencing to.

For example, a skeleton asset is a hierarchical collection of bone rigid bodies, and each of its bone rigid bodies has unique ID that references to the involved skeleton model and the rigid body itself. When analyzing skeleton bones, its ID value needs to be decoded in order to extract the segment rigid body ID, and only then, it can be used to reference its descriptions.