Live Streaming in Unreal Engine 5

Axis Software Setup

You have the option to either stream real-time data or playback data, so first, either get your mocap setup connected and ready to record, or put a pre-recorded take on loop.  If you are using Axis Studio, you can do both at once.   

Axis Neuron/Pro Settings

  • Go to File > Settings >Output Format and enable Displacement
  • In the Settings menu select Broadcasting and enable TCP or UDP, both are supported, there are just minor setup differences in the setup in unreal
  • Enable BVH and select Binary format

Axis Studio Settings

  • Go to the Main Menu>Settings.
  • Select BVH Broadcasting.
  • Enable BVH-Capture, BVH-Edit, or both:
    • Capture is for live data from your mocap suit
    • Edit is for pre-recorded data
  • Enable: Displacement 
  • Protocol: TCP or UDP
  • Skeleton:
    • Axis Studio Skeleton (Axis Studio - Thumb Open if you are using gloves) if you are broadcasting to a character in T - Pose
    • Unreal Engine skeleton if you are broadcasting to an Unreal character in A - pose

mceclip1.png

Plugin Setup

Installation

To receive Perception Neuron data from Axis Software you will need to install our NeuronLiveLink. You'll find information on downloading and installing the plugin in the following article

Neuron Live link Installation – PERCEPTION NEURON (neuronmocap.com)

Link Data

  1. In order to establish a connection in our Neuron Livelink, Axis software data must be active and properly setup prior to proceeding to the next step.
  2. In your UE Editor, go to Windows >Virtual Production>Live link. 
  3. Click Source --> Axis Neuron Live to view the following window options. Neuron Live Link supports UDP or TCP protocol to receive mocap data.mceclip3.png
    1. UDP: Update the local address number to match Axis Studio and make sure the port numbers match
    2. TCP: Deselect Is UDP. Update the Remote Address to match Axis studio.
  4. Data from Axis Studio data should appear under Subject Name which will indicate the connection was successful. The subject name will match the name of the character in Axis Studio.

Retargeting Axis Data to a Character

Next, you can either use the Noitom Robot skeleton that comes with the plugin, the UE4 Mannequin, or import your own FBX character.

  • Import a character into the content browser.
  • If you are importing a character:
    • Make sure Skeletal Mesh is enabled
    • Under Skeleton select none
    • Enable Use T0 as Reference Pose and Import Morph Targetsmceclip5.png
  • The import should create a few assets needed for real-time streaming:
    • Skeletal Mesh - Magenta Color
    • Skeleton - Blue Colormceclip6.png

Animation Blueprint for live sessions 

Create an Anim Blueprint to tell our character to use the Live Link data. 

  • Right click the Skeleton or Skeletal Mesh
  • Select Create --> Anim Blueprint 
  • Open the Animation Blueprint

mceclip7.png

  • In the blueprint space, create a Live Link Pose node and connect the out pin to the Output Pose input pin
    • Type in the subject  name under Live Link Subject Name  

mceclip8.png

Select the Live Link Pose node and in the Details panel, select a Retarget Asset.  Depending on what type of character you are using, different remaps will be needed

  • Live Link Remap Asset is Unreal’s base class.  Do not use this one with Axis
  • The Manual Remap Asset is a custom character remap.  You will need to open the GetRemappedboneName Function to modify it
  • The Mixamo OLD Live link Remap works with Mixamo characters with “Mixamorig” Prefix
  • The Neuron Remap drives characters with the Perception Neuron and Perception Neuron Pro Rigs
  • The Studio Remap drives Characters with the Studio Skeleton and Mixamo Rigs without the “Mixamorig” prefix
  • UE Mannequin Live Link Remap Asset drives CC3 characters and may work for other 3rd party characters

mceclip9.png

  • After choosing a remap asset make sure to compile and save

 

Corrective T-Pose

This step is not necessary if you are using the UE4 Mannequin or another UE4 character in A-pose.  Make sure to select the Unreal preset in Axis Studio instead.

 

If your character is not in a T-pose, correct it following these steps, otherwise.  Some software have their characters in A pose by default, but T-pose is necessary to retarget data from the Axis Studio skeleton.

  • Open the character skeleton that needs correction
  • Use the skeleton tree to select the joints that need correcting to more closely match the Axis Skeleton
  • Make sure to look at the T-pose from the front, top, and side to make sure it looks correct

  • Once the desired pose is achieved, create an Animation Asset with the current pose
  • Name the Animation Asset something memorable and save it in the same folder as the mesh and skeleton
  • Add the corrective pose animation to the character's Anim Blueprint
  • Connect it’s output to the input of Live Link Pose
  • Compile and save. This will update your Anim Blueprint

Create Actor Blueprint Class for live sessions

Next, we are going to create an Actor Blueprint Class  

  • Right click on the Content Browser and select Blueprint Class

mceclip10.png

  • Select Actor and open the blueprint

mceclip11.png

  • Click Add Component --> Live Link Skeletal Animation Component and Skeletal Mesh component

mceclip12.png

  • Select the Skeletal Mesh component
    • In the Details panel, choose your character’s Skeletal Mesh under Animation --> Mesh

mceclip13.png

  • Now, just drag the Actor class into the scene

mceclip14.png