This article will have information on steps for setting up Perception Neuron and Unreal using our UNREAL Plugin.

AXIS NEURON SETUP

Broadcasting

In Axis Neuron configure the broadcast settings to UDP. 

Enable BVH data stream and leave the default ClientPort of 7002

Output Format

Disable Displacement option so that the Unreal Plugin ignores Axis Neuron's bone translation data and only uses rotation data. Feet movement will still be tracked.

Note: If you enable displacement, the body size of the avatar inside of Axis Neuron will have 100% affect on the assigned Unreal Character.

Character Assignment

Our Unreal Plugin relies on the naming of your character inside of Axis Neuron. You will need to input the name from Axis Neuron correctly inside of the Unreal. (image e.g)

 

UNREAL SETUP

PROJECT SETUP

Open the Unreal Create a Blank Project and use the following name “NEURONDEMO”. For the project settings select No Starter Content, recommended.

Close the Unreal Editor

Create a new folder in the root director of the Project. Rename the new folder “Plugins”.

Download and Unzip the PerceptionNeuron plugin from Neuronmocap.com downloads page. Based on the Unreal Editor you have use the corresponding version.

Copy the PerceptionNeuron folder found in the unzip file onto the Plugins folder of your Project

Restart the editor and open the NEURONDEMO project

Once the NEURONDEMO project is open, in the Unreal Editors Menu go to Edit->Plugins to check whether the PerceptionNeuron plugin has successfully installed and has started with editor as below:

 

Create an Animation Blueprint 

What is an Animation Blueprint?                                                                                                                                              An Animation Blueprint is a specialized blueprint that controls the animation of the Skeletal Mesh.

 

Begin with importing a premade model with a skeleton. For this exercise use the FBX file attached at the bottom of the exercise.

In the Content Browser click on Import

In the FBX Import Options select “Import as Skeletal”

NeuronRobot_SingleMesh and NeuronRobot_SingleMesh_Skeleton have been imported to engine successfully in Content Browser:

Create an Animation Blueprint based on NeuronRobot_SingleMesh_Skeleton. Right-click on NeuronRobot_SingleMesh_Skeleton>Create>Anim Blueprint to create Animation Blueprint:

 

We can see the newly created NeuronRobot_SingleMesh_Skeleton_AnimBlueprint in ContentBrowser:

Double left-click on the Animation Blueprint to edit it. Right-click on Anim Graph to create a NewPoseCalc node and link it`s output to FinalAnimationPose node.

Of course you can also do some work on the output of NewPoseCalc node according to your requirement, and then output the final result to FinalAnimationPose node.

Compile Anim Graph:

Skeleton Retargeting

PerceptionNeuron plugin defines a standard skeleton structure. The standard skeleton structure is made of 59 bones, they are: 

Hips
RightUpLeg
RightLeg
RightFoot
LeftUpLeg
LeftLeg
LeftFoot
Spine
Spine1
Spine2
Spine3
Neck
Head
RightShoulder
RightArm
RightForeArm
RightHand
RightHandThumb1
RightHandThumb2
RightHandThumb3
RightInHandIndex
RightHandIndex1
RightHandIndex2
RightHandIndex3
RightInHandMiddle
RightHandMiddle1
RightHandMiddle2
RightHandMiddle3
RightInHandRing
RightHandRing1
RightHandRing2
RightHandRing3
RightInHandPinky
RightHandPinky1
RightHandPinky2
RightHandPinky3
LeftShoulder
LeftArm
LeftForeArm
LeftHand
LeftHandThumb1
LeftHandThumb2
LeftHandThumb3
LeftInHandIndex
LeftHandIndex1
LeftHandIndex2
LeftHandIndex3
LeftInHandMiddle
LeftHandMiddle1
LeftHandMiddle2
LeftHandMiddle3
LeftInHandRing
LeftHandRing1
LeftHandRing2
LeftHandRing3
LeftInHandPinky
LeftHandPinky1
LeftHandPinky2
LeftHandPinky3

If the imported skeleton is not standard, you may need to do skeleton retargeting to guarantee the character's movements driven by final motion capture data to reach your expectation. Select NewPoseCalc node in Anim Graph, it will display editing options about skeleton retargeting in Details panel, as shown below:

We can expand BoneMap to edit SkeletonRetargeting:

The left column are standard skeleton names defined by PerceptionNeuron, the right column are custom skeleton`s bone names. If the imported skeleton`s bone names completely fit the standard ones defined by PerceptionNeuron, then the two columns of skeleton names will automatically mapped correctly (pictured directly above, shown as 2). If the custom skeleton`s bone names contain the names which are not defined in the standard ones, then the plugin will automatically using Invalid to map the bone name (pictured directly above, shown as 1), and this bone will maintain relative still to the parent bone when animation is playing.

We can adjust the bone mapping relationship of skeleton retargeting by clicking the bone names in the two columns:

We can add and delete a couple of mapping of bone names:

Note: It’s meaningless that any bone of skeleton is driven by multiple PN bones, so the names in the right column must be unique. The number of the bones in BoneMap may be more than actual imported one. This is because UE recognize mesh as skeleton, maybe it’s a bug. We can delete the bone mapping which name is invalid in the left column to improve performance when it is necessary.

Create Actor Blueprint

 We create a Blueprint class and will use it to create test character:

 For simplicity we choose Actor as ParentClass, create an actor Blueprint and named it as "AvatarBlueprint" 

We can see the newly created AvatarBlueprint in ContentBrowser, double  left-click  on AvatarBlueprint to open the BP editor to edit AvatarBlueprint:

Click AddComponent to add PerceptionNeuron and SkeletalMesh component:

Click the newly created SkeletalMesh component, set SkeletalMesh component`s properties in the Details panel on the right side

Result:

Compile AvatarBlueprint and close the BP editor.

 

Create Actor Instance Setup

Drag and drop newly created AvatarBlueprint to the scene to create a new actor instance:

Select the newly created actor in the scene, then select PerceptionNeuron component under the actor in Details panel. We can find the properties Avatar Name and BVH Asset under PerceptionNeuron in Details panel:

In the Avatar Name section, use the same naming found in Axis Neuron.

ATTRIBUTE NAME	DESCRIPTION
AVATAR NAME	Avatar Name is used to identify network data source stream Name used by an actor when one actor uses network data source to drive animation. (Each avatar`s motion capture data sent by Axis Neuron is identified by an Avatar Name, we can use the Avatar Name to match an actor with it`s data source stream). Network data source will be disabled when the property BVH Asset is assigned and the actor animation will be driven by BVH asset data source.
BVH Asset	BVH Asset property is a reference of BVH asset. To create BVH asset we need to use BVH motion capture data. Network data source will be disabled when the property BVH Asset is assigned and the actor animation will be driven by BVH asset data source

 

Network Data Source

We should create Perception Neuron Manager and Perception Neuron Spawner before using network data source：

Perception Neuron Manager is used for creating the network communication environment and disposing received motion capture data. Perception Neuron Spawner contains some properties for creating an actor, such as class type, initial born position, network data stream name and so on.

First, we put a Perception Neuron Manager into the scene:

Set network properties for Perception Neuron Manager. IP can’t be edited, plugin can get the host IP automatically and show it in Details panel; Set the UDP port number with the port in Axis Neuron settings.

Create a Perception Neuron Spawner in the scene to indicate the initial spawn position for the actor:

I use HeroTPP to make HeroTPPBlueprint and HeroTPPAnimBlueprint, and deal with the skeleton retargeting:

Set the newly created HeroTPPBlueprint to spawner and set the receiving network data stream name with Avatar00 for spawner:

Note:The default network data stream name for Axis Neuron is Avatar00. The default value is used here, but the name can be modified in Neuron Axis:

 Click Play and run Axis Neuron to send motion capture data, we can see that a new HeroTPP actor has been created dynamically in the scene and started to play animation:

We can rotate the direction of spawner to run different results:

Note: Axis Neuron provides TCP and UDP two kinds of protocol to send network data, but PerceptionNeuron plugin only supports UDP protocol to receive network data currently.