This setup is currently in beta. Please follow the steps to begin setting up face capture inside of UNITY
Prerequisites
- iPhoneX
- Character should be compatible with the PNS skeleton and ARKit blendshapes
Network Setup
To begin with, you'll need to setup your network PRIVATE, as this allows for special permissions in the data transfer required by the UNITY LIVE FACE app.
Go to Windows Settings --> Network and Internet
Locate your WIFI network connection and choose Properties.
Set the Network Profile Type to Private
Creating a Rule
Go to Control Panel --> Systems and Security --> Windows Defender Firewall --> Advanced Settings
Select Inbound Rules --> New Rule
-
- Rule Type: Program
- Program: This Program Path
- Browse to the Unity.exe for the corresponding engine
- Action: Allow the Connection
- Profile: Private
- Deselect Domain and Public
- Name: Something that will make sense to you such as Unity Live Capture
Unity Live Capture Setup
Unity Live Capture is Unity's new application that allows users to connect to the Unity Virtual Camera and face capture applications. For this setup, we will only focus on setting up the package to communicate to the Unity Face Capture for the purpose of capturing face movement. Live Capture is available through Unity's Package Manager.
In UNITY menu bar navigate to the Window menu a select Package Manager
Select Unity Registry from the Packages drop down menu
Select Live Capture Package and Install
Open the Connection Window
Window --> Live Capture --> Connections
Click Create Sever --> Companion App Server
-
- Set the Port number
- Must be different from the one in Axis Studio
- Click Start Server
- Set the Port number
Character Setup
Follow the usual setups for streaming Axis Studio data to a Unity character:
Live Stream data into Unity – PERCEPTION NEURON (neuronmocap.com)
Right click on your character in the Project Window
Create --> Live Capture --> ARKit Face Capture --> Mapper
Open the Mapper
-
- Add the Left Eye and Right Eye
- Add all the Render Options available
- Create a Prefab from the version of you character in the scene
- Drag the character from the scene to the Project Window
- Click on your character prefab
- Add Component -->ARKit Face Actor
- Add your Mapper to the Mapper Option
- Click GameObject --> Live Capture --> Take Recorder
- Open the Take Recorder
- Click the "+" under Capture Devices and select ARKit Face Device
-
- This will create a child component called ARKit Face Device, click on it
- Under Actor, select your character
- This will create a child component called ARKit Face Device, click on it
iPhone Setup
Unity provides the companion app on the App Store. On your iPhone, open the App Store and search for UNITY FACE CAPTURE
Information on the app can be found here: Unity Face Capture on the App Store (apple.com)
Open the app
If everything has been set up properly, the server should pop up as a connection option
Press play and you should now be able to capture face and body in Unity