Jump to content

[VBS & PC VR ] - How to use Facial Tracking on Focus3

Vivi Wu

Recommended Posts

PC VR contents could now enjoy Focus 3 Facial Tracking via OpenXR Facial tracking extension or  VIVE Sense - Eye and Facial Tracking.

Support information:

  1. Eye gaze supported
    • Support combined data and separate-eye data
  2. Facial Tracking:
    • Eye expression(blendshapes) supported
    • Lip expression (blendshapes) supported


  • Update your Focus 3 Version : 5.0.999.646 or later
  • Install SR Runtime  (Please joint Beta program)
    • Install or update the latest SR Runtime included in VIVE Software form  SteamVR  ( or later)or OOBE ( or later), and check your SR_Runtime status
      (SRanipal Runtime Version or later) 

VBS Facial Tracking for developers

There are two open SDKs that VBS supports to bring Focus 3 Facial Tracking to PC contents.
Choose one of the plugin below to use: 

To enable  facial tracking data from AIO headset:

  1. Make sure the VBS version is 1.09.8 or later
  2. Install VBS app in your headset
    Go to VBS > Settings > AboutVersion> Click  Install



  • Like 2
Link to comment
Share on other sites

  • 1 month later...

Yeah, so I just can not get this to work. I've been searching all of the scattered tutorials for this, and they seem incomplete because even following the steps, things don't work or break entirely.

My use-case is simple - full PC VR over VBS with the Focus 3 with the integrated eye trackers. I want eye tracking, but all i really care about is getting gaze data (eye vectors etc, like i used to with the vive pro eye with SRAnipal). I don't care about facial tracking or avatar stuff.

The instructions mention a few things, but none of them work for me. SRAnipal doesn't work, firstly i *can* get eye tracking "enabled" using the steamVR overlay button for SRAnipal eye tracking, but then when it asks me to calibrate, i can get past the first step (Raise/lower HMD) but the second step (modify IPD) does not progress because it seems that the IPD adjustment dial on the F3 does not talk to the SRAnipal runtime and i cant get it to move.

So even if i skip the calibration, im getting no values in ue4 Preview In Editor. I am using the "Focus" node in blueprints and getting no data from Gaze Target.

So the other option seems to be to ignore SRAnipal, and to use OpenXR completely, which is said is recommended. The instructions say that you need to enable openXR in plugins, and disable steamVR in plugins. When i disable the steamVR plugin, my HMD can no longer track at all. I am signed in to VBS, and in my HMD i am in the steamVR home environment, but when i start ue4 VR preview, the headset does not hook. The other thing is that the instructions here https://developer.vive.com/resources/openxr/openxr-pcvr/tutorials/unreal-engine/integrate-facial-tracking-data-your-avatar/ say

"Please make sure the “ OpenXR Facial Tracking extension ” is enabled, the setting is in Edit > Project Settings > Plugins > Vive OpenXR > Facial Tracking: "

this setting doesnt exist in my poject settings plugin settings. There is no Vive Open XR settings at all, even though i have the openXR plugin enabled.


The (third??) option seems to be using wave SDK. ive tried that too, i get no values. using the "Get eye tracking status" node, it returns "UNSUPPORTED". The eye tracking device works when running the focus 3 normally, doing the calibration demo.


  • Like 1
Link to comment
Share on other sites

Replying to my above comment (its still being approved 17 hours later apparently) while the tutorial mentioned using SDK, i found a newer one for ue4.27 that is version That one comes with then ViveOpenXR plugin. So i now have that installed into my project.

So the update is that now i have the openXR facial tracking extension enabled in the vive openXR plugin settings. Still not getting any data, still the node "is eye tracking available" returns false. thats under the waveSDK.

Under openXR, i try createFacialTracker - that returns false. But then right away i check isFacialTrackingEnabled and it returns true. riiiiiight. but openXR doesnt seem to have any eye tracking gaze data functions.


So. I assume i need to use openXR, and waveSDK, but NOT SRAnipal. then it comes down to two options, have steamVR plugin enabled, or not.

1. if i have it enabled, i get TRUE from createFacialTracker. but i still get not values from waveSDK's "getCombinedEyeDirectionNormalized".

2. if i have steamVR disabled, i get FALSE from createFacialTracker, and still get nothing from "getcombinedEyeDirectionNormalized". But now i lose my motionController for the focus 3



Edited by Ryam
Link to comment
Share on other sites

Hi Ryam,

Please update your Focus 3 Version to  5.0.999.646 and do the eye calibration on the headset setting.

Pleas check what platform are you using?


Link to comment
Share on other sites

@Vivi Wu

I do not have a facial tracker, just eye trackers. I have the most recent vive business streamer, and my focus 3 has the latest version.

Can you please clarify - how do i get eye tracking values out of the openXR facial tracking nodes? I have looked at the nodes and they only have facial tracked pose information, no eye gaze data. that would be for using openXR and NOT using SRAnipal, i assume.

So here's where im at, i currently have the ViveOpenXR plugin, and the SRanipal plugin.


run VBS v 1.09.8 beta on pc in administrator.

put on F3, run eye calibration. success.

Run VBS in F3. connect to steamVR.

run sr_runtime.exe (version as administrator. while im not playing the game, it shows the blue icon (attachment 1)

i run the eye tracking calibration inside the headset. it succeeds.

I now have GREEN eyes in the sranipal icon.

i run my packaged ue4 game as administrator that contains the eye tracking nodes for getting gaze data, screenshot attached.

the game runs fine until i put the headset on and it see's my eyes, then it crashes. if i run the project in ue4 editor it crashes and tells me there is an unhandled exception, exception_access_violation (usually this is for not running things in admin, but i have run everything in admin). screenshot attached.

i have tried the above setup with openXR and openXR eye tracking plugins enabled AND disabled in plugin settings.

This is the closest i have gotten.





Link to comment
Share on other sites

Hi @Ryam,

Let us clarify more about the situation your are facing.

ViveOpenXR and SRAnipal are the two interfaces which support facial tracking either from Vive Pro Eye or Vive Focus3 via VBS.

You should choose only one of it to develop your game.


As the post said, you install the corresponding plugin when you have chosed one of the interface you would like to develop with.

(OpenXR interface is recommanded)

For example, if you choose OpenXR interface to develope with, you don't have to install any SRAnipal SDK or plugins, you just need to download the OpenXR plugins and start to develop as described in the OpenXR tutorial.

And please do not enable SteamVR plugin when you are using OpenXR plugin.

One thing needs to be mentioned here - Vive OpenXR plugin provides the OpenXR facial tracking (the blend shapes of facial expression) while eye gaze is provide by Unreal OpenXR plugin.


Hope these could help you step forward.


Link to comment
Share on other sites

  • 2 weeks later...

I am facing a similar issue: trying to record eye tracking data on the headset or spool it out. I understand the support for Tobii pro lab is not longer there.

Can you please provide alternative solutions for analyzing gaze data?  


Link to comment
Share on other sites

  • 2 weeks later...

This doesn't help. The "Unreal OpenXR plugin" that you mention does not return any values at all.

I DO NOT HAVE A FACIAL TRACKER. There is no facial tracking happening at all. I only have an EYE tracker.

Look at the link that was shown further up as a tutorial for eye tracking:



You say that for eye tracking that I can use OpenXR, or SRAnipal, and that OpenXR is recommended. But where is the eye tracking for openXR? openXR only has FACIAL TRACKING.


I am guessing that you are not actually testing anything that is being said here, because you keep answering that open xr will handle eye tracking (it doesnt, its only facial tracking) or that SRAnpial will work but is not recommended.


There is no tutorial that works. You need to go through your own tutorials and test it for yourself. Test openXR with a HTC focus 3, with EYE TRACKER MODULE, in ue4, over wireless vive business streaming. it doesnt work.

Link to comment
Share on other sites

Hi @Ryam,

Eye gaze is provided by Unreal Engine.
Unreal engine provides the Eye Tracking feature by enabling the OpenXREyeTracker plugin from Plugin > Built-In > Virtual Reality > OpenXREyeTracker.
After enabled the Eye Tracking feature, you can access the Eye Tracking data from UEyeTrackerFunctionLibrary interface.

Hope these could help you step forward. Thanks.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...