VIVE Full Face Tracker for XR Elite - Eye Blend Shapes

[Klary]

Hi everyone,

I've been trying out the face tracker for the XR Elite with OpenXR (https://registry.khronos.org/OpenXR/specs/1.0/html/xrspec.html#XR_HTC_facial_tracking).

I noticed that the blend shape weights for XR_EYE_EXPRESSION_LEFT_WIDE_HTC and XR_EYE_EXPRESSION_RIGHT_WIDE_HTC are always 0. The others seem to work fine. 

Does the eye tracker not support these two blend shapes?

On the official website for the tracker (https://business.vive.com/uk/product/vive-full-face-tracker/) it seems like the female example moves the eyebrows slightly.

I would be grateful for any insights!

Thank you and best,

Klary

[VIVE_chengnay]

Hi @Klary,

Currently, these two values are not supported.

Will this cause any problem during your development?

[Klary]

Hi @VIVE_chengnay and thank you for your response.

Sorry for my late reply, I did not follow the post.

Does the hardware of the face tracker not support it, or is it just not connected in software? Would I have access to them if I used SRanipal?

On 4/12/2024 at 5:42 AM, VIVE_chengnay said:

Will this cause any problem during your development?

Yes. It would be extremely helpful to have them, as they add a lot more expressivity to the face. Without them some emotions are very hard to express.

Previous users seemed to get the blendshapes with v2 eye data from SRanipal:

Thank you and best,

Klary

[VIVE_chengnay]

Hi @Klary,

SRanipal only works for Vive Pro Eye hardware.

11 hours ago, Klary said:

Does the hardware of the face tracker not support it, or is it just not connected in software?

I was told that those two values are not supported now.

I will check internally.

[VIVE_chengnay]

Hi @Klary,

Maybe I am wrong with SRanipal statement.

I will double check again today and reply you later.

[VIVE_chengnay]

Hi @Klary,

SRanipal does support data from XR Elite's face tracker mapping to SR runtime.

But, it does not support OpenXR.

My colleague told me that Wave plugin does support right and left wide values.

[Klary]

Hi @VIVE_chengnay,

I finally got around to testing things with the face tracker.

Unfortunately, everything but the right and left wide values worked.

I tested several example scenes provided with the VIVE Unity packages:

• Unity Wave Plugin
  • I ran the "EyeExp" scene, which shows the values for each blendshape in VR. All blendshapes worked with the exception of the left and right wide values.
• VIVE Eye and Facial Tracking SDK 1.3.6.8
  • I ran the "EyeSample_v2" scene. Same results, left and wide values not working. The rest worked fine.
• VIVE Open XR
  • "FaceModel" scene. Same results.

Because the left and right wide eye values are consistently 0, I wonder if the XRElite does not provide this information at all?

Thank you and best,

Klary

[Klary]

Hi again,

On 4/12/2024 at 5:42 AM, VIVE_chengnay said:

Currently, these two values are not supported.

To clarify this based on a post I read on the VRCFaceTracking Discord server: These values seem to be disabled on the hardware side. So I can do nothing on the software side to make it work.

Any news of when they will be enabled?

[Klary]

And hi again,

Could I downgrade to a previous driver for the face tracker so the wide eyes are enabled again?

 

[VIVE_chengnay]

Hi @Klary,

For OpenXR, it is not supported now.

We plan to support in future release by end of July.