Jump to content

Problems with hand detection running on VIVE Pro with OpenXR API

Recommended Posts


I've been trying to get hand detection going for my project and this is going to be a bit lengthy, because I've already tried out so many things, so here goes:

First I started with hand tracking in OpenXR. I followed the tutorial described here: https://developer.vive.com/resources/openxr/openxr-pcvr/tutorials/unity/how-integrate-hand-tracking-data-your-hand-model/

Had to fix a couple of problems in the tutorial code, notably adding declaration for locations: private static XrHandJointLocationsEXT locations;

And also a few other problems. So the script now compiles and runs, but unfortunately when I call res = feature.xrLocateHandJointsEXT(feature.m_leftHandle, locateInfo, ref locations);, the locations.isActive is always 0.

I've debugged as far as I could, but I just don't seem to be getting hand tracking updates. Any suggestions would be very appreciated as I am not very familiar with OpenXR or unity.

I'm using Unity 2021.3.5f1. Also camera is enabled in SteamVR settings and tested. It just looks like hands are either not detected or camera input is not reaching OpenXR. I also tried to create a completely new project and just run the sample code, launching HandTrackingScene.unity from the samples folder, that we can import with VIVE OpenXR for Windows. That also didn't work- same thing, the hands don't even appear.

And same thing happens in Unity 2020.3.13f1. All I do is follow this tutorial:
And then the OpenXR setup part of this tutorial:
And then I only run the sample project and it doesn't do anything. If I look at the code and print out a debug statement, then it is obvious that line 46 in UpdateHandData.cs always evaluates to FALSE: 
so the branch of: if (locations.isActive == 1) 
never executes.
The source code of 
public int xrLocateHandJointsEXT(ulong handTracker, XrHandJointsLocateInfoEXT locateInfo, ref XrHandJointLocationsEXT locations) inside HandTracking_OpenXR_API
seems to be closed source, so I don't know what happens beyond.
I guess one more useful thing to add: I can get the hand tracking to work on Unity 2019.4.30f1 using the Hand Tracking SDK: https://hub.vive.com/storage/tracking/unity/setup.html
But that requires to use OpenVR, which is deprecated in later unity versions. Unfortunately Unity 2019 is not an option for me, because I need to also run ROS-TCP-endpoint and that one only works on Unity 2020.2.x and above. 
I also tried out importing SteamVR plugin Version 2.7.3 (sdk 1.14.15) - February 23, 2021 into Unity 2021.3.5f1 and that allows me to use the old Hand tracking SDK with OpenVR. But that gives me two new (albeit less severe) problems:
1) Unity crashes the moment I stop my app
2) When I start my app, StemVR displays a complaint in the headset about missing actions.json file in the "Assets\StreamingAssets\SteamVR" folder. The error says: the action manifest for [Testing]<my project name> was not found. I created an empty actions.json file there and it still can't find it, so I don't know what else to try. 
I desperately need help on this please! I am working with a VIVE PRO headset. I am using Windows 11.
  • Like 1
Link to comment
Share on other sites

  • 11 months later...

Hello Arturs,

I am curious, did you get it to work? I am facing the same problem now. I have debugged like you, everything looks good in the OpenXR runtime Debugger, giving XR_SUCCESS messages too, but the hand is never detected. This implementation is closed to us developers and since the documentation mentions support for the VIVE pro, I am a bit confused. I believe the company should provide a clarification to developers regarding this.

Best Regards

  • Like 1
Link to comment
Share on other sites

  • 4 weeks later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...