VBS + SteamVR + Handtracking

[LezT]

Hi, 

Since I am using the Focus 3 last year, I did not get the hand tracking correctly working with/in Steam VR environment.

I am currently using VBS 1.13.8b on my PC/ 1.0.19.12 on my device in combination with Steam VR 2.1.10 Standalone. 

The mismatched tracking is already starting in the native VBS/Steam VR environment.

Here I already notice that the controller models, which representing my hands are placed from my hand joins up my underarms (basically like my hands are cut off, and I am using my underarm to control)

Also, there is another mismatched feeling:

While my physical hands are rotated with my thumbs showing "up", hand palms facing each other and underarms are slightly rotated, while holding the controllers in a native position, the tracking in Hand tracking mode this is drastically different:

- My back of my hands have to show up, and I have to rotate and transform my underarm to nearly get the native 3D model of the controller to a position similar to the above case (using physically controller's native position) with more different/strange rotations and transformations going on. 

This cant be intended, but I am not sure where to address this problem, and how I can fix it from my side. Otherwise, I would say the bridge between VBS and Steam VR needs some improvements.  

But anyway its good that it is technically possible to Use the VR-Device with pc streaming and still be able to use handtracking.

[LezT]

A small Update: I noticed the handtracking mode is working well and as intended in the SteamVR Home environment, but the above behavior is happening in the "VBS-Steam-Environment" - the night-time valley with nothern lights.

[LezT]

But this may be the case because the SteamVR Home App is apperently running with Steams Source Engine 2 and processes things differently.

[jiunlin]

Hi @LezT

Since SteamVR doesn't really support nature hand tracking, so what you see in the SteamVR grid space is actually virtual controllers emulated from nature hand tracking by VBS. VBS did take the wrist joint as the center of virtual controller, so you see the offset.

VBS also did a rotation as you discovered. The purpose is to support those SteamVR contents which will draw hand model based on controller.

As long as the content takes hand poses to render hand model (OpenVR hand skeleton or OpenXR hand tracking), such as The Lab and SteamVR Home, the hand should works well.

We have plan to improve the virtual controller orientation with hand tracking, but currently not in priority.