Vive Pro cameras freezing in SteamVR when using Vive hand tracking

[User_25506]

Hi folks,

my company ships and sells turnkey simulators and we use the Vive Pro Eye as well as the the HP VR G2 Backpack computers. We have recently implemented the ViveHandTracking SDK to our software and it works great.

When we first implemented the SDK, we noticed after a random period of time, the camera would freeze in SteamVR and in turn, hand tracking completely stops working. This could be anywhere from 30 minutes or over 24 hours before it would freeze and stop working. For us, this was good enough to work in our simulators as people usually don't use it for more than an hour at a time.

Recently, we ran into a huge issue that I have been trying to solve for over a week now. This same problem is occurring on one of the systems we are building and it is happening on both computers (the system is 2 backpack computers with each their own Vive Pro Eye). Right now, on average, the cameras freeze within about 1 minute. Some times it is as quick as 30 seconds, other times it can go for 5 minutes. We have had a couple times where it went an hour, but that's only happened 2 out of probably 50 tests so far, and this is consistent on both computers. So it is freezing on these computers way too fast and therefore it is not usable at all.

The really strange part is that we have the exact same hardware in another simulator that we use in-house that does not have this problem to this extent. It still can almost always go at least 30 minutes and on average is usually even longer, like I mentioned above. We have our own installer that we made in house and we run it after installing Windows 1809 on every single computer... my point being is that the process to get the computers up and running our software is the exact same every single time on the exact same hardware, yet, my 2 new computers are having the same issue of freezing after a minute or so usually. The odd part is that the camera is actually functioning fine 90% of the time it freezes if I access it from another program like OBS, but SteamVR sees the camera as frozen until I restart SteamVR.

From here, we set aside that simulator with the issue as a whole and built another one from scratch in the exact same way with the exact same hardware, like we do every time. This also has the same issue on both backpacks. So 4 of our new computers have it despite every other system in the past 2 months usually only freeze after 30 minutes or so at the absolute worst and usually will last much longer.

So our in-house simulator with the same hardware and software and our 5 test benches we have (same hardware, but use the regular Vive pro and run in the Unity editor instead) do not have the issue to the same extent our new simulators/computers do.

I have tried everything to figure out what is causing this. I tried drivers, but they're all on the exact same drivers for the camera and all USB hubs associated with the Vive Pro eye on all computers. I tried different Windows versions with newer drivers and absolutely no change. I have tried different SteamVR versions (we use 1.11.11) with no change. I have tried to disable every power saving option (USB selective suspension and disabling power savings on all USB hub drivers) with no change. I have changed how we toggle on and off the GestureProvider with no changes. I have installed every chipset driver available for the computers with no change. I updated the bios with no change. I even reverted back to an older version the ViveHandTracking lobby by swapping the artiso_interface.dll and that also had no effect.

And here's where it gets interesting... I have swapped the Vive Pro Eye HMDs with the in-house sim that works fine and that changed nothing. I then swapped the drives of one of our not working computers with a working computer (remember, exact same hardware) and the problem persisted with the drive, NOT the hardware. The drive from the working in-house computer worked just fine in one of the computers having issues.

So this is clearly a software problem. I've tried copying folders over from working computers to see if something with the installer changed and it had no effect either. We are also always plugged in to power while doing tests, despite these being backpack computers (we're not running on battery while doing this).

I am completely lost. I know Windows very well and other than maybe something in the registry that affects this that I'm not aware of, I literally cannot think of a single thing at this point causing the issue. Drivers and pretty much everything are the exact same across computers, yet I cannot find a change after playing with this for a week. It seems like some power saving method is affecting this, but I have disabled all of those. I have heard of this issue with the Vive wireless adapter, but I cannot seem to find any more info on this. I find this surprising considering the camera still will always end up freezing on any computer we use, despite it usually taking an hour or so or more to freeze on all our computers except these 4 new ones having the issue.

I have wasted days on this and I am growing extremely frustrated and defeated as the hand tracking is absolutely incredible when it works. I have updated our software and sent it out to over 80+ users before having this issue pop up on our newer computers and I am worried that some others may also have this issue too upon updating. It just went out last week and they usually don't update immediately, so we haven't heard of any customer having this issue.

Rather there's an API call I can make to SteamVR to reset the camera, or a way to bypass SteamVR for camera access (I doubt), or a way to prevent the camera from freezing all together, or even a way to just mitigate it most of the time like all of our other computers work, at this point, anything will help. I am hoping any advice or thoughts or experiences people have with this will reply here so I can have something more to work with.

Thanks in advance, I will be following this thread and will likely respond quicky.

[zzy]

Hi @User_25506

Thanks for the info. I think the camera from HMD will freeze if you try to access the camera device outside of SteamVR. I can also confirm this, if you try to use Camera app from Windows to access the HMD camera, it will freeze. I think this maybe caused by the driver or there are some softwared problem inside SteamVR.

I would suggest to not access the camera using other other APIs. Or if other process must access camera on start, you can try to plugin a usb camera first, and plugin your HMD after it. Then the default camera device will the usb camera and the process will not access the HMD camera.

[User_25506]

3 minutes ago, zzy said:

Hi @User_25506

Thanks for the info. I think the camera from HMD will freeze if you try to access the camera device outside of SteamVR. I can also confirm this, if you try to use Camera app from Windows to access the HMD camera, it will freeze. I think this maybe caused by the driver or there are some softwared problem inside SteamVR.

I would suggest to not access the camera using other other APIs. Or if other process must access camera on start, you can try to plugin a usb camera first, and plugin your HMD after it. Then the default camera device will the usb camera and the process will not access the HMD camera.

Hi there, thanks for replying.

I am not accessing the camera outside of SteamVR, I just mentioned this to clarify that the camera is only freezing in SteamVR, and in turn, making hand tracking stop working. It works outside of SteamVR, even when it has frozen in SteamVR, I just mentioned that so it's obvious that only SteamVR sees the camera as frozen.

I will reiterate that we do NOT use the camera outside of SteamVR, nor is that the problem at all. The problem is the camera freezes inevitably after a minute or two, and if we're lucky, it lasts 10 minutes. Some of my older computers/setups (same exact PC hardware and a Vive Pro Eye) can last much longer on average, usually around 20 minutes or more.

Overall, all of these are still problems and ideally we would figure out how to stop the camera from freezing all together or at least significantly less. 

Thanks in advance.

[zzy]

Hi @User_25506

From my previous experience, when SteamVR experience camera failure (e.g. package lost) it will mark camera failed and never retry. This makes the camera freeze.

So I would recommend to try to make sure camera communication is working by reducing usage of USB bus by other devices. Here are my advices:

1. Always make sure Vie Pro is connected to USB 3.0 port.
2. Try to remove other usb devices if possible.
3. Try to plugin other devices (e.g. keyboard and mouse) to USB 2.0 ports if any.
4. If you are using other usb devices that is also transmitting a lot of data, you may consider adding USB extension hardware to make sure all devices can runs at full speed.

[User_25506]

Hi @zzy,

I appreciate the reply.

1. The Vive Pro Eye is always connected to USB 3.0 in our system

2. & 3. No other devices are plugged into USB anywhere on the computer, only the Vive Pro Eye

4. We don't even use the mic nor the speakers on the Vive Pro Eye, so the only data is the tracking/watchman dongles, and the Vive Pro Eye camera 

My worry is that this is a fundamental issue with the driver or SteamVR and the only proper way to fix this would be to somehow tell SteamVR to refresh the camera connection, since this seems to be an issue on all of our computers. Not to mention that this has been a known issue that I have experienced for years with this HMD and SteamVR.

Is there any way to bypass SteamVR for using the camera for hand tracking or is the best method possibly reverse engineering SteamVR so I can refresh the camera? Or any other ideas?

Thanks again.

[zzy]

Hi @User_25506

Hand Tracking SDK need to use OpenVR API to get camera info and frames, so I think it's difficult to bypass SteamVR. I think the only way to refresh the camera is to restart SteamVR, but that doesn't solve the problem if camera freezes every minute or so.

I'm now running out of advice. Maybe you can consider using Vive Cosmos series instead, it doesn't use SteamVR to acess camera.

[User_25506]

Hi @zzy

We are a multi-million dollar company and have over 100 customers with an established product, we cannot just switch from the Vive Pro Eye to the Cosmos, especially because we use lighthouse tracking.

Thanks for the replies, but it would be great to at least get some direction or someone proper to talk to or we're going to have to switch to the Valve Index.

Thanks again in advance.

[zzy]

Hi @User_25506

I understand your concern. One direction I can give is that you can try with Vive Elite, which also supports lighthouse tracking and avoids SteamVR camera API.