Tracking error correction and smoothing

[johhnry]

Hi,

 

I have a simple question : does the VIVE tracking mechanism handle error correction? Because I suppose that if at one time there's a perturbation in the infrared signal returned to the base station, it can disturb the movement of the controllers or the headset. Is it implemented using the previous saved position?

 

If so, is there a kind of smoothing applied on the motion of the controllers? Because I am setting up a virtual production set in Unreal Engine and I need the camera feed to be synchronized with the motion of the controllers (used for tracking).

 

Thanks in advance for the details

 

Joseph

[HackPerception]

@johhnry - There isn't smoothing per say but there's a few things at play under the hood. All VR devices including basestation tracked devices experience something called "judder" in which the pose estimate of the device will shift slightly from frame to frame, even if the device is stationary. This a result of the sensors not being able to update fast enough and accurately enough to firmly anchor the pose IRL. On basestation tracked devices, the judder will depend on your basestation placement and your overall environmental conditions (e.g. shiny surfaces).

In order to help reduce judder and to overall increase the device's tracking refresh rate, sensor fusion is employed in order to take IMU data from the controllers to supplement the basestation's fixed cycle rate. The IMUs can update much quicker and can be used from time to time to provide pose estimations for frames in which there isn't enough basestation data to derive an estimate, assuming that you eventually get basestation data in a future frame. This IMU update is a form of smoothing in a sense because it smooths out the pose estimate data and adds resolution but it's buried really deep inside of SteamVR's hardware stack so you can't modify it and the devices would have much greater judder without it. This is happening at hundreds of hertz per seconds - it's microsmoothing not macrosmoothing.

So, overall - there isn't a high level of smoothing with SteamVR tracking data but there is some level of IMU-based smoothing using sensor fusion that occurs as a requisite for the system to be somewhat accurate since the IMUs have crazy fast refresh rate. If anything, the judder itself can pose a problem for virtual production and may require smoothing in and of itself.

This is a helpful video on how to create a virtual steadycam in UE4 https://onsetfacilities.com/virtual-steadycam-in-unreal-engine-4-24/

This is the most detailed video in the public domain about how basestation tracking works and I highly recommend anybody working professionally with basestations to watch it. It explains some of what I alluded to in much greater detail: 

 

[johhnry]

Hi @VibrantNebula,

 

Thank you very much for the details you provided, it's exactly what I was looking for.

I'll check this "in depth" video about the tracking engineering! And also nice idea for the steadycam to smooth the movement in Unreal.

 

Have a great day!