Ghosting effect caused by Async Reprojection

[SimoneSantillo]

TL;DR

Is there a way to effectively disable Async Reproject permanently? Is it possible to run an application at its real frame rate (<90 Hz) without interpolated frames?

 

Context

I'm working on a mixed reality project developed in Unity that runs at 30 FPS (due to heavy computation) and I am using the HTC Vive Pro 2 as the HMD. The real world frames are provided by a ZED Mini camera, attached on top of the two Vive Pro 2 front cameras. We do not use the cameras for inside-out tracking, nor the base station. In our project tracking is explicitly solved through Aruco markers and that part just works. We employ the HTC Vive Pro 2 just as a display.

I noted that the rendering on the HMD suffers from a "ghosting" effect when the user turns his head around. I suppose that the problem is caused by the "Motion Reprojection" (or Async Reprojection) that is done under-the-hood to maintain a 90 Hz framerate on the HMD while our application can only provide frames at 30 Hz. 

Why I think it is caused by reprojection:

• The SteamVR Mirror Screen does not show this ghosting effect;
• If I disable the heavy part of computation in my app and make it run ~90 Hz, we don't see the effect any longer. Also from 30 to 90 Hz the effect seems to disappear somehow gradually.

 

This is an image of what we perceive by looking inside the HMD.

Note that the above (right) image has been created ad-hoc just to let you understand the issue, but we have no way of actually capturing the true issue as it does not appear in the Mirror Screen. The arrow in the (right) image shows the direction of the camera movement.

 

I could not find any way to disable this feature and allow me to stream 30 Hz video stream to the HMD.

Strangely enough, I took a SteamVR System report and it shows that Async Reprojection is disabled, but later in the report it shows 

 

Here's what I've tried up to now:

• Hit the Shift+A key binding in the SteamVR mirror screen to disable "Async Reprojection", but this did not effectively disable the reprojection (both VIVE Console and SteamVR System report show reprojected frame count >> 0) 
• Double checked that ZED camera is not performing any inside-out tracking
• Created a clean new project with just ZED Mini CameraRig: by forcing the application to run a framerate < 90 Hz the ghosting effect starts to appear, and it intensifies the lower the frame rate is. 
• I run ZED Unity Plugin's "Dark Room" demo and found out that even if the application runs at 120 FPS (as shown in Unity's Stats tab), the ghosting effect is visible when ZED Mini is set to a framerate lower than 90 Hz.

 

Additional information:

• Windows 10 latest update
• i9-11900K
• NVIDIA GeForce RTX 3080 Laptop (v511.65 drivers)
• 64 GB RAM
• DirectX 12
• Even if my hardware should be compatible, I cannot enable "Hardware-accelerated GPU scheduling" (windows settings tab does not show me the switch to enable it)

Question

Am I right on thinking that the problem is the "Async Reprojection"? If so, do you have any suggestion on how to disable it / effectively run an application at its real frame rate, i.e. without interpolated frames?  @C.T.

[JackSpire]

@C.T.Could you take a look at this?