UE4 OpenXR Headset Render Target loses individual eyes?

[star-sri]

On version 12 of business streaming, I had been having a fatal crash in UE4 while using a component a coworker wrote. I switched to the beta branch, and on v1.13.4, it no longer crashes! It does this:

Oculus OpenXR:

HTC Focus 3 with VIVE Business Streaming and SteamVR:

Where did the two separate eyes go? Obviously something is wrong...I think I saw this once with a Pico Neo 3 over SteamVR, but I haven't been able to reproduce. This happens EVERY TIME with the Focus 3.

Clearly UE knows there are two eyes, as you can still see the ‘lighting needs to be rebuilt’ message in two locations. I’m using the stock VRPawn with no changes. Also, once this happens once, it seems to persist until I close the editor, even on levels that don’t have the custom components.

What causes this effect? Is this something in the VBS OpenXR stack, or is it all on the UE side? Thanks for the help.

Edited November 17, 2023 by star-sri

[jiunlin]

Hi,

We are not able to reproduce this issue here. Might need your support to clarify below two things.

1. Since you are using OpenXR, will you try with UE5 to see if still see this issue?

2. please ensure to disable other headset plugin, since you are testing with difference headsets. Might need to pay attention to use the correct plugin for different headset.

Looking forward to more information. Thanks.

[star-sri]

Thanks for the reply!

To clarify two things: First, this is a PCVR application - I have no plugins from ANY manufacturers, only OpenXR. The application is running on the computer, and via Steam and VIVE Business Streaming the video is going to the headset. I do change the OpenXR runtime on my computer between Steam, Windows Mixed Reality, and Oculus, but the program wont even start if these aren't set correctly.

Secondly, I am now fairly certain this is caused by some DirectX11 code I am running in Unreal. I can't send the whole segment, but i can try to extract what features are causing the issue.

// get reference to Unreal's Render Device and context so that we can make D3D calls
g_d3d11Device = (ID3D11Device*)GDynamicRHI->RHIGetNativeDevice();
g_d3d11Device->GetImmediateContext(&g_d3d11Context);

// Create a new texture
ID3D11Texture2D* ue4Texture11 = (ID3D11Texture2D*)m_textureUE4->Resource->TextureRHI->GetTexture2D()->GetNativeResource();

// Create a texture in a file-mappped memory location (Create() uses CreateFileMappingA from the windows API)
m_sharedTexture11 = new SharedTexture11;
m_sharedTexture11->Create(g_d3d11Device, name, m_width, m_height, format);

The shared texture uses File-Mapped memory, which is provided by Microsoft.

Theoretically, this should not interfere with video streaming to the headset, as I haven't had the issue with other SteamVR, WMR, or Oculus headsets. But, before VBS update 1.13 the program would crash around here. UE would throw this error stack trace. Now, it just does this.

This is about where my knowledge stops...any help or avenues to try is appreciated!

Edited January 17 by star-sri