[PC VR/VBS] VIVE Streaming MR with passthrough

[dchesnut]

Hello,

Would it be possible for someone to provide an example doing this in unreal engine. Right I have the VR template project from unreal and vive buisness streaming with MR enabled. What currently happens is I see nothing but complete pass through with default project settings. I tried jiunlin's suggest and that causes me to see nothing but a black image. I know VBS is connected because tracking is working based on the preview window on screen. If anyone has any insight it will be much appreciated.

 

Thank you

[XIAORAN]

On 8/3/2023 at 3:23 AM, dchesnut said:

Hello,

Would it be possible for someone to provide an example doing this in unreal engine. Right I have the VR template project from unreal and vive buisness streaming with MR enabled. What currently happens is I see nothing but complete pass through with default project settings. I tried jiunlin's suggest and that causes me to see nothing but a black image. I know VBS is connected because tracking is working based on the preview window on screen. If anyone has any insight it will be much appreciated.

 

Thank you

I'm experiencing the same issue as you

[XIAORAN]

On 5/25/2023 at 4:46 PM, Vivi Wu said:

submitting alpha layer to SteamVR Runtime

hello，how can i submit aplha layer to steam VR in unreal engine 5.1

[JL Chen]

On 7/24/2023 at 4:36 PM, Ilija said:

@Vivi Wu
Hello again, I have another question. You see, I managed to make it work with Vive Business Streaming on Focus 3, but the workaround with devmode json doesn't seem to work with Vive Streaming Hub. Updating VSH to beta also doesn't seem to help much -- the passthrough option appears but the app itself shuts down after a few seconds of working. Do you have an advice for that? Help will be much appreciated, thank you in advance!

As long as you can see the option in VS Hub console, the devmode.json works as expected to enable this option. Please check your XR Elite ROM version and the VIVE Streaming client app are both the latest.

If still met app quit back to launcher, please submit VIVE Streaming log via Report issue to us to check. (don't forget to connect USB cable for headset log collecting.)

Edited August 9, 2023 by JL Chen

[JL Chen]

On 7/25/2023 at 9:32 AM, Ray Su said:

hello, is it possible to get passthrough result in webxr, or how can i get the origin result of passthrough

i'd want to implement mr in web by myself, but i don't know where to get the passthrough result

Nope. Passthrough images are not provided to PC content.

[Jerome]

On 7/3/2023 at 8:07 AM, jiunlin said:

You can try in Project Settings > Rendering > Postprocessing, set Enable alpha channel support in post processing to Allow through tonemapper.

So it works but not out of the box with only this option. I had to create a post process to set alpha in frameBuffer post the ToneMapper... find this information on other AR headset supplier documentation... Could be interesting to have a more clear documentation on this.
Now do you think it is possible to get the the depth information of the headset so we can make depth occlusion with 3D ? I tested with a focus 3 so I know it will not possible with this headset but I plan to switch my solution to Elite XR which I think could do this. (still through VBS)

[tito]

Hello everyone , been waiting for this feature since launch of elite XR ! sadly I could not make it work so far ,  got my XR elite updated , streaming Hub running on devmode and MR active ,  tested on unreal 5.1 and 4.27 (source version)  , with the respective version of SDK project files , (the plugin project wich has wave plugin and all the testing maps)

Have all related plugin active , wave VR ,  open XR 

It seems that the pawn does not have hands or fails to active them  , and once i play the levels on VR , all i got its a 100% passtrough vision. with no chance to change anything,  tried to launch the passtrough map , setting on level start some of the options  presented on the widget , like alpha , overlay /underlay , etc , nothing seems to work or at least the 100% passtrough does not change at all.

is there any tutorial on how to make it work on unreal ? do i need to set any specific render setting on unreal aside the tone mapper on alpha channel ? and how to make the pawn work with the wave hands or controls ?

thank you so much !

 

[VincentCXR]

@tito

You can only have either WaveVR or OpenXR active at a time, you cannot have both plugins active together at the same time.  Try disabling the OpenXR plugin, as it seems you're using the Wave sample code.  Hope this helps!

On 8/11/2023 at 9:28 AM, tito said:

Have all related plugin active , wave VR ,  open XR 

 

[loneill]

Hello, does this work with depth occlusion?

[jiunlin]

On 8/22/2023 at 11:29 PM, loneill said:

Hello, does this work with depth occlusion?

depth occlusion is not supported, since currently VBS does not provide scene depth information.