[UPDATE][Wave 4.5.0] Demonstration on how to use WAVE Passthrough Underlay/Overlay APIs

[Tony PH Lin]

There’re many inquires on how to use Passthrough image or access raw camera images to create a Mixed Reality experience or MR effects.
Due to privacy policy, we do not offer ways to access the raw camera images, but here’re some examples on how to use our existing WAVE Passthrough Underlay/Overlay APIs.
The following message will show you the difference between the Passthrough Underlay and Overlay, you may choose which one is best suited for your content design.

Passthrough Underlay

By calling "WVR _ ShowPassthroughUnderlay" API, you can show the Passthrough images underneath the original rendering content.

Here are some examples that use Passthrough Underlay, such as VIVE Room Setup, VR boundaries, and 3D Objects.

Passthrough Overlay

By calling "WVR _ ShowPassthroughOverlay", the Passthrough image will be showing on top of everything in the scene.

[UPDATE] From Wave 4.5.0, it's allowed to set passthrough overlay alpha value. 

[michela]

>Due to privacy policy, we do not offer ways to access the raw camera images, but here’re some examples on how to use our existing WAVE Passthrough Underlay/Overlay APIs.

Are these APis supported on Focus 3? @Tony PH Lin

[Tony PH Lin]

Hi @michela,

Yes, these APIs are supported on Focus 3 from Wave 4.x series.

Let us know if you need further detail info.

Thanks.

 

[Yeongjae_Choi]

2 hours ago, Tony PH Lin said:

Hi @Tony PH Lin,

I am in need of using Passthrough Overlay/Underlay function on Focus 3 from Wave 4.x series.

Could you give me further detail info?

I have no experience in Wave Native SDK because I develop XR contents with Unity environment.

Could you let me know how to access Passthrough Overlay/Underlay function under Unity environment?

Specifically, I need further detail info regarding how to use access Wave Native SDK in C# script.

Thanks in advance.

 

Edited March 10, 2022 by Yeongjae_Choi

[Alex_HTC]

10 hours ago, Yeongjae_Choi said:

 

Hi @Yeongjae_Choi

An example showing how to use these apis is here:
https://github.com/hardcoded2/WavePassthroughOverlayExample

It is my understanding that the above will work on focus3 and flow support is forthcoming.

[michela]

> Yes, these APIs are supported on Focus 3 from Wave 4.x series.

Thanks Tony! Would you have an Unreal Engine example?

[Yeongjae_Choi]

On 3/10/2022 at 11:53 PM, Alex_HTC said:

An example showing how to use these apis is here:
https://github.com/hardcoded2/WavePassthroughOverlayExample

It is my understanding that the above will work on focus3 and flow support is forthcoming.

Hi, @Alex_HTC

Thank you for your sharing 'WavePassthroughOverlayExample' project.

To build and run your project, I did the following steps:

(Step 1) Open your project with Unity 2019.4.30f1.
(Step 2) Open 'SampleScene' under 'Assets > Scenes'
(Step 3) Switch 'Build Settings' from PC to Android by clicking 'Switch Platform' button
(Step 4) Click 'Build And Run' button

But, it fails to run on my Vive Focus 3 device with the following error message:

Could you give me some help for fixing the problem?

PS1.
To keep my Vive Focus 3 device up-to-date,
I updated to VIVE Focus 3 Software (FOTA 3.3) 3.0.999.368 which was released February 23rd, 2022.

PS2.
I also tried to change your project's minimum API level from 26 to 24 under 'Project Settings > Player > Other Settings > Minimum API Level',
but it still fails to run on my Vive Focus 3 device with the following error message:

Edited March 14, 2022 by Yeongjae_Choi

[Yeongjae_Choi]

19 hours ago, Yeongjae_Choi said:

Hi, @Alex_HTC

Thank you for your sharing 'WavePassthroughOverlayExample' project.

To build and run your project, I did the following steps:

(Step 1) Open your project with Unity 2019.4.30f1.
(Step 2) Open 'SampleScene' under 'Assets > Scenes'
(Step 3) Switch 'Build Settings' from PC to Android by clicking 'Switch Platform' button
(Step 4) Click 'Build And Run' button

But, it fails to run on my Vive Focus 3 device with the following error message:

Could you give me some help for fixing the problem?

PS1.
To keep my Vive Focus 3 device up-to-date,
I updated to VIVE Focus 3 Software (FOTA 3.3) 3.0.999.368 which was released February 23rd, 2022.

PS2.
I also tried to change your project's minimum API level from 26 to 24 under 'Project Settings > Player > Other Settings > Minimum API Level',
but it still fails to run on my Vive Focus 3 device with the following error message:

 

Sorry for giving you inconvenience.

I found that the problem was due to the improper recognition of Vive Focus3 device on my desktop PC.

I repeated connecting and disconnecting Vive Focus3 device to my desktop PC, several times.

And it is finally recognized correctly as follows:

 

Thank you for your help.

[michela]

>Yes, these APIs are supported on Focus 3 from Wave 4.x series.

 

On UE4.26.2 this example didn't work for us. Can you confirm 

 

Error	LNK2019	unresolved external symbol WVR_ShowPassthroughUnderlay referenced in function "public: virtual enum WVR_Result __cdecl FWaveVRAPIWrapper::ShowPassthroughUnderlay(bool)" (?ShowPassthroughUnderlay@FWaveVRAPIWrapper@@UEAA?AW4WVR_Result@@_N@Z)

from adding the following function to WaveVRAPIWrapper.h
Noting that WaveVRAPIWrapper is also using functions from the same Library (wvr_system.h)
/* Passthrough */
virtual WVR_Result ShowPassthroughUnderlay(bool show) { return WVR_ShowPassthroughUnderlay(show); }

[hyunaseo]

Underlay will be very useful to many researchers. Thanks!!

Is this underlay API available on the Unreal Engine too?