Mixed reality project on HTC Vive Pro

[SimoneSantillo]

Hi everyone, I'm new to the forum and new to the development on HTC Vive Pro.

I would like to replicate a project - that already works on another HMD - to run on the HTC Vive Pro. The project involves Mixed Reality and I need to grab the pass-through image from the cameras.

I started by following the quick start guide, that actually consists of: i) installing VIVE Registry Tool, ii) installing VIVE Input Utility (VIU) and iii) importing a ViveCameraRig from Prefabs of VIU. After this step, I click on Play button and Unity Scene is correctly rendered to the HMD.

Now, I found out that I should import SRWorks for mixed reality on HTC Vive Pro, but I encounter two problems:

• Following the getting started guide, there are some "camera setting requirements" that I'm not able to find anywhere. Can you give me some hint on this?
• When importing the package in my fresh unity project, I get some errors (see attached image) belonging to the package. These errors suggest that the package is a little bit outdated.

Am I doing everything right? Is SRWorks the right way to achieve mixed reality on HTC Vive Pro?

Thanks in advance for any support! @joshn_vive @Alex_HTC @Corvus

[Alex_HTC]

Hi @SimoneSantillo

The simplest way to get MR example working is with the call:
 

Wave.Native.Interop.WVR_ShowPassthroughUnderlay(show);

Which is shown in a working example project here:
https://github.com/hardcoded2/WavePassthroughOverlayExample

Thanks,
Alex
 

[SimoneSantillo]

Hi @Alex_HTC

I cannot find any tutorial on how to setup a project that is able to run your proposed example.

Anyway, I think you missed my point: I do not need to just display real world, I would like to access to the images of the cameras in my code.

More specifically, my desired workflow is:

• grab the pass-through camera images
• pass them to my C++ DLL
• render the results to the HTC Vive Pro 2

Do you have any tutorials/pointers to follow?

[Alex_HTC]

Hi @SimoneSantillo

The camera apis seem to be restricted, and if you want to use them it is my understanding that you'll need to work with the team to get an enterprise key to use that api.

An alternative would be to use render cameras to capture the overlay or underlay and then pass that frame buffer to an api.

Most use cases like this also want virtual objects displayed with real objects, which is what the underlay feature does.

Thanks,
Alex

[kilic21]

On 12/22/2021 at 12:57 AM, Alex_HTC said:

Hi @SimoneSantillo

The simplest way to get MR example working is with the call:
 

Wave.Native.Interop.WVR_ShowPassthroughUnderlay(show);

Which is shown in a working example project here:
https://github.com/hardcoded2/WavePassthroughOverlayExample

Thanks,
Alex
 

 

Thank you for the link. This exemple is helpful.

[BlueprintBen]

Is there an example Unity project for the Wave SDK that supports PC VR? I haven't been able to use the Wave SDK on Windows - The Samples from any of the 3 packages throw a ton of compile errors in Unity 2020 LTS...

[BlueprintBen]

To provide a bit more data, a blank project that has had the Vive Wave Essense SDK package installed, as well as the linked GitHub project,  throw DllNotFoundException: wave_api from WaveXR_RuntimeInitializeOnLoad.Awake() (these are using v4.3.0 and 4.2.0  of the Wave SDK packages respectively).

Would love to understand how to get this to work as the SRWorks SDK hasn't worked with the Vive Pro for myself and others since its release (I experience the same complete application hang that this post details).

[HackPerception]

@BlueprintBen WaveSDK does not currently have AR/MR capabilities, and there isn't support for SRWorks on Focus 3 via PC Streaming (Vive Business Streaming). 

The only supported headsets for SRWorks are Pro 1 and Pro 2. @Daniel_Y @Alex_HTC @Corvus

[BlueprintBen]

OK, I guess I was mislead by the mention of the Wave SDK in the first place then since the OP is about a Vive Pro. Unfortunately as I mentioned, the SRWorks SDK hasn't worked for us with a Vive Pro for a number of years, exhibiting the problem that this other user mentioned here. SRWorks passthrough has worked well enough for the Cosmos so we'd love to have the Pro 1 (&2) supported but are currently blocked.

[Alex_HTC]

On 1/30/2022 at 10:33 AM, BlueprintBen said:

To provide a bit more data, a blank project that has had the Vive Wave Essense SDK package installed, as well as the linked GitHub project,  throw DllNotFoundException: wave_api from WaveXR_RuntimeInitializeOnLoad.Awake() (these are using v4.3.0 and 4.2.0  of the Wave SDK packages respectively).

Would love to understand how to get this to work as the SRWorks SDK hasn't worked with the Vive Pro for myself and others since its release (I experience the same complete application hang that this post details).

One thing that stands out is that the wave essence and xr sdk should be the same version - so try upgrading the wave xr package to 4.3.0 as well @BlueprintBen