Use Depth Image to Occlude Virtual Objects in Unity

[pablocael]

Hi! Im trying to use Depth Image Texture to discard pixels of virtual objects whose depth are behind real objects. For instance, if my hand passes in front of a virtual cube, it would occlude it.

 

The first naive ideia is to use a shader to directly compare depth values from virtual scene (using Depth Buffer) and real world (using HTC Depth Texture). However, those textures does not have the same proportion.

If there a way of doing this using resources from SRWorks?

 

A more depth think brought me to think that I would need to sample world points and generate a texture for each eye (since occlusion occurs differently for each eye). However I think I would need some kind of "raycast" for generating the depth values.

 

Thanks!

[Ethan Lin]

Hi pablocael,

 

Yes, you can do this in SRWorks now. Please try the following steps:

[in Attached Pic 1]

1. Place a quad under "[ViveSR]->DaulCamera(head)->TrackedCamera(Left)->Anchor(Left)" (As a sibling to ImagePlane-left, and also, check the layer is still "Default")

2. Make the quad local position=(0,0,2), scale=(4.57, 3.42, 1). (I believe that is what you mentioned "does not have the same proportion")

3. Use CameraDepthMaskMaterial, Color Write to "None" (Set to "All" for visualization)

[in Attached Pic 2]

4. When running with enabling depth processing, check the toggle box "Update Depth Material"

5. You can try to toggle some other settings in the blue squares

6. There you go :smileyhappy: !

 

Cheers

[pablocael]

Thanks! It worked!!

[vladstorm]

wait, is the hand tracking work already or it's just hands depth on the photo?

 

[dario]

Yes and no, if you check the "Run Depth Mesh Collider" you will be able to collide the dynamic meshes (which can include your hands) with virtual objects but not just your hands exclusuvely.  We'll be sharing code to detect the hands soon enough but you can start here with this solution depending on your use case.

[sjobom]

Is there a way to do something similar in Unreal?

[dario]

There should be as the underlying native APIs are the same, so depending if you're ok with looking at and calling native code.  We will be looking into providing similar solutions for UE4 in future updates.

 

 

[DaKenpachi]

, ,

Is there a way to enable depth processing and it´s settings by default (meaning not during runtime), so I don´t have to enable them again everytime I start?

[dario]

ViveSR_DualCameraImageCapture.EnableDepthProcess(true);

[dario]

For an example that shows how to set these settings in code check this hand interaction and occlusion example:  

 

https://github.com/ViveSoftware/ViveSRWorksHand