Expanded depth map refinement

[Æber]

I've been working on a project using the depth map occlusion offered by SRWorks and am relatively happy with the possibilities offered by the SDK. The depth map does have quite a bit of noise in it, presumably due to the limited resolution of the cameras. Enabling refinement improves the image, but I there are still some artifacts present.

For my project I only need to differentiate between two regions: close and far. Everything that is beyond the far occlusion distance should be culled. Since I only need such a simplified depth map, I thought about using OpenCV to process the map to my requirements. Unfortunately, I ran into trouble trying to convert my OpenCV mat back into the Rfloat texture format seemingly required by the material/shader. SetPixels only works for limited formats, so I'm at a loss for how to turn my mat back into a Rfloat.

Is there a way to either configure the refinement process used by SRWorks, or implement additional processing using OpenCV to apply connected component analysis or similar?

Attached is a screenshot of the current depth map with occlusion visualised. Ideally, I would like to refine the map so that the small islands disappear.

 

 

 

 

[Daniel_Y]

You could try Eroding and Dilating in OpenCV to remove small islands, referring to https://docs.opencv.org/2.4/doc/tutorials/imgproc/erosion_dilatation/erosion_dilatation.html

[Æber]

Thank you for your quick answer. Indeed I have already tried both dilation and bilateral texture filtering, but my bigger problem remains how I can convert the processed OpenCV mat back into the Rfloat Texture2D format.

@Daniel_Y

 

 

[dario]

There's several ways, one is Texture2D.LoadRawTextureData but more efficient would be Texture2D.setPixels32 (after resizing, converting to argb and mempy in C++) or you can use some utils in libraries like opencvsharp.

 

@Æber