Chroma Keying with Vive XR Elite

[msoson]

I'm looking to build chroma keying functionality into a research application I’m building.  This would need to be through the Vive or OpenXR SDK in Unity or Unreal game engines.  I understand that Vive does not allow access to the camera image textures to developers for privacy reasons, so wanted to know if there was any planned functionality in those SDK’s for the Vive XR Elite.  The end result would be someone in headset, looking at someone else against a green background, but the background is keyed out and replaced with game space.  Thus the need for some kind of chroma key functionality in headset (not another camera, as with LIV, from my understanding) for my mixed reality experience.

[Konshu]

Could something like this work: https://youtu.be/6_wFfWSNmx0

[msoson]

Hi @Konshu.  There are many ways to chroma key a normal video or image texture in Unreal, even just using materials and not an external plugin, so that's not really the problem I'm inquiring about (though I appreciate the thought).  I'm specifically asking about access to the Vive Elite passthrough texture layer, which Vive has been previously explicit about devs not having access to (for privacy reasons), and then applying chroma keying, or really any material post processing to.