Raw Camera Data in Unity

[rengle820]

Is there a way to get the raw camera pixel data from the Vive Focus in Unity? I have the Camera Sample working, but I can't seem to get the pixels out in a performant way.

 

Any help is appreciated!

[rengle820]

I'm not sure it's the right way to do it, but I figured out a way to get the pixel buffer.

 

New question... Is there a way to get a higher res image from the cameras? 1280x400 is pretty low res. When I do pass-through natively it looks a lot higher res than that.

[norair]

Could you tell me how you managed to get the "pixel buffer"?

 

Have you tried with the WaveVR_CameraTexture class?

[rengle820]

Yeah, I used the WaveVR_CameraTexture class and the Camera sample to get started.

I created a function to process the data that gets called BEFORE update. I was calling it after update, which was not working.

 

Here's the gist of the function:

 

Texture mainTexture = meshrenderer.material.mainTexture;            if (destTex == null) {                destTex = new Texture2D(mainTexture.width / 2, mainTexture.height, TextureFormat.RGBA32, false);            }            RenderTexture currentRT = RenderTexture.active;            if (renderTexture == null) {                renderTexture = new RenderTexture(mainTexture.width, mainTexture.height, 32);            }            Graphics.Blit(mainTexture, renderTexture);            RenderTexture.active = renderTexture;            destTex.ReadPixels(new UnityEngine.Rect(0, 0, renderTexture.width / 2, renderTexture.height), 0, 0);                        RenderTexture.active = currentRT;

 var data = destTex.GetRawTextureData<Color32>();

And from there you can access the raw pixel data using the "data" variable.

 

Framerate seems pretty good, and you can even pass destTex to OpenCV for computer vision processing.

[NachoC]

 Hi, what frame rate are you getting?

 

Whenever I get the cameras texture, just to display it on a quad, like the 'CameraTexture_Test' sample, my framerate drops from 72 to 60 fps.

 

By the way, how do you use 'data' var? I tried saving a frame in order to find the lens distortion this way, but did not work:

byte[] bytes = Color32ArrayToByteArray(data.ToArray());File.WriteAllBytes(Application.persistentDataPath + "/SavedScreen" + Time.frameCount + ".png", bytes);

Thanks!