Access camera resources to openCV

[aroboyko]

I  try to use find edge in Unity. I think it is to import the camera image to openCV process(for find edge), and import back to the material of unity.
Is any one know how to get camera resource import to openCV?(I use unity SRWorks SDK)
Or there is a better way to achieve the effect I want

@Daniel_Y @Andy.YC_Wang

[もりぞぅ]

I wanted to get the texture of the front camera and used ViveSR_DaulCameraImageCapture.GetUndistortedTexture with the following code, but I could not get the texture well.
 

            Texture2D imgTexture_Left = new Texture2D(ViveSR_DualCameraImageCapture.UndistortedImageWidth, ViveSR_DualCameraImageCapture.UndistortedImageHeight, TextureFormat.RGBA32, false);
            Texture2D imgTexture_Right = new Texture2D(ViveSR_DualCameraImageCapture.UndistortedImageWidth, ViveSR_DualCameraImageCapture.UndistortedImageHeight, TextureFormat.RGBA32, false);
            int frameIndex, timeIndex;
            Matrix4x4 poseLeft, poseRight;
            ViveSR_DualCameraImageCapture.GetUndistortedTexture(out imgTexture_Left, out imgTexture_Right, out frameIndex, out timeIndex, out poseLeft, out poseRight); // Line 97
            quad.GetComponent<Renderer>().material.mainTexture = imgTexture_Left; 

The runtime errors are as follows:

NullReferenceException: Object reference not set to an instance of an object
Vive.Plugin.SR.ViveSR_DualCameraImageCapture.GetUndistortedTexture (UnityEngine.Texture2D& imageLeft, UnityEngine.Texture2D& imageRight, System.Int32& frameIndex, System.Int32& timeIndex, UnityEngine.Matrix4x4& poseLeft, UnityEngine.Matrix4x4& poseRight) (at Assets/ViveSR/Scripts/ViveSR_DualCameraImageCapture.cs:353)
******.*******.Start () (at Assets/********/MyScript.cs:97)

Line 353 of ViveSR_DaulCameraImageCapture is the following code.

           if (DataInfoUndistorted[(int)SeeThroughDataMask.UNDISTORTED_FRAME_LEFT].ptr != IntPtr.Zero && DataInfoUndistorted[(int)SeeThroughDataMask.UNDISTORTED_FRAME_RIGHT].ptr != IntPtr.Zero)

 Please let me know if there is a solution.

Other methods are acceptable.

Thank you.

 

@Sean_Su

[もりぞぅ]

Trying GetDistortedTexture and GetDepthTexture of ViveSR_DaulCameraImageCapture also gave the same error.

Does SRWorks SDK have a function to display the front camera image on the HTC VIVE Pro display?

 

@Daniel_Y @Andy.YC_Wang @Sean_Su

[Daniel_Y]

Could the Unity pre-built demo program run on your PC?

 

@もりぞぅ

[Federica]

@Sean_Su @Daniel_Y

On 12/6/2019 at 10:10 AM, Sean_Su said:

You can get the texture buffer of camera image in ViveSR_DaulCameraImageCapture.cs
There are four textures in our script.(TextureDistoredLeft, TextureDistoredRight, TextureUndistoredLeft, TextureUndistoredRight)
And you can find out more detailed information in Vive.Plugin.SR.ViveSR_DualCameraImageCapture.Initial().

The texture buffer can be converted to opencv mat.

 

@aroboyko

Hi, I can't understand how in the ViveSR_DaulCameraImageCapture.cs script I can convert the texture to opencv mat.

I would like to apply a gaussian blur filter to the textures left and right. 

Can you help me? Thank you.

Edited March 31, 2021 by Federica