Hand Tracking in AR

[Danisinu]

Hi,

I would like to test Hand Tracking SDK making an app with Unity to Android with augmented reality but the website don't show enough information to make it possible. How can I do that? I tried to take "Sample" scene and to change Main camera for ARCamera. Later, I put into of ARcamera GestureProvider.cs script, set param to CPU and Points2D. And finally, in Player Settings marked Vuforia Augmented Reality, set Minimun API to Android 24 and orientation to landscape left. However, nothing happened. I don't know if I did right or not. I need help!! Please!

Thank in advance

Dani

@zzy

[zzy]

Hi @Danisinu

To be honest, we haven't tested Vive Hand Tracking SDK with Vuforia AR SDK. Here is something that I think might be the reason:

1. Make sure camera permission is granted to the app. We need to use rear camera, so need to have camera permission. If you are using Unity 2018.2 or newer, the SDK can display the dialog at runtime. Otherwise, I would suggest you to grant permission manually.

2. From my understanding, Vuforia is also using the rear camera. I don't think Android API supports sharing some camera stream to 2 callers. To identify this, I would suggest you to see the log using "adb logcat -s Unity Aristo" to show both log of Unity and our plugin (you can show vuforia too if you know the keyword). If Vive Hand tracking failed to start the camera, the log will output errors, which might be a little hint.

Please let me know about your findings, thank you.

Best Regards,

zzy

[Danisinu]

Hi,

I'm using Unity 2018.4 and camera permission is granted. On the other hand, to avoid conflicts, I disabled Main camera and availabled AR camera and put GestureProvider script into. I attached screenshot of camera's configuration and script. Anyway, I will see the log, but...any idea else?

Thanks in advance,

Danisinu

@zzy

[zzy]

Hi @Danisinu

The screenshot looks correct to me, considering Vive Hand Tracking part.

For other ideas, I would suggest you to try Sample scene in Vive Hand Tracking and see if it can detect your hand.

By the way, which mobile phone are you using? Some mobile phone might not support ndk camera2 API, so we cannot start camera on these phones. To test if native camera2 API is supported or not, I recommend you to follow the steps in this post.

Best Regards,

zzy

[Danisinu]

Hi @zzy,

Ok. I tried Sample scene (with VR) using my Xiaomi Redmi Note 5. On the one side, I've not installed any NDK in Unity (as you can see in the first screenshot), I didn't know that I needed NDK camera 2 API. On the other side, I don't know if that make my phone failed when it trys to use the NDK (second screenshot). Finally, I am sure that camera permission is available (third screenshot -> Viewport app).

May it be fixed if I set NDK Camera 2 API in Unity settings (first screenshot)? If it will be possible, where can I get that NDK?

Thanks so much in advance,

Danisinu

 

[zzy]

Hi @Danisinu

There is no need for you to install NDK. The NDK API is used by our native android library. From the adb log, it seems your mobile phone does not support NDK camera API, which makes the detection fail to start. I would suggest you to change to another mobile phone or try to upgrade system ROM if possible.

Another way to double check if the API is available, you can try the NDK Camera2 Sample as I mentioned in previous post.

Best Regards,

zzy

[Danisinu]

Hi @zzy,

I forgot to tell you what happened finally.

As you supposed, the problem was my phone, what didn't support Camera2. So I had to look into another solution to my thesis.

Thank you so much for everything!

Regards,

Danisinu