Hand tracking test 0.91 Vive Cosmos

[davide445]

@VibrantNebula

looking at other options the only I can find are Occipital Structure, Intel RealSense and Orbbec Astra Mini, all with Gestoos sw for gesture recognition. 

Any experiences with one of them? 

[davide445]

11 hours ago, davide445 said:

@VibrantNebula

looking at other options the only I can find are Occipital Structure, Intel RealSense and Orbbec Astra Mini, all with Gestoos sw for gesture recognition. 

Any experiences with one of them? 

PS also can anyone check if a 13x80mm (0.5x3.2 inch) object (the Leap Motion sensor) can fit horizontally into the Cosmos without occluding the cameras? Movements limited to the office here, so can't check by myself.

[davide445]

I think will be possible. Next step I suppose receive a Leap Motion for a test. Problem will be to fix it, also not sure if is any better place under or over the two cameras, depending on what's behind. 

[davide445]

@zzy @VibrantNebula Received the Leap Motion, was looking at the best way to mount it. Since the 3M stich it's one use didn't want to trial and error.

In this way I'm using his mounting kit, will create troubles with the headset tracking? Controllers tracking it's not more a problem in this case

Edited March 19, 2020 by davide445

[davide445]

I finally mounted the Leap Motion using a biadhesive so to have (for test purpose) the most flexible installation

That said the experience it's better than with HTC SDK but not reliable as was supposed to be in my expectations. 

Waiting for any improvement suggestions from Ultraleap, wanted to ask @zzy if wearing a glove with stamped different markers on any finger will help improving the tracking using the native SDK.

I'm starting to wonder if in general the whole hand tracking its still an r&d topic and not ready for prime time. 

Edited March 21, 2020 by davide445

[zzy]

Hi @davide445

Glad to know you have mounted leap motion successfully. About your question, wearing a glove does not help for Vive Hand Tracking SDK, since our SDK is trained with hands without gloves. Here are some suggestion you can try:

1. Roll up sleeves and take off your watch if any
2. Avoid light sources in the background, including window and lights
3. Try to make your room brighter (e.g. turn on the lights even in day time), but don't look at light sources directly

[davide445]

Thanks @zzy but my idea was not simply gloves but gloves with attached specific QR markers for each finger.

Since you are using a DNN I suppose you will need to train it with also this markers, but this might improve the recognition. 

Also from the user point of view I didn't see a problem in wearing passive simple gloves provided with markers stamped. 

I'm searching and effective solution to a problem I didn't see one so far. 

[davide445]

Just to give an idea. In this way you can also I suppose increase your training set size by also generating it trough 3d simulation, since your focus will be not in fact a real hand but a glove, easier to render.

I know I'm now speaking more about R&D, but if correctly working reliably a native hand tracking using just Cosmos cameras would be the perfect solution.

[zzy]

Hi @davide445

Thanks for the suggestion. I agree that a native working hand tracking using HMD front camera would be perfect solution, this is our goal for the Hand Tracking SDK.

We will consider your suggestion, however since this will require more resources and not as scalable as raw hands, we will put it at lower priority.

[davide445]

@zzy in the meantime I need to find a working solution.

Apart Leap Motion own Orion software, was looking various suppliers of 3d depth cameras are suggesting Gestoos as gesture recognition software. Didn't find it mentioned in this forum, you ever have had experience with it