Play a real piano while in VR

[alforno]

Hi
would it be possible to play a real (physical) grand piano, i.e. see the whole keyboard and at the same time be in a VR environment. Has this been done?
 

[HackPerception]

@alforno - It's technically possible but the technology is not fully there yet. There is a sidequest app called VRtous that demos the idea but at the end of the day handtracking technology isn't fully baked right now and there's a limit on how much precision you can get. In another generation or two - pass through and hand tracking will be good enough for this to work really well.

 

[alforno]

Thank you VibrantNebula, but what I meant is much simpler. I would just need a camera showing me the real piano keys with my real hands at the same time as creating the virtual space around it, no tracking of hands necessary. In a way just like augmented reality where I have part of my real environment plus the VR. 

[HackPerception]

@alforno Like this? This is an overlay application that lets you create pass through frames into any SteamVR app.  https://store.steampowered.com/app/864790/FragmentVR/

[alforno]

Yeap, exactly. Great a lthanks a lot! Is there any latency in the movements and does this work with any headset that supports steam?

Edited January 17, 2021 by alforno
spelling

[Eric11101992]

That's awesome. The next level would be a haptic feedback glove, so you can practice without a piano :)

[andreasnielsen]

That is some next level stuff.