Vive Cosmos OpenXR Hand Tracking Support - Unreal Engine 4.26 Sample Project

[MariosBikos]

Here is a sample project built using Unreal Engine 4.26 showing how to get started using OpenXR Hand Tracking with Vive Cosmos headsets (Project is attached at the bottom of this page).

Please make sure you follow the instructions specified here first to enable OpenXR in the Vive Console Runtime: 

 

Sample Project

The project comes with 2 pre-installed project plugin:

• Vive Cosmos Controller Plugin defines input subcategories for Cosmos controllers.
• OpenXR Vive Cosmos Controller Plugin allows using Vive Cosmos controllers input for your OpenXR applications as it adds the Vive Cosmos controller interaction profile to OpenXR Input (XR_HTC_vive_cosmos_controller_interaction ) 

We have also enabled the following plugins in the project:

• OpenXR Plugin since we want to build an OpenXR app.
• OpenXR Hand Tracking to support the hand tracking extension of OpenXR

• XR Visualization Plugin allows quickly rendering HMDs,controllers,hand meshes using the relevant data as parameters.This makes it easier to quickly render a representation of a virtual hand based on the information we get about each joint. Of course this is optional and it's not required to use it in your project.

 

 

Implementation

After you open the sample project using Unreal Engine 4.26, please check the Event Graph of the Level Blueprint of the default Level "HandTrackingTest".  

We use the GetMotionControllerData function passing as a parameter the Left or Right Hand and we get back information about the MotionControllerData that can be used to render virtual hands. After that we use the RenderMotionController function from the XRVisualization Plugin to render a virtual representation of hands. You can also break the MotionControllerData structure and use the data about the hands in a different way depending on your use case.

Remember that when asking to "GetMotionControllerData" the C++ side of things will try to get Hand Tracker data via the function "FOpenXRHMD::GetMotionControllerData". While trying to get OpenXRHandTracking data, the engine will get data from the runtime and the .json and .dll files provided as shown below. This is automatically handled after you enable the OpenXR runtime on Vive Console.

 

Here's what you should see after hitting the button Play in VR:

OpenXRHandTest.zip

[Morton UE4 Eyetracking Dev]

Heya MariosBikos!

Thank you so much for your support of Vive for Unreal! Your posts and hard work has no doubt helped alot of devs.

I had a question for you, Have you tested SRanipal's Unreal Eye plugin with UE4.26? I can see I works well on UE4.25!

Thanks for your time 

-M

 

[dario]

Hi Morton,

I'm attaching a demo of OpenXR support for eye tracking (and yes using 4.26)

Details forthcoming later this week

 

To clarify this is for Vive Pro Eye using SteamVR OpenXR (but support is installed with Vive OpenXR via an API layer)

OpenXR426_EyeTrackingDartBoard.7z

[Morton UE4 Eyetracking Dev]

Many thanks!

Ah so OpenXR effectively replaces the need for the use of SRanipal's plugin?

Amazing work.

[dario]

10 hours ago, Morton UE4 Eyetracking Dev said:

Many thanks!

Ah so OpenXR effectively replaces the need for the use of SRanipal's plugin?

Amazing work.

No it doesn't techically replace it - it wraps it into a common interface -  I'll be naswering question live in an hour at the VRTO conference

[Piotr Karkocha]

Hi guys,

thanks for posting, the hand tracking helped me a lot.
Yet there is one thing that bothers me - i work currently with 4.27 and whenever I enable OpenXR plugin, there appears a kind of jitter/unstability of the image. When switching back to SteamVR, the image is very stable and the whole experience more fluid.

Have you ever encountered this effect?
I just ported the project from   @MariosBikos to 4.27 and this kind of jitter also appears now (it's definitelly not there on 4.26).

I am using the Focus 3 headset. 

[cloudbeing]

Have you solved this problem? I also use focus3 here, and there are also problems of jitter and instability in the tracking process. The engine version used is 5.1.1