DragonDreams

@Alex_HTC @Tony PH Lin I did some more testing. The XR_HTC_facial_tracking seems to have crashes because the weights array passed in had the correct size according to the specification but the driver seems to expect a larger array. Just made it somewhat larger than required and this made the crashing stop. But this produced another problem. 1) Enabling XR_EXT_eye_gaze_interaction and not enabling XR_HTC_facial_tracking makes eye graze tracking work 2) Enabling XR_HTC_facial_tracking and not enabling XR_EXT_eye_gaze_interaction makes the facial tracking work 3) Enabling XR_EXT_eye_gaze_interaction and enabling XR_HTC_facial_tracking makes facial tracking work but eye gaze tracking always returns 0 values. Hence XR_EXT_eye_gaze_interaction and XR_HTC_facial_tracking can not be enabled at the same time. If both are enabled eye gaze tracking fails to work. Can you please fix this bug in the driver?

I did some more testing. The VIVE provide Unreal example OpenXR project does not seem to use XR_EXT_eye_gaze_interaction nor XR_HTC_facial_tracking so it is of no use to test this problem. Instead I went ahead and modified my own code to individually switch on and off these extensions. My findings are the following. If XR_EXT_eye_gaze_interaction is enabled but XR_HTC_facial_tracking is not enabled then there is no crash and the eye gaze tracking is working. I think the implementation is though violating the OpenXR specification. I had to apply an 180 degree Y-axis rotation to the readings to get a correct orientation. According to the specification such a rotation is not present and thus is a bug. If XR_EXT_eye_gaze_interaction is disabled and XR_HTC_facial_tracking is enabled the driver crashes always. The same if XR_EXT_eye_gaze_interaction is enabled together with XR_HTC_facial_tracking; the driver crashes. Hence the problem with the OpenXR driver is enabling XR_HTC_facial_tracking. This causes the driver to crash.

@Alex_HTC @Tony PH Lin I'm seriously starting getting pissed here! Since weeks I'm getting no response at all! This is NO developer support in the slightest. This here is a serious driver problem. I've not tried to get OpenXR running with Vive on multiple machines using Windows 10 and Windows 11, on fresh installations multiple time, using VIVE Installer and Steam VIVE Console App multiple times. I've tried the Unity, Unreal and my own engine OpenXR implementations and they are either report nothing or (in 90% of cases) plain out crash from inside the Vive OpenXR dlls. This is a huge mess and has to be escalated! We are not talking here about some setup problems but major league issues with your drivers making it impossible to get the hardware running AT ALL! And even if you managed to get the application up without the VIVE driver crashing the performance tanks to under 20FPS due to the driver. Attached is 1 of the many crash logs. In my latest attempts even the eye tracking started to fail to initialize. I'm fed up with this. Please provide some REAL support on this matter. ViveSR_Log.zip

@Tony PH Lin Alex_HTC said you might have some more information about the issue.

I tried to download the SRWorks C++ SDK to examine the pass through support. The latest download file is "SRWorks_v0.9.7.1_Native_C_Library". Inside though are outdated headers (from 2017 totally not matching the documentation website) and starter guide PDF (it lists release version 0.8.5.1). Can you please fix the SRWorks C++ SDK to contain the version 0.9.7.1 as labelled?

@Alex_HTC Any news about this issue?

I checked with vive and steam for updates and there are none. These are the versions I can find: VivePort Desktop: 1.4.16.3 SRanipal Runtime Version: 1.4.6.11 Eye Camera Version: 2.41.0-942e3e4 Lip Camera Version: 50100 I managed now to also test the C example. There I can create the eye and lip engine but I always get 0 value readings. So something is definitely broken there.

I got a bit further. I could not get the unity example working since it desperately tried to use a wand controller and I'm using index controllers. Temporarily using those managed to start the example. There the eye and face tracking works... sort of. The result is mediocre but something is working. Since the demo uses the DLL and I'm using OpenXR I suspect the OpenXR Vive driver to be bugged somehow or there is a hidden switch for the OpenXR which is unknown to me. So the question now is how to get the OpenXR extension working? It is enabled and XrSystemHandTrackingPropertiesEXT.supportsEyeGazeInteraction, XrSystemFacialTrackingPropertiesHTC.supportEyeFacialTracking and XrSystemFacialTrackingPropertiesHTC.supportLipFacialTracking are all "true" yet nothing is reported.

The robot symbol has green eyes and green mouth. The eye calibration tool test (light dots by looking at them) is working. With the Unity example at startup the logo head is visible but then it crashes (or just shuts down) after some time. The C example reports error to not be able to create tracking engine (timeout or something like this). The USB cable I did change already. It's a USB 3 cable on a USB 3 port. This one is working fine and can be ruled out as problem source. The problem is in my test I get for gaze xrLocateSpace returns all 0 (especially flags valid bit and tracking bit not set) and for facial tracking xrGetFacialExpressionsHTC returns all 0. Could it be the VR runtime does not grant my app (and the example apps) access to the hardware although the SRanipal did successfully start up. Any ideas?

@Alex_HTC Is anybody doing support in here or what's this forum for?

@Alex_HTC I found some strange log entries. I don't think they are the reason for the problems but I still think they are strange: II [OpenXR] [2023-04-13 16:04:11] Debug: xrGetSystemProperties: VUID-XrSystemHandTrackingPropertiesEXT-next-next - Invalid structure(s) in "next" chain for XrSystemHandTrackingPropertiesEXT struct "next" II [OpenXR] [2023-04-13 16:04:11] Debug: xrGetSystemProperties: VUID-XrSystemProperties-next-next - Invalid structure(s) in "next" chain for XrSystemProperties struct "next" II [OpenXR] [2023-04-13 16:04:11] Debug: xrGetSystemProperties: VUID-xrGetSystemProperties-properties-parameter - Command xrGetSystemProperties param properties is invalid The code used is this: https://github.com/LordOfDragons/dragengine/blob/e9408aacbb5e546c80dcb3f25f4044327043bb2d/src/modules/vr/openxr/src/deoxrSystem.cpp#L59 With line 96 being the call leading to this log entry. I think the call is correct and I do get the positive result back stating the eye gaze and facial tracking is present. Could it be the validation layer has a bug here? Other than that I get no messages from validation so why am I not getting any input neither from the eye tracker nor from the facial tracker?

I did some more testing and the XR_HTC_facial_tracking also reports always 0 weights for anything. It is as if something is missing so my application gets no gaze nor facial information although the extensions are enabled. I noticed there are VIVE_APILAYER_* layers reported. Are these required for the extensions to work properly? I can not find any information by searching for these layer names on the internet. Right now nothing works at all which is frustrating since the documentation is 0 in that regard.

I have completely removed SRanipal runtime and VIVE Console and reinstalled VIVE Console. The problem persists. I then completely removed all again and tried the VIVE Software download instead. This one seems to install and I can now enable the extensions XR_EXT_eye_gaze_interaction and XR_HTC_facial_tracking. There are some problems where I'm stuck. 1) XR_EXT_eye_gaze_interaction always reports no pose at all although I've done calibration and the calibration test (light up dots by looking at them) is working. I get no error yet the pose action as described in the OpenXR specification is receiving no input. 2) I'm using Index Controllers for finger tracking. The VIVE Pro Eye now for some reason does front camera finger tracking which sort of replaces/overlays the Index Controller finger tracking. This results in the fingers to look broken (bending in the wrong direction) and the hand location is high up in the sky. I have the suspicion that the SRanipal runtime sources both Index Controller and front camera hand tracking and sort of adds them together. I don't know what I can do against this.

Looks like VIVE Console on Steam is required to get the extension. But "VIVE Console" seems to be unstable and crashes. I launched into a debugger and found this: The code in my application where the runtime crashes is this: XrFacialTrackerCreateInfoHTC createInfo; memset( &createInfo, 0, sizeof( createInfo ) ); createInfo.type = XR_TYPE_FACIAL_TRACKER_CREATE_INFO_HTC; createInfo.facialTrackingType = XR_FACIAL_TRACKING_TYPE_EYE_DEFAULT_HTC; instance.xrCreateFacialTrackerHTC( session.GetSession(), &createInfo, &pEyeTracker ); The instruction pointer of this thread is inside this function call. The crashing thread is the SR runtime as shown by the debugger here: > ucrtbase.dll!abort() Unknown ucrtbase.dll!terminate() Unknown vcruntime140.dll!FindHandler<__FrameHandler3>(EHExceptionRecord * pExcept, unsigned __int64 * pRN, _CONTEXT * pContext, _xDISPATCHER_CONTEXT * pDC, const _s_FuncInfo * pFuncInfo, unsigned char recursive, int CatchDepth, unsigned __int64 * pMarkerRN) Line 776 C++ vcruntime140.dll!__InternalCxxFrameHandler<__FrameHandler3>(EHExceptionRecord * pExcept, unsigned __int64 * pRN, _CONTEXT * pContext, _xDISPATCHER_CONTEXT * pDC, const _s_FuncInfo * pFuncInfo, int CatchDepth, unsigned __int64 * pMarkerRN, unsigned char recursive) Line 399 C++ vcruntime140.dll!__CxxFrameHandler3(EHExceptionRecord * pExcept, unsigned __int64 RN, _CONTEXT * pContext, _xDISPATCHER_CONTEXT * pDC) Line 283 C++ ntdll.dll!RtlpExecuteHandlerForException() Unknown ntdll.dll!RtlDispatchException() Unknown ntdll.dll!KiUserExceptionDispatch() Unknown [External Code] ViveSR_Client.dll!00007fff19d5e24b() Unknown ViveSR_Client.dll!00007fff19d5d476() Unknown ViveSR_Client.dll!00007fff19d530f6() Unknown ViveSR_Client.dll!00007fff19d518d4() Unknown ViveSR_Client.dll!00007fff19d521e7() Unknown ViveSR_Client.dll!00007fff19d511a9() Unknown ucrtbase.dll!thread_start<unsigned int (__cdecl*)(void *),1>() Unknown kernel32.dll!BaseThreadInitThunk() Unknown ntdll.dll!RtlUserThreadStart() Unknown Why is ViveSR_Client.dll crashing? Is "VIVE Console" broken? What can I do to prevent this crash?

I managed to get the problem with the SRanipal runtime not running (bad USB cable). Now I tried the SDK examples (FaceGym) to see if the runtime works. This example though crashes and quits which means something is wrong. I then started my DEMoCap (which is the reason for all this) since it supports the required OpenXR extensions XR_EXT_eye_gaze_interaction and XR_HTC_facial_tracking. This did not work either since the SteamVR using the SRanipal runtime does not report these two extensions as present: II [OpenXR] [2023-04-10 19:35:15] Start Runtime II [OpenXR] [2023-04-10 19:35:15] Create OpenXR Instance II [OpenXR] [2023-04-10 19:35:15] Extensions: II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_vulkan_enable: 8 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_vulkan_enable2: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_D3D11_enable: 9 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_D3D12_enable: 9 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_opengl_enable: 10 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_win32_convert_performance_counter_time: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_win32_appcontainer_compatible: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_binding_modification: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_composition_layer_depth: 6 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_visibility_mask: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_active_action_set_priority: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_dpad_binding: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_frame_composition_report: 4 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_hand_tracking: 4 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_hand_joints_motion_range: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_hp_mixed_reality_controller: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_palm_pose: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_FB_display_refresh_rate: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_HTC_vive_cosmos_controller_interaction: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_HTC_vive_focus3_controller_interaction: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_MND_headless: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_VALVE_analog_threshold: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_HTCX_vive_tracker_interaction: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_debug_utils: 4 II [OpenXR] [2023-04-10 19:35:15] Supported Extensions: II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_opengl_enable: 10 II [OpenXR] [2023-04-10 19:35:15] - XR_KHR_visibility_mask: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_hand_joints_motion_range: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_hand_tracking: 4 II [OpenXR] [2023-04-10 19:35:15] - XR_HTCX_vive_tracker_interaction: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_hp_mixed_reality_controller: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_HTC_vive_cosmos_controller_interaction: 1 II [OpenXR] [2023-04-10 19:35:15] - XR_HTC_vive_focus3_controller_interaction: 2 II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_debug_utils: 4 II [OpenXR] [2023-04-10 19:35:15] Not upported Extensions: II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_eye_gaze_interaction II [OpenXR] [2023-04-10 19:35:15] - XR_FB_passthrough II [OpenXR] [2023-04-10 19:35:15] - XR_HTC_facial_tracking II [OpenXR] [2023-04-10 19:35:15] - XR_EXT_samsung_odyssey_controller II [OpenXR] [2023-04-10 19:35:15] - XR_HUAWEI_controller_interaction II [OpenXR] [2023-04-10 19:35:15] - XR_MSFT_hand_interaction Obviously without this extension being reported this all can not work. The runtime reports this in the about dialog: What is missing? Without this extension neither eye tracking nor face tracking is possible.