vinciDa

The g_foveatedRenderingEnabled has nothing to do with the frame rate drop because, besides sranipal, I have also cursor-based tracking, which is working fine. The good news that, I have just solved the frame drop problem with the C++ thread. One more question, I am doing the calibration with steamvr. So, while I am running the sranipal runtime, I guess that is sharing the calibrated eye data. Am I right? Or, do I need to add calibration in the code as well?

This is not okay for me. For a research project, it is too much to use a game engine like UE or Unity. Also, the plugin works for UE 5.0 or latest. Back to my problem, (1) how can I get a 120 Hz tracking frequency and (2) that should not block my actual frame rate? I am using DirectX11, OpenVR.

Hi @VIVE_chengnay, I'm back to the problem now. After some re-evaluation, I am now getting an average frame rate of around 32 fps, sometimes 32.99, but never 33 or higher. Without eye tracking, the average frame rate is 90 fps or slightly higher if I even use a mouse cursor as the tracker. So there are currently two challenges (Using Vive Pro Eye, on Windows 10, NVidia VRWorks code base, OpenVR): Enabling eye tracking lowers the frame rate (1/3). How to resolve this problem? I am sure the eye tracker has a 120 Hz frequency. So, How do achieve that? Some random googling suggests it may be an `asynchronous` problem. Another random google even suggest to use ` SR_ANIPAL int RegisterEyeDataCallback(VivesrEyeDataCallback callback);` under `SRanipal_Eye.h`, but do not know how to use it. Here is what I have tried, and looking for your suggestions: #include "include/SRanipal.h" #include "include/SRanipal_Eye.h" #pragma comment (lib, "SRanipal.lib") using namespace ViveSR; ViveSR::anipal::Eye::EyeData eye_data; // as global variable ... bool VRWorksSample::Init(HINSTANCE hInstance) // vrworks method { ... ViveSR::anipal::Initial(ViveSR::anipal::Eye::ANIPAL_TYPE_EYE, NULL); return true; } ... void VRWorksSample::OnRender() // main rendering loop { if (g_foveatedRenderingEnabled) { float GazeNormX = 0.0f; float GazeNormY = 0.0f; ... if (g_pNvFoveatedRenderingHelper->m_eyeTrackingMode == EYE_TRACKING_MODE_SRANIPAL) { ViveSR::anipal::Eye::GetEyeData(&eye_data); GazeNormX = eye_data.verbose_data.combined.eye_data.gaze_direction_normalized.x; GazeNormY = eye_data.verbose_data.combined.eye_data.gaze_direction_normalized.y; } // vrworks side g_pNvFoveatedRenderingHelper->UpdateGazeData(GazeNormX, GazeNormY, IsVROrFakeVRActive()); g_pNvFoveatedRenderingHelper->LatchGazeData(); g_gazeFinalLeft = util::makefloat2(g_pNvFoveatedRenderingHelper->m_gazeDataParams.sStereoData.sLeftEye.fGazeNormalizedLocation[0], g_pNvFoveatedRenderingHelper->m_gazeDataParams.sStereoData.sLeftEye.fGazeNormalizedLocation[1]); g_gazeFinalRight = util::makefloat2(g_pNvFoveatedRenderingHelper->m_gazeDataParams.sStereoData.sRightEye.fGazeNormalizedLocation[0], g_pNvFoveatedRenderingHelper->m_gazeDataParams.sStereoData.sRightEye.fGazeNormalizedLocation[1]); }

@VIVE_chengnay, but one thing I have noticed, integrating eye tracking in the code-base (VRWorks' Variable Rate Shading) is dramatically dropping the frame rate. For instance, the `mouse cursor tracking = 60 fps (avg)` whereas when using the `gaze tracking = 7 fps (avg)`. Will it be call normal, or I am doing something wrong? I need the 90 fps

Hi @VIVE_chengnay, I have solved my problem. One last question, what is the difference between version 1 and version 2? For example, there are SRanipal_EyeData_v1.h SRanipal_EyeData_v2.h Also, in Unreal, I see the versions. Which version should I use? V2?

I am talking about the`C_API` actually.

Hi @VIVE_chengnay, thanks for taking your time. the ETM_SRAnipal case is the user interface. You are right, the X and Y values are (0,0). I need the real-time gaze direction. Do I first initialize the eye-engine to get the gaze directions? I am using NVIDIA's VRWorks SDK, and it uses OpenVR. So, if I integrate OpenXR would be a good idea or not. Because OpenVR and OpenXR serve the same purpose. However, I am aware of the OpenXR script for eye tracking. So what are you suggesting, I should ignore the sranipal (legacy) and use openxr? I am using Vive Pro Eye.

#include "SRanipal_SDK/include/SRanipal.h" #include "SRanipal_SDK/include/SRanipal_Eye.h" #include "SRanipal_SDK/include/SRanipal_Enums.h" ... int2 offset = makeint2(rtSceneDesc.width / 2, rtSceneDesc.height / 2); int2 gazeOffset = makeint2(0, 0); switch (m_eyeTrackingMode) { case ETM_MousePointer: { float2 normalizedMousePosition = makefloat2(float(m_mousePosition.x) / float(m_clientWindowSize.x), float(m_mousePosition.y) / float(m_clientWindowSize.y)); int2 scaledMousePosition = makeint2(int(std::round(normalizedMousePosition.x * rtSceneDesc.width)), int(std::round(normalizedMousePosition.y * rtSceneDesc.height))); gazeOffset = scaledMousePosition - offset; break; } case ETM_SRAnipal: { //ViveSR::anipal::Eye::VerboseData::combined; ViveSR::anipal::Eye::EyeData_v2 eye_data_v2; //float *gaze = eye_data_v2.verbose_data.combined.eye_data.gaze_direction_normalized.elem_; int2 gazeOffset = makeint2(int(eye_data_v2.verbose_data.combined.eye_data.gaze_direction_normalized.x),int( eye_data_v2.verbose_data.combined.eye_data.gaze_direction_normalized.y)); break; } The mouse pointer tracking is working, but the eye tracking is fixed at the center of the screen.

@Hank_Li, what are the differences between the versions: `v1` and `v2`

C++ rendering project: [VRWorks Graphics SDK](https://developer.nvidia.com/vrworks#tabs-section); it has variable_rate_shading_dx sample code I am using visual studio 2017 on the win10 (64bit) framework. p.s. which sranipal sdk version (current 1.3.6.8) and sranipal runtime version would you suggest for a stable implementation?

I need the gaze directions (L and R eyes) for C++ rendering engine. Using `v1.3.3.0`. After importing the `bin, lib, include` folders in the rendering engine, I am a bit lost. I can see from the c API, runtime is working: https://imgur.com/e2Z4lf2https://imgur.com/e2Z4lf2