VIVE Eye Tracking at 120hz

[deediliu1225]

On 2/12/2021 at 4:30 PM, Corvus said:

@apellisscot Here are my specs and I'm getting 8/9ms updates (~120hz). DM me if you would like a build to test/verify with.

• Windows 10
• Intel i7-9750H
• 16GB Ram
• RTX 2070 Max-Q

 

• Unity 2019.4.17f1

 

using UnityEngine;
using ViveSR.anipal.Eye;
using System.Runtime.InteropServices;
using UnityEngine.UI;

/// <summary>
/// Example usage for eye tracking callback
/// Note: Callback runs on a separate thread to report at ~120hz.
/// Unity is not threadsafe and cannot call any UnityEngine api from within callback thread.
/// </summary>
public class test120hz : MonoBehaviour
{
    private static EyeData eyeData = new EyeData();
    private static bool eye_callback_registered = false;

    public Text uiText;
    private float updateSpeed = 0;
    private static float lastTime, currentTime;


    void Update()
    {
        if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING) return;


        if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == true && eye_callback_registered == false)
        {
            SRanipal_Eye.WrapperRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = true;
        }
        else if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == false && eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }

        updateSpeed = currentTime - lastTime;
        uiText.text = updateSpeed.ToString() + " ms";
    }

    private void OnDisable()
    {
        Release();
    }

    void OnApplicationQuit()
    {
        Release();
    }

    /// <summary>
    /// Release callback thread when disabled or quit
    /// </summary>
    private static void Release()
    {
        if (eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }
    }

    /// <summary>
    /// Required class for IL2CPP scripting backend support
    /// </summary>
    internal class MonoPInvokeCallbackAttribute : System.Attribute
    {
        public MonoPInvokeCallbackAttribute() { }
    }

    /// <summary>
    /// Eye tracking data callback thread.
    /// Reports data at ~120hz
    /// MonoPInvokeCallback attribute required for IL2CPP scripting backend
    /// </summary>
    /// <param name="eye_data">Reference to latest eye_data</param>
    [MonoPInvokeCallback]
    private static void EyeCallback(ref EyeData eye_data)
    {
        eyeData = eye_data;
        // do stuff with eyeData...

        lastTime = currentTime;
        currentTime = eyeData.timestamp;
    }
}

 

Hi everyone,

Thanks for all the discussion. 

I am relatively new on C sharp scripting. Currently, I am trying to extract the focus data for my current research project. 

Can anyone please direct me some examples about how to output the eye data into a file (such as .csv file) based on the code that @Corvus provided? 

I already installed the SRanipal, successfully calibrated the HTC Vive pro eye, and tried the example scenes from Vive SR. 

Any suggestion will be appreciated.

[Playnary]

On 2/10/2021 at 2:23 AM, Corvus said:

 

Unity Sample Code

---

Hi everyone - I get around 230hz or 4-5ms. That is without rendering from Unity though, I only use it to extract data. (Output is coming from a separate application)

Specs:

Win 11 / Unity 2020.2.4
Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz   2.90 GHz
32,0 GB
RTX 3070

Something peculiar though when I use the OP's code - editor and runtime crash on OnApplicationQuit / when I stop it. Full freeze. The only thing that is logged before is:

Quote

[XRInputSubsystem] A device disconnection with the id 1 has been reported but no device with that id was connected.

Anyone having a similar issue?

[Playnary]

Regarding the freeze - found the solution and described it in this thread, because someone else had the same problem there:

 

[bfalandays]

Hi all,

I'm trying to get this data recording method to work when the Tobii XR SDK is also active in a scene. Can anyone help me out with this? I know that Tobii is just using SR Anipal SDK, so I'm not sure why this fails when they're both runninng.

[HackPerception]

@bfalandays It kind of the opposite situation. I'm not an expert on this product, but I know that SRAnipal is a licensed version of Tobii's runtime a subset of the Tobii XR SDK APIs/featureset. We've basically white-labeled Tobii's runtime and APIs to provide an SDK that enables core functionality out of the box. I think in this case you'd defer to Tobii's stuff completely to prevent interference.

[bfalandays]

@HackPerception thanks for that response, didn't realize that. However, I actually tried using Tobii's GetEyeTrackingData() function in the same function, and that is also returning nothing for some reason.

[bfalandays]

Quick update on my previous post: I figured out that the Tobii SDK is for some reason interfering specifically with the MonoPInvokeCallback function. Does anyone know what types of things to look for in their SDK that could prevent that from running?

[bfalandays]

Finally figured out the conflict between Tobii SDK and SRanipal! For anyone interested, it turns out that having Tobii SDK active will automatically set "EnableEyeDataCallback" in the Vive SR Framework to 'false', even if you have checked the box in the GUI. I'm still not sure what's causing that, but if you go into SR_Anipal_Eye_Framework.cs and set EnableEyeDataCallback = true, this script @Corvus's script will work with Tobii SDK running.

[neomimiz]

@Corvus thank you for your code!

I tried using it (as is with minor modifications for saving the data to a csv, see the attached) in different unity-sdk-runtime combinations (sdk- 1.3.1 and 1.3.3,runtime 1.3.1and 1.3.2, unity- 2019.0.4- 2019.4.17)

In all combinations I am not able get a fixed 120 hz sampling. Usually 45% of the frames  have an time interval larger than 8.33 ms. 

What is the reason for this behavior?

I attach here steam vr system report. 

I will be glad to receive your response.

Thank you in advance!

Neomi

 

 

SteamVR-2022-03-20-PM_03_17_29.txt Eyes_Corvus_Method.cs

[neomimiz]

@Corvus Just as a clarification to my previous post, I attach here a picture of the histogram of time intervals between frames I get. The intervals vary between 5- 10 ms. 

As I am using SR anipal for research I would like to get a much narrower range focused around 8.33 ms. Is that possible? How can I achieve that?

Thank you again!

Neomi