Jump to content

VIVE Eye Tracking at 120hz


Corvus
 Share

Recommended Posts

On 2/12/2021 at 4:30 PM, Corvus said:

@apellisscot Here are my specs and I'm getting 8/9ms updates (~120hz). DM me if you would like a build to test/verify with.

  • Windows 10
  • Intel i7-9750H
  • 16GB Ram
  • RTX 2070 Max-Q

 

  • Unity 2019.4.17f1

 

using UnityEngine;
using ViveSR.anipal.Eye;
using System.Runtime.InteropServices;
using UnityEngine.UI;

/// <summary>
/// Example usage for eye tracking callback
/// Note: Callback runs on a separate thread to report at ~120hz.
/// Unity is not threadsafe and cannot call any UnityEngine api from within callback thread.
/// </summary>
public class test120hz : MonoBehaviour
{
    private static EyeData eyeData = new EyeData();
    private static bool eye_callback_registered = false;

    public Text uiText;
    private float updateSpeed = 0;
    private static float lastTime, currentTime;


    void Update()
    {
        if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING) return;


        if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == true && eye_callback_registered == false)
        {
            SRanipal_Eye.WrapperRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = true;
        }
        else if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == false && eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }

        updateSpeed = currentTime - lastTime;
        uiText.text = updateSpeed.ToString() + " ms";
    }

    private void OnDisable()
    {
        Release();
    }

    void OnApplicationQuit()
    {
        Release();
    }

    /// <summary>
    /// Release callback thread when disabled or quit
    /// </summary>
    private static void Release()
    {
        if (eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }
    }

    /// <summary>
    /// Required class for IL2CPP scripting backend support
    /// </summary>
    internal class MonoPInvokeCallbackAttribute : System.Attribute
    {
        public MonoPInvokeCallbackAttribute() { }
    }

    /// <summary>
    /// Eye tracking data callback thread.
    /// Reports data at ~120hz
    /// MonoPInvokeCallback attribute required for IL2CPP scripting backend
    /// </summary>
    /// <param name="eye_data">Reference to latest eye_data</param>
    [MonoPInvokeCallback]
    private static void EyeCallback(ref EyeData eye_data)
    {
        eyeData = eye_data;
        // do stuff with eyeData...

        lastTime = currentTime;
        currentTime = eyeData.timestamp;
    }
}

 

Hi everyone,

Thanks for all the discussion. 

I am relatively new on C sharp scripting. Currently, I am trying to extract the focus data for my current research project. 

Can anyone please direct me some examples about how to output the eye data into a file (such as .csv file) based on the code that @Corvus provided? 

I already installed the SRanipal, successfully calibrated the HTC Vive pro eye, and tried the example scenes from Vive SR. 

Any suggestion will be appreciated.

Link to comment
Share on other sites

  • 2 weeks later...
On 2/10/2021 at 2:23 AM, Corvus said:

 

Unity Sample Code


Hi everyone - I get around 230hz or 4-5ms. That is without rendering from Unity though, I only use it to extract data. (Output is coming from a separate application)

Specs:

Win 11 / Unity 2020.2.4
Intel(R) Core(TM) i7-10700 CPU @ 2.90GHz   2.90 GHz
32,0 GB
RTX 3070

Something peculiar though when I use the OP's code - editor and runtime crash on OnApplicationQuit / when I stop it. Full freeze. The only thing that is logged before is:

Quote

[XRInputSubsystem] A device disconnection with the id 1 has been reported but no device with that id was connected.

Anyone having a similar issue?

Link to comment
Share on other sites

  • 1 month later...

Hi all,

I'm trying to get this data recording method to work when the Tobii XR SDK is also active in a scene. Can anyone help me out with this? I know that Tobii is just using SR Anipal SDK, so I'm not sure why this fails when they're both runninng.

Link to comment
Share on other sites

@bfalandays It kind of the opposite situation. I'm not an expert on this product, but I know that SRAnipal is a licensed version of Tobii's runtime a subset of the Tobii XR SDK APIs/featureset. We've basically white-labeled Tobii's runtime and APIs to provide an SDK that enables core functionality out of the box. I think in this case you'd defer to Tobii's stuff completely to prevent interference.

Link to comment
Share on other sites

Quick update on my previous post: I figured out that the Tobii SDK is for some reason interfering specifically with the MonoPInvokeCallback function. Does anyone know what types of things to look for in their SDK that could prevent that from running?

Link to comment
Share on other sites

Finally figured out the conflict between Tobii SDK and SRanipal! For anyone interested, it turns out that having Tobii SDK active will automatically set "EnableEyeDataCallback" in the Vive SR Framework to 'false', even if you have checked the box in the GUI. I'm still not sure what's causing that, but if you go into SR_Anipal_Eye_Framework.cs and set EnableEyeDataCallback = true, this script @Corvus's script will work with Tobii SDK running.

Link to comment
Share on other sites

  • 2 months later...

@Corvus thank you for your code!

I tried using it (as is with minor modifications for saving the data to a csv, see the attached) in different unity-sdk-runtime combinations (sdk- 1.3.1 and 1.3.3,runtime 1.3.1and 1.3.2, unity- 2019.0.4- 2019.4.17)

In all combinations I am not able get a fixed 120 hz sampling. Usually 45% of the frames  have an time interval larger than 8.33 ms. 

What is the reason for this behavior?

I attach here steam vr system report. 

I will be glad to receive your response.

Thank you in advance!

Neomi

 

 

SteamVR-2022-03-20-PM_03_17_29.txt Eyes_Corvus_Method.cs

Link to comment
Share on other sites

@Corvus Just as a clarification to my previous post, I attach here a picture of the histogram of time intervals between frames I get. The intervals vary between 5- 10 ms. 

As I am using SR anipal for research I would like to get a much narrower range focused around 8.33 ms. Is that possible? How can I achieve that?

Thank you again!

Neomi

 

Time_diffrences_betwen_frames_sr_anipal.jpg

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...