Jump to content

VIVE Eye Tracking at 120hz


Corvus
 Share

Recommended Posts

VIVE Eye Tracking at 120hz


 

Note: First setup the VIVE Eye Tracking SDK and verify basic functionality (calibration & demo scene)

Note: Callback runs on a separate thread to report data at ~120hz

Note: Unity is not thread-safe and cannot call any UnityEngine api from within callback thread.

 

Unity Guide


 

Setup Project for Eye Tracking

  • Import VIVE Eye Tracking SDK Unity Plugin (SRanipal)

  • Add the SRanipal Eye Framework to the Unity scene

    • Drag "SRanipal Eye Framework" prefab into scene hierarchy

      or

    • Attach "SRanipal_Eye_Framework.cs" script to gameobject in scene

  • Setup Framework Settings

    • Check "Enable Eye" (enabled by default)

    • Check "Enable Eye Data Callback"

 

SRanipal Eye Framework Settings

 

Setup EyeData Callback

 

Register EyeData Callback

Note: Only register callback when the eye tracking framework status is working and do not register more than one callback at once.


SRanipal_Eye.WrapperRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));

 

Use EyeData Callback

        private static EyeData eyeData = new EyeData();

        /// Required class for IL2CPP scripting backend support
        internal class MonoPInvokeCallbackAttribute : System.Attribute
        {
            public MonoPInvokeCallbackAttribute() { }
        }

        [MonoPInvokeCallback]
        private static void EyeCallback(ref EyeData eye_data)
        {
            eyeData = eye_data;
            // do stuff with eyeData...
        }

 

Unregister EyeData Callback

Note: Must unregister callback when complete or application is disabled/quit.

                SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));

 

Unity Sample Code


using UnityEngine;
using ViveSR.anipal.Eye;
using System.Runtime.InteropServices;

/// <summary>
/// Example usage for eye tracking callback
/// Note: Callback runs on a separate thread to report at ~120hz.
/// Unity is not threadsafe and cannot call any UnityEngine api from within callback thread.
/// </summary>
public class CallbackExample : MonoBehaviour
{
    private static EyeData eyeData = new EyeData();
    private static bool eye_callback_registered = false;

    private void Update()
    {
        if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING) return;

        if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == true && eye_callback_registered == false)
        {
            SRanipal_Eye.WrapperRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = true;
        }
        else if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == false && eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }
    }

    private void OnDisable()
    {
        Release();
    }

    void OnApplicationQuit()
    {
        Release();
    }

    /// <summary>
    /// Release callback thread when disabled or quit
    /// </summary>
    private static void Release()
    {
        if (eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }
    }

    /// <summary>
    /// Required class for IL2CPP scripting backend support
    /// </summary>
    internal class MonoPInvokeCallbackAttribute : System.Attribute
    {
        public MonoPInvokeCallbackAttribute() { }
    }

    /// <summary>
    /// Eye tracking data callback thread.
    /// Reports data at ~120hz
    /// MonoPInvokeCallback attribute required for IL2CPP scripting backend
    /// </summary>
    /// <param name="eye_data">Reference to latest eye_data</param>
    [MonoPInvokeCallback]
    private static void EyeCallback(ref EyeData eye_data)
    {
        eyeData = eye_data;
        // do stuff with eyeData...
    }
}
  • Like 1
Link to comment
Share on other sites

v2 added more face blend morphs (EyeExpression  expression_data) but otherwise SHOULD be the same. There was a report recently of v2 eyedata timestamp reporting differently than v1 timestamp, if so that would be a bug and will try to get fixed in next release.

Link to comment
Share on other sites

Thanks @Corvus

I followed your instructions and could still not reach the 120 Hz (instead, it is acquired at ~62 Hz). I tried v1 and v2 and only had a static counter incrementing in the callback function. Could it be that it is limited by our HW? We have following specs:

  • Windows 10Pro
  • Intel i7-10750H (specs can be found here)
  • 32GB Ram
  • GeForce RTX 2070 with Max-Q Design

Following tool versions are used:

  • SRanipal SDK & Runtime 1.3.1.1
  • Unity 2019.4.18f1

Am I right in the assumption that the key limiting factor is the CPU and not the GPU? We will soon have a better performing machine to try it out.

PS: If you need a code snippet, please see refer to this thread. Thx!

Link to comment
Share on other sites

@apellisscot Here are my specs and I'm getting 8/9ms updates (~120hz). DM me if you would like a build to test/verify with.

  • Windows 10
  • Intel i7-9750H
  • 16GB Ram
  • RTX 2070 Max-Q

 

  • Unity 2019.4.17f1

 

using UnityEngine;
using ViveSR.anipal.Eye;
using System.Runtime.InteropServices;
using UnityEngine.UI;

/// <summary>
/// Example usage for eye tracking callback
/// Note: Callback runs on a separate thread to report at ~120hz.
/// Unity is not threadsafe and cannot call any UnityEngine api from within callback thread.
/// </summary>
public class test120hz : MonoBehaviour
{
    private static EyeData eyeData = new EyeData();
    private static bool eye_callback_registered = false;

    public Text uiText;
    private float updateSpeed = 0;
    private static float lastTime, currentTime;


    void Update()
    {
        if (SRanipal_Eye_Framework.Status != SRanipal_Eye_Framework.FrameworkStatus.WORKING) return;


        if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == true && eye_callback_registered == false)
        {
            SRanipal_Eye.WrapperRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = true;
        }
        else if (SRanipal_Eye_Framework.Instance.EnableEyeDataCallback == false && eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }

        updateSpeed = currentTime - lastTime;
        uiText.text = updateSpeed.ToString() + " ms";
    }

    private void OnDisable()
    {
        Release();
    }

    void OnApplicationQuit()
    {
        Release();
    }

    /// <summary>
    /// Release callback thread when disabled or quit
    /// </summary>
    private static void Release()
    {
        if (eye_callback_registered == true)
        {
            SRanipal_Eye.WrapperUnRegisterEyeDataCallback(Marshal.GetFunctionPointerForDelegate((SRanipal_Eye.CallbackBasic)EyeCallback));
            eye_callback_registered = false;
        }
    }

    /// <summary>
    /// Required class for IL2CPP scripting backend support
    /// </summary>
    internal class MonoPInvokeCallbackAttribute : System.Attribute
    {
        public MonoPInvokeCallbackAttribute() { }
    }

    /// <summary>
    /// Eye tracking data callback thread.
    /// Reports data at ~120hz
    /// MonoPInvokeCallback attribute required for IL2CPP scripting backend
    /// </summary>
    /// <param name="eye_data">Reference to latest eye_data</param>
    [MonoPInvokeCallback]
    private static void EyeCallback(ref EyeData eye_data)
    {
        eyeData = eye_data;
        // do stuff with eyeData...

        lastTime = currentTime;
        currentTime = eyeData.timestamp;
    }
}

 

Link to comment
Share on other sites

Thanks @Corvus, I had removed the call to "EyeMeasureDataWriter" and placed a static counter there - so similarly as you do. The only difference I see is that I register the callback in the Monobehaviour's "Start()" method and you do it in the "Update()" method. Although I do not see any reason why this should change something, I will give it a try next week.

It would be great if you could send me the build. I've sent you a DM.

Edited by apellisscot
Link to comment
Share on other sites

Thanks @Corvus, it still captures data at a sampling frequency of around 60 Hz (see video attached). Sometimes the period drops to 8 ms, but this occurs rarely. Are there any CPU configurations to perform? I tried with deactivated hyper-threading, but did not have any luck.

Soon we will have a higher performing PC to test with. I will let you know the results there.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

×
×
  • Create New...