[How to get output from the eye-tracker faster than Update in Unity]

I found a good Black Friday deal, so I bought a new laptop. And now it looks like I get 8 ms intervals.

The values in the 100ns column increase by 40000 per row (which I think relates to 4 ms) and then I get the same SRanipal output, in two rows so that would be 8 ms.

I still will try to test this with Unity update rates higher than 120 fps, as I need that for my application. But I expect that the eyetracking will still run at 120Hz

[Vive Pro Eye: Finding a single eye origin in world space]

@monetl Yes, thanks, I looked at that article and am trying out the script.

[How to get output from the eye-tracker faster than Update in Unity]

Thanks @imarin18

I did let the test run for a while. I did the practice tests and the first run (60 seconds, if I remember correctly). But even at the end of the output file the interval is 16/17 ms. Do you know if there is a way to check the clock cycle, besides running an empty while loop. Is that related to the CPU frequency?

And thanks for the explanation of the 'hanging'. I didn't actually examine the script, I just ran the test. I did notice that he output file size was still increasing when I thought the editor was hanging, so now I understand.

[How to get output from the eye-tracker faster than Update in Unity]

@imarin18 Thanks a lot for this link. I helps me understand the eyetracking a lot better.

And I used the project as a check to see if I could get 120Hz. But even at the end of the output file I still see 16/17 ms between samples. So I think this indicates that my computer is simply too slow (it's a five year old laptop, high-end at the time, but not anymore 😃).

Strangely, I did get errors in Unity when I wanted to build the application to see if that made any differences in the sample speed compared to running it in the editor:
'Assets\Scripts\Saccade_measure_rev1.cs(227,17): error CS0234: The type or namespace name 'EditorApplication' does not exist in the namespace 'UnityEditor' (are you missing an assembly reference?)'

I used Unity 2019.2.17, since I had that already installed, but I find it weird that those errors don't show up when I run the application in the editor. I did notice that after running the application in the editor and clicking Play again to stop the application the editor is hanging. And I think I've seen that in another forum post somewhere.

[Vive Pro Eye: Finding a single eye origin in world space]

@monetl To be honest, I don't really remember what I was looking for at that time 😃 I think I solved it by using an avatar head  at the camera position and use the line renderer as a component of that with local coordinates. It may not be 100% accurate, but it looks good enough for now.

My issue right now is getting the 120Hz output of the eyetracker to see if that makes the eyetracking more accurate:

 

[Grab object using script]

Thanks @Dario

What I want is that the grab occurs at the start of the scene. Some background: I have a tennis application that is used for training (not a game) and the athlete can choose from a number of drills. It starts with a selection scene where the athlete can select if they are right- or lefthanded and if they use the controller in their hand or the tracker on the racket. And then they select the drill they want to do. Typically during a practice session they will do several drills. Each drill is a separate Unity scene, so at the start of each scene the racket should be attached automatically to the correct controller or to the tracker and at the correct position and rotation.

You're right, the stickyness itself is not related to the collision detection. But if I use the Transform option the collision is not detected. And I do need the stickyness to avoid that the racket is released right at the next frame after attaching, because the trigger is no longer pressed. This is what happened when I tried the SteamVR interaction system earlier this year (https://steamcommunity.com/app/250820/discussions/7/3148519094654061096/).

I created a new topic for the Following Duration question: 

 

[Set Grabbable Following Duration to less than 0.02]

Is it possible to make the Following Duration shorter than 0.02 (seconds, I guess)? I see a remark in the code that it depends on the Update rate, so if I would set fixedDeltaTime to let's say 0.001 (1 ms), can I then get a following duration of 0.001 as well? ( I know I need to add both the [SteamVR] and [ViveInputUtility] prefabs in the scene and in both uncheck the 'Lock physics update rate to render frequency' option to avoid falling back to 90 fps at runtime).

The remark I referred to is in RigidPose.SetRigidBodyVelocity(), which is called from GrabbableBase.OnGrabRigidBody().

I understand that you need some time to calculate the velocity and that more time means greater accuracy. But more time also means more lag; the attached object seems to be on an elastic band to the controller rather than securely fixed. So my ideal solution would be to specify this in frames rather than seconds. Then as a developer I can select how much accuracy I want by selecting the number of frames and I can influence the lag by setting the framerate (fixedDeltaTime). Higher framerate (lower fixedDeltaTime) means less lag.

[How to get output from the eye-tracker faster than Update in Unity]

@Asish What issues did you need to solve? I am also getting around 60/70 Hz output. I added a timestamp in the log and there is 15 or 16 miliseconds between the entries in the csv. It makes no difference if I run the scene in the editor or in a build. I have set the fixed Timestep in Unity to 0.008.

Also the gaze ray does not work (which looks logical, as it is only set in StartRecord, so that would mean only once, right?).

[Grab object using script]

@chengnay Here's what I did as a test:

I started with the ColliderEvent example scene and imported my racket and ball. I found out that the behaviour I need is the one that is on the StickyBockableRigidbody, so I copied the Sticky Grabbable script component from that object to my tennis racket. Now when I move either the left controller or the right controller over the tennis racket and press the trigger the racket is attached to the controller. And it's sticky, so I can release the trigger.

What I would like to happen is that I have my own script, RacketControlSelector which does something like this (pseudo-code);

switch (player.dominantHand)
{
    case Right:
      GrabRacketWithController(RIght);
      break;
    case Left:
       GrabRacketWithController(Left);
       break;
}

void GrabRacketWithController (controllerToUse)
{
    // Call the functionality in StickyGrabbable that is executed when the user presses the trigger Is this OnGrabRigidBody()?
    // You could see it as a virtual trigger press by controllerToUse
    racket.GetCompnent<StickyGrabbable>().OnGrabWithController(controllerToUse);
}

The remark about the Following Duration is in RigidPose.SetRigidBodyVelocity(), which is called from GrabbableBase.OnGrabRigidBody()

[Grab object using script]

I am investigating to see if Vive Input Utility can replace NewtonVR for proper collision detection in my application. It looks like the Sticky Grabbable component works in a similar way and my tennis racket does not go right through the ball, so that's promising.

But It only works when pressing a button. There is a large range of possible buttons to select from, but in my application I am using a script to attach the racket to the correct controller (depending on the player's dominant hand) or tracker if that is used. I don't want to bother the player with the action of picking up the racket in every drill they do.

How can I achieve this?

And another question (but maybe I should make a separate thread out of it): Is it possible to make the Following Duration shorter than 0.02 (seconds, I guess)? I see a remark in the code that it depends on the Update rate, so if I would set fixedDeltaTime to let's say 0.001 (1 ms), can I then get a following duration of 0.001 as well? ( I know I need to add both the [SteamVR] and [ViveInputUtility] prefabs in the scene and in both uncheck the 'Lock physics update rate to render frequency' option to avoid falling back to 90 fps at runtime)

Thanks!

[Dropdown selection problem]

Great, thanks! That worked for me as well.

And I have to admit I did not check the Unity forum this time, only the Vive forum 

[Dropdown selection problem]

I am using a dropdown menu in my application and I noticed that it is very hard to select an option. It looks like the problem is that the selection is not registered when I am moving the controller while clicking. This happens very easily.

The problem also occurs in the sample UGUI scene. The other UGUI items such as a toggle are a lot less sensitive and they are easy to select.

I hope someone can help me get this fixed.
Thanks!

[Unity fixed update rate problem]

I found out that the [ViveInputUtility] prefab has its own Lock Physics Update Rate flag. I need to set both to false to be able to modify the framerate. 

So the issue is solved, but I find it strange that VIU has its own flag and does not use the SteamVR flag.

[Vive Pro Eye Calibration Initialization Error Troubleshooting]

It's the Windows User Account Control.

Rightclick on the Start menu icon and select Search. Then type UAC and should see something like (don't have the English Windows version) 'Change settings for User Account Control'. You can also get there through the Configuration screen --> User Accounts --> User Accounts.

In the popup you need to move the slider all the way down. Yes, that is the 'Not recommended' setting.

[Vive Pro Eye Calibration Initialization Error Troubleshooting]

I got rid of the popup by disabling User Account Control completely. This is of course not what you want and also not what you want to tell your customers.

On the other hand, one of the partners that I develop for does not report the issue, so it could be related to which account is used when installing the Vive software. Other threads also suggest that running it when looged in with the Windows administrator account works (which is the actual Administrator account, not a user account with Administrator properties, I guess, so also Run As Administrator will not help). I also noted before I disabled UAC that if I had allowed it once it woulld run without popup for subsequent checks. But as soon as I restarted the PC it needed permission again.

[Vive Pro Eye: Finding a single eye origin in world space]

The sample itself does not match my use case. It uses Focus() to check if the gaze hits a collider. I need to know the origin and direction of the gaze, even if the user is not looking at an object. I log that data and show it to the user afterwards to give them feedback on where they were looking at.

And I traced the method calls and saw that Fccus() calls GetGazeRay(index, out ray) and then GetGazeRay(index, out origin, out direction) and that last one is the same as I am using.

But I did learn in the meantime that I can use the LineRenderer with local space. If I do that and set the points to:
0,gazeOriginLocal
1,gazeOriginLoca+gazeDIrectionLocal*lengthOfRay
I get a quite accurate gaze pointer.

It still would be great to get world coordinates as well, since it requires converting the gazeDirectionLocal to a world verctor, using the head rotation, so if someone has a good method for that I would be very interested (Vector3.Cross is not the right one, I think, as that gives a vector perpendicular to two other vectors and we need something like an addition).

And to get back to my original question, it looks like the asnwer is yes, system origin in the diagram is camera.transform.position in Unity.

[Vive Pro Eye: Finding a single eye origin in world space]

Ok, I thought I would be able to edit my previous post, but I can't seem to find that option. I did some more testing using VIU's ViveCameraRig and here's what I found.

I am using the Combined ray and I use a LineRenderer to draw a line from the origin in the direction gazeDirectionLocal.
gazeOriginLocal is fairly constant at (0.001, 0.007, -0.05). When I add that to camera.transform.position, it looks like the origin of the line is behind my eyes. If I look up or down the ray moves a lot more than when it would come from between my eyes. So I discarded the z-component of gazeOriginLocal. Then the ray was more natural, but still its origin seemed to be slightly too high. So I also discarded the y-component and now it looks quite ok.

Since the x-component of gazeOriginLocal is very small I am inclined to say that camera.transform.position is in fact the position right between my eyes. That would mean that gazeOriginLocal has no added value, which sounds strange. Am I correct?

I am now struggling with the rotation, since the direction is local. I notice that when I keep my head straight the line follows my eyes pretty ok. But when I tilt my head up or down, the line goes crazy, it seems to point in the vertically opposite direction (so the line goes up when I tilt my head down and look straight). I expect that it is relative to the hedaset, in other words relative the camera.transform.forward. Is that correct?

It would really help if someone could give me some feedback or support on this. It is very hard to figure this out by myself, with the headset on. Whenever I try to look at what is happening I influence the gazedata of course.

[SRanipal Getting Started Steps]

Thanks, I had found the getting started guide, but I must have missed the second link.

And apologies for the late reaction, I did not get an email notification of your reaction.

[Vive Pro Eye: Finding a single eye origin in world space]

How does this diagram relate to Unity's SteamVR CameraRig or Vive Input Utility's ViveCameraRig? Is System Origin equivalent to the camera's transform.position?

@Daniel_Y @Corvus @Jad @Hank_Li

[SRanipal Getting Started Steps]

I must be missing it, but where can I find the SRanipal Unity API documentation?

Couldn't find it in this thread, nor in the FAQ or in the downloaded zip file. Tried googling it, but I keep coming back to the same pages.

Thanks!

[Vive Pro Eye Calibration Initialization Error Troubleshooting]

I am running into this problem both fom the SteamVR menu and from Unity. I did some troubleshooting, but haven't found a clear repeatable behaviour yet. I tried setting EyeCalibration.exe to run as admin always, but that does not seem to make a difference. Since I need to invoke it from Unity I can't set this at runtime (but I still have to, but I'll get to that). Also running this .exe manually before running my Unity application is not a good solution for my customers. I have the latest SteamVR and Sr and Tobii version installed.

Here's what I did and the calibration behaviour. I ran these test with EyeCaliration not running as administrator.

Testcase 1
Restart my PC
Run calibration from inside SteamVR: got the UserAccountControl popup asking for permission for SRanipal. Clicked allow en got Initialization Failed in the calibration.
Run it again from inside SteamVR and now it works fine, without UAC popup
Run it once more from inside SteamVR and again runs fine.
Quit SteamVR
Restart SteamVR
Run calibration from inside SteamVR and this time it failed, but without the UAC popup
Run it again from inside SteamVR and it runs fine.
Quit SteamVR
Restart SteamVR
Run calibration from inside SteamVR and this time it worked fine.
Start my Unity application from the .exe file
Run calibration from inside Unity exe and it worked fine
Quit Unity application exe file
Start my Unity application from the Unity editor
Run calibration from inside Unity editor and it worked fine

Testcase 2
Restart my PC
Start my Unity application from the .exe file
Run calibration from inside Unity application: got the UserAccountControl popup asking for permission for SRanipal. Clicked allow en got Initialization Failed in the calibration.
Run it again from inside Unity application and now it works fine, without UAC popup
Quit Unity application exe (this caused a SteamVR crash, don't now why)
Start my Unity application from the Unity editor
Run calibration from inside Unity editor: got the UserAccountControl popup asking for prermission. Clicked allow en got Initialization Failed in the calibration.
Run it again from inside Unity editor and now I got Initialization Failed, but without UAC popup
Run it once more from inside Unity editor and now it worked fine
Quit Unity editor (SteamVR is still running)
Run calibration from inside SteamVR and now it works fine, without UAC popup

So, no real pattern. At one point I thought I crashed only the first time because of the UAC popup, but the tests show that that is not always true. I noticed that when I quit my application exe file (had to do this from the task manager because it was running fullscreen) that the EyeCalibrationDashboard.exe was running. I checked if that started at the same time as EyeCalibration.exe, but it seems not to.

Hope someone can find a way tro solve this. Like I said, for customers this is not user-friendly.

Thanks.

@Corvus

[Unity fixed update rate problem]

I'm not sure if this is the right place to ask this question (I posted it on the Unity site as well), but I'm a Vive developer and I need support, so that brought me to this forum 🙂

 

I am developing a baseball application for the HTC Vive using SteamVR. Since a lot of people have difficulty hitting the ball I wanted to try and change the fixed update rate to see if that made it better. So I made a script that allows me to change Time.fixedDeltaTime and display the value on screen.

 

I know that SteamVR by default locks the fixed update rate to its display framerate (90 fps, Time.fixedDeltaTime=0.0111111), so I added the [SteamVR] prefab to my scene and set the checkbox that controls this locking to false. Made a build and tested it and it seems to work. I have the feeling I get better results at smaller fixedDeltaTime values. But then I tried it as well in the editor and there the confusion set in.

 

In the editor the checkbox in [SteamVR] was set to true at runtime, even though I had set it to false at design time. I modified my script to show the value  that the script sees and it showed False. So the editor does not represent the situation in code. Also, the fixedDeltaTime value in project settings is still 0.011111, even though my scene is running and on screen I see a different value (0.004, 250 fps).

 

To make it even worse, I tried the same for my tennis app, which is still running on Unity 2017 (the baseball app is Unity 2019). And guess what, in Unity 2017 the editor checkbox is false and the project settings show the same fixedDeltaTime as on screen!

 

So I hope someone can explain why this happens and reassure me that the values in the code are the correct ones. Thanks!

 

@Jad

[How to manage your trackers?]

I found the reason. In the selection Scene I am not using VIU. I have the regular SteamVR [CameraRig]. When I added the VivecameraRig it works.

I guess this should be added to the documentation as pre-requisite.

 

@chengnay

[How to manage your trackers?]

@chengnay I tried it in my start-up scene. And just now as a test I tried it in two different scenes and there it works. I can see the pop-up menu and I can see the VIUBindingInterface(Clone) GameObject. In my selection scene that GameObject is not created. The selection scene has an on-screen canvas and in-world canvasses, but one of the other scenes as well.

I tried a different key combination (Esc + B) to make sure the old one was not used for something else, but that also did not work in the selection scene.

[How to manage your trackers?]

@Dario, unfortunately it does not work.

Upgraded to the latest VIU (although in the asset store it said v1.10.7 and in the settings I see now that it says 1.10.6, even after restarting my project. Weird....). The settings are as shown in the link you sent.

When I press right-shift and B nothing happens. No pop-up screen, no errors or warnings, just nothing. I tried the repair define symbols, just to see if that made a difference, but no effect.

Is there a pre-requisite that is not mentioned in the description? I am using SteamVR 1.2.3, because when updating to 2.x earlier the open bindings button did not work. I just now tried the update again (to 2.5.0) and altough that button works I get a gazillion of compile errors because SteamVR_Controller and _ControllerManager and _TrackedObject no longer exist. Not only in my code but also in NewtonVR, which is no longer maintained. Great! And after this SteamVR update I don't see the VIU settings in the preferences anymore, so that seems to be broken now as well. Great!

@chengnay