Jump to content

Getting Screen Position of the Gaze in Unity

Recommended Posts

  • 1 month later...



I have recently started working with unity and Sranipal.  I have tried to follow the steps described in the recent posts here but still have a few questions related to the conversion between gaze direction screen and world coordinates...

1. In what way gaze direction is related to the screen coordinates in unity? is this the correct conversion to pixels?

  Left_E_gaze_direction_normalized = verboseData.left.gaze_direction_normalized;

x_pix_resolution=1440;  y_pix_resolution=1600;

 left_eye_x_pixels = Left_E_gaze_direction_normalized.x / x_pix_resolution;
  left_eye_z_pixels = Left_E_gaze_direction_normalized.z / y_pix_resolution;

2. Conversion to world coordinates:

Left_eye_position_world_coordinates= Camera.ScreenToWorldPoint( left_eye_x_pixels, left_eye_z_pixels, left_eye_y_pixels);

3. In what way gaze origin is related to these transformations? /or if it isn't why is it provided? 

I would greatly appreciate your help:)






  • Like 2
Link to comment
Share on other sites

  • 4 weeks later...

Hi, @neomimiz @scyudits @Playnary

Validity of the data: 





If I understand correctly, the following shows how it works, though I am not sure if this is correct.


Binary digit if valid

Decimal digit if valid
















Calculation of visual angle:

Are you asking how to convert a GAZE vector into a rotated vector?
we have sample code for this In SDK unity and unreal sample plugin  
AvartarSRanipal_v2.cs line:189

Gaze origin and pupil position: 
These are the values generated by Tobi, I think they should be used to calculate the gaze vector

Link to comment
Share on other sites

  • 2 years later...
On 11/6/2020 at 1:03 AM, AdamStreck said:

I did not figure it out. Would you please post details @Av ?

I know where the issue lies, though, it's a difference between the view provided by "Left Eye" vs the full image rendered to the headset.

I have made a simple example, where the camera coordinates of an object are printed directly on screen. The project is attached (EyeTrackingIssue.zip), but the full code is the following:

using UnityEngine;
using UnityEngine.UI;
using UnityEngine.XR;

public class DisplaySSCoordinates : MonoBehaviour
    public Transform debugSphere;
    public Text displayText;
    public Camera vrCamera;
    private void Update()
        var coords = vrCamera.WorldToScreenPoint(debugSphere.position, Camera.MonoOrStereoscopicEye.Left);
        var eyeTextureSize = new Vector2(XRSettings.eyeTextureWidth, XRSettings.eyeTextureHeight);
        var units = new Vector2(coords.x / eyeTextureSize.x, coords.y / eyeTextureSize.y);
        displayText.text = $"({units.x:F2},{units.y:F2})";

You can see that in the bottom-left and top-right corners, the coordinates are not what I would expect. For the center it is correct:




However when I switch from the "LeftEye" to "OcclusionMesh", the coordinates are correct:


This is obviously not SRanipal issue only, probably will also happen for other EyeTracking systems, however I think this is the prescribed way of getting the screen-space position (it's a while I'm not sure, I'm happy for a different workflow).

I noticed that the error changes with the aspect ratio, which makes sense as the screen-crop of the "LeftEye" changes too.

I'd need to obtain the information how the "EyeTexture" is cropped into the game view to calculate the position correctly. Any recommendations?



EyeTrackingIssue.zip 23.88 kB · 26 downloads

Hi, I had a similar problem with my recent development. Where is the "OcclusionMesh" you mentioned turned on or off? I hope to be able to get at least the exact screen coordinates of the eye in this way. Looking forward to your reply.

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Create New...