eye tracking Getting Screen Position of the Gaze in Unity

[AdamStreck]

Update - Unity has reproduced the issue on their end and flagged a bug: https://issuetracker.unity3d.com/issues/xr-sdk-cameraworldtoscreenpoint-returns-offset-coordinates

[Corvus]

@AdamStreck Thanks for sharing the update!

[neomimiz]

 

Hi,

I have recently started working with unity and Sranipal.  I have tried to follow the steps described in the recent posts here but still have a few questions related to the conversion between gaze direction screen and world coordinates...

1. In what way gaze direction is related to the screen coordinates in unity? is this the correct conversion to pixels?

  Left_E_gaze_direction_normalized = verboseData.left.gaze_direction_normalized;

x_pix_resolution=1440;  y_pix_resolution=1600;

 left_eye_x_pixels = Left_E_gaze_direction_normalized.x / x_pix_resolution;
  left_eye_z_pixels = Left_E_gaze_direction_normalized.z / y_pix_resolution;

2. Conversion to world coordinates:

Left_eye_position_world_coordinates= Camera.ScreenToWorldPoint( left_eye_x_pixels, left_eye_z_pixels, left_eye_y_pixels);

3. In what way gaze origin is related to these transformations? /or if it isn't why is it provided? 

I would greatly appreciate your help:)

Neomi

 

 

 

 

[scyudits]

I am also interested in the answers to the above post by @neomimiz

[Playnary]

 

On 4/5/2021 at 8:14 AM, scyudits said:

I am also interested in the answers to the above post by @neomimiz

Me too!

[michael_wang]

Hi, @neomimiz @scyudits @Playnary

Validity of the data: 

 

 

If I understand correctly, the following shows how it works, though I am not sure if this is correct.

Enumerator	Binary digit if valid	Decimal digit if valid
SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY	00001	1
SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY	00010	2
SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY	00100	4
SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY	01000	8
SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY	10000	16

Calculation of visual angle:

Are you asking how to convert a GAZE vector into a rotated vector?
we have sample code for this In SDK unity and unreal sample plugin  
AvartarSRanipal_v2.cs line:189

Gaze origin and pupil position: 
These are the values generated by Tobi, I think they should be used to calculate the gaze vector