Pupil dilation in mm vs. eye openess parameter

[kbombeke]

Hi,

I am trying to write a blink/missing data interpolation algorithm for the recorded pupil dilation in mm. To this extent, I was thinking of using the EyeOpeness parameter to identify when the pupil was not visible by the HMD and interpolate those specific values.

However, when I plot the pupil size in mm (BLUE line) and the eye openess parameter (ORANGE line) on top of each other, there's a big chunck of missing data in the middle for which the pupil diameter is -1 (which means missing, I guess) but the eye openess parameter shows 1 (which means the eye is completely opened). This seems weird to me and makes the Eye Openess parameter useless, or am I missing something here?

Thank you!

 

[imarin18]

Dear @kbombeke

 

I observed the similar situation previously. I recommend using the information of eye validity first.

 

 

 

If I understand correctly, the following shows how it works, though I am not sure if this is correct.

Enumerator	Binary digit if valid	Decimal digit if valid
SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY	00001	1
SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY	00010	2
SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY	00100	4
SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY	01000	8
SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY	10000	16

 

 

I hope this helps.

 

Best regards,

imarin18

 

[kbombeke]

Dear @imarin18,

Thank you for your fast reply. I will definitely look into this "eye validity" variable. We know for sure that the participant closed his eyes during this interval of missing data, so it's still weird that a parameter called "eye openess" returns the maximum value of 1 (meaning "eyes are open") for that entire interval.

I'll let you know when we find a solution.

Best regards,
Klaas

 

[Daniel_Y]

Have you tested SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY before retrieve openness value?

 

@kbombeke

[alwyuyang]

       public struct SingleEyeData
            {
                /** The bits containing all validity for this frame.*/
                public System.UInt64 eye_data_validata_bit_mask;
                /** The point in the eye from which the gaze ray originates in meter miles.(right-handed coordinate system)*/
                public Vector3 gaze_origin_mm;
                /** The normalized gaze direction of the eye in [0,1].(right-handed coordinate system)*/
                public Vector3 gaze_direction_normalized;
                /** The diameter of the pupil in meter miles*/
                public float pupil_diameter_mm;
                /** A value representing how open the eye is.*/
                public float eye_openness;
                /** The normalized position of a pupil in [0,1]*/
                public Vector2 pupil_position_in_sensor_area;

                public bool GetValidity(SingleEyeDataValidity validity)
                {
                    return (eye_data_validata_bit_mask & (ulong)(1 << (int)validity)) > 0;
                }
            }

Here is the source code of the SingleEyeData, but I don't know how to run GetValidity(SingleEyeDataVadity validity). What is the input variable validity?  I can easily access to the property of SingleEyeData through: eye.data.gaze_origin_mm, but when it comes to GetValidity, I don't know which is the input. Could you please do me a favor?

@imarin18 @Daniel_Y@kbombeke

Thank you in advance. 

Edited February 24, 2020 by alwyuyang

[imarin18]

Dear @alwyuyang

I suppose you can get the validity data from "eye_data_validata_bit_mask" in your code. The type of the output is UInt64, not bool.

 

Best regards,

imarin18 

[alwyuyang]

21 hours ago, imarin18 said:

Dear @alwyuyang

I suppose you can get the validity data from "eye_data_validata_bit_mask" in your code. The type of the output is UInt64, not bool.

 

Best regards,

imarin18 

 

Dear @imarin18,

I am checking the timestamp and frame_sequence, but I found the following the results in the log file. 

Column name: timestamp, frame_sequence, eye_data_validata_bit_mask

row 1: 62613 4848 31

row 2: 4205382 4964 31

row 3: 4205415 4968 31 

row 3: 4205457 4973 31

@Daniel_Y

I don't understand why there is a huge skip from row 1  to row 2 regarding the timestamp. The row 1 is first output I obtained from the system. 

In addition, what does "31" means for the eye_data_validata_bit_mask? I only know it is uint64 data type, but how to interpret it?

 

Thank you.

 

 

 

 

 

 

 

[imarin18]

Dear @alwyuyang

As explained in the following post, the timestamp cannot be correctly measured with the current SDK version according to the technical support team.

 

If I understand correctly, I reckon that the value of validity follows the table below. If you sum all the decimal digits (1+2+4+8+16), you get 31. Though again I am not sure if my understanding is correct.

Enumerator	Binary digit if valid	Decimal digit if valid
SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY	00001	1
SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY	00010	2
SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY	00100	4
SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY	01000	8
SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY	10000	16

 

Best regards,

imarin18

[alwyuyang]

3 minutes ago, imarin18 said:

Dear @alwyuyang

As explained in the following post, the timestamp cannot be correctly measured with the current SDK version according to the technical support team.

 

If I understand correctly, I reckon that the value of validity follows the table below. If you sum all the decimal digits (1+2+4+8+16), you get 31. Though again I am not sure if my understanding is correct.

Enumerator	Binary digit if valid	Decimal digit if valid
SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY	00001	1
SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY	00010	2
SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY	00100	4
SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY	01000	8
SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY	10000	16

 

Best regards,

imarin18

Dear @imarin18,

Thank you for sharing your opinions. However, 

1, although the timestamp is not measured precisely, how could it jump from 62613 to 4205382  in two subsequent collection?

2, i agree with your explanation of "31". However,  I also got uint64  "8"  for the eye_data_validata_bit_mask when it failed to measure, then it seems hard to explain by the summing theory. 

 

                writeLeftEyeData.WriteLine("{0} {1} {2} {3} {4} {5} {6} {7} {8} {9} {10} {11} {12} {13} {14} {15} {16} {17} {18}",
                                          Sample_MainThread.UnityTime,
                                          unixTime,
                                          data.no_user,
                                          data.timestamp,
                                          data.frame_sequence,

                                          data.verbose_data.left.eye_data_validata_bit_mask,
                                          
                                          data.verbose_data.left.gaze_origin_mm.x,
                                          data.verbose_data.left.gaze_origin_mm.y,
                                          data.verbose_data.left.gaze_origin_mm.z,

                                          data.verbose_data.left.gaze_direction_normalized.x,
                                          data.verbose_data.left.gaze_direction_normalized.y,
                                          data.verbose_data.left.gaze_direction_normalized.z,

                                          data.verbose_data.left.pupil_diameter_mm,
                                          data.verbose_data.left.eye_openness,

                                          data.verbose_data.left.pupil_position_in_sensor_area.x,
                                          data.verbose_data.left.pupil_position_in_sensor_area.y,

                                          data.expression_data.left.eye_frown,
                                          data.expression_data.left.eye_squeeze,
                                          data.expression_data.left.eye_wide                                          
                                          );

 

I attach my code and the corresponding output file so you can see that I got 31 when it worked and 8 when it failed. Anything to comment? Thank you:)

@Daniel_Obrien @Daniel_Y

Best regards,

 

 

 

 

leftEyeDataSub.txt

[imarin18]

Dear @alwyuyang

I suppose that the technical support team meant that the huge gap from 62613 to 4205382 in timestamp data would be also a part of improvement.

I'm not sure exactly about what you meant by "when it failed to measure", but when you get 8 in your validity data, I reckon that you have the valid data only for eye openness (SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY) having the value of 8 and the other four parameters have the value of 0, meaning that the four are not valid.

Best regards,

imarin18