AdamStreck Posted February 10, 2021 Author Posted February 10, 2021 Update - Unity has reproduced the issue on their end and flagged a bug: https://issuetracker.unity3d.com/issues/xr-sdk-cameraworldtoscreenpoint-returns-offset-coordinates 1
neomimiz Posted March 29, 2021 Posted March 29, 2021 Hi, I have recently started working with unity and Sranipal. I have tried to follow the steps described in the recent posts here but still have a few questions related to the conversion between gaze direction screen and world coordinates... 1. In what way gaze direction is related to the screen coordinates in unity? is this the correct conversion to pixels? Left_E_gaze_direction_normalized = verboseData.left.gaze_direction_normalized; x_pix_resolution=1440; y_pix_resolution=1600; left_eye_x_pixels = Left_E_gaze_direction_normalized.x / x_pix_resolution; left_eye_z_pixels = Left_E_gaze_direction_normalized.z / y_pix_resolution; 2. Conversion to world coordinates: Left_eye_position_world_coordinates= Camera.ScreenToWorldPoint( left_eye_x_pixels, left_eye_z_pixels, left_eye_y_pixels); 3. In what way gaze origin is related to these transformations? /or if it isn't why is it provided? I would greatly appreciate your help:) Neomi 2
scyudits Posted April 5, 2021 Posted April 5, 2021 I am also interested in the answers to the above post by @neomimiz
Playnary Posted April 28, 2021 Posted April 28, 2021 On 4/5/2021 at 8:14 AM, scyudits said: I am also interested in the answers to the above post by @neomimiz Me too!
michael_wang Posted May 6, 2021 Posted May 6, 2021 Hi, @neomimiz @scyudits @Playnary Validity of the data: If I understand correctly, the following shows how it works, though I am not sure if this is correct. Enumerator Binary digit if valid Decimal digit if valid SINGLE_EYE_DATA_GAZE_ORIGIN_VALIDITY 00001 1 SINGLE_EYE_DATA_GAZE_DIRECTION_VALIDITY 00010 2 SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY 00100 4 SINGLE_EYE_DATA_EYE_OPENNESS_VALIDITY 01000 8 SINGLE_EYE_DATA_PUPIL_POSITION_IN_SENSOR_AREA_VALIDITY 10000 16 Calculation of visual angle: Are you asking how to convert a GAZE vector into a rotated vector? we have sample code for this In SDK unity and unreal sample plugin AvartarSRanipal_v2.cs line:189 Gaze origin and pupil position: These are the values generated by Tobi, I think they should be used to calculate the gaze vector
Drifter Posted March 13 Posted March 13 On 11/6/2020 at 1:03 AM, AdamStreck said: I did not figure it out. Would you please post details @Av ? I know where the issue lies, though, it's a difference between the view provided by "Left Eye" vs the full image rendered to the headset. I have made a simple example, where the camera coordinates of an object are printed directly on screen. The project is attached (EyeTrackingIssue.zip), but the full code is the following: using UnityEngine; using UnityEngine.UI; using UnityEngine.XR; public class DisplaySSCoordinates : MonoBehaviour { public Transform debugSphere; public Text displayText; public Camera vrCamera; private void Update() { var coords = vrCamera.WorldToScreenPoint(debugSphere.position, Camera.MonoOrStereoscopicEye.Left); var eyeTextureSize = new Vector2(XRSettings.eyeTextureWidth, XRSettings.eyeTextureHeight); var units = new Vector2(coords.x / eyeTextureSize.x, coords.y / eyeTextureSize.y); displayText.text = $"({units.x:F2},{units.y:F2})"; } } You can see that in the bottom-left and top-right corners, the coordinates are not what I would expect. For the center it is correct: However when I switch from the "LeftEye" to "OcclusionMesh", the coordinates are correct: This is obviously not SRanipal issue only, probably will also happen for other EyeTracking systems, however I think this is the prescribed way of getting the screen-space position (it's a while I'm not sure, I'm happy for a different workflow). I noticed that the error changes with the aspect ratio, which makes sense as the screen-crop of the "LeftEye" changes too. I'd need to obtain the information how the "EyeTexture" is cropped into the game view to calculate the position correctly. Any recommendations? EyeTrackingIssue.zip 23.88 kB · 26 downloads Hi, I had a similar problem with my recent development. Where is the "OcclusionMesh" you mentioned turned on or off? I hope to be able to get at least the exact screen coordinates of the eye in this way. Looking forward to your reply.
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now