Jump to content

cte

Verified Members
  • Posts

    21
  • Joined

  • Last visited

Reputation

5 Neutral

3 Followers

Recent Profile Visitors

578 profile views
  1. Thanks for getting back! I was interested in getting a more direct link to the eye tracker in the vive than is currently offered in the UE4 plugin. The idea is to build non-engine specific data pipelines so I can extract dilation and gaze (using ROS in my case).
  2. Hi, Recently I've been needing to find the local address (think IP or similar) for the eye tracker in the HMD to extract or write data gathered there direct to an application that relies on R for data processing. Is there a host address for the eye tracker presently that I can call in the SRanipal SDK?
  3. UPDATE: πŸŽ‰πŸŽ‰Found the fix and now have functional dilation extraction going. πŸŽ‰πŸŽ‰ ----------------------------------------------------------- SRanipalEye_FunctionLibrary.cpp bool USRanipalEye_FunctionLibrary::GetPupilDiameter(EyeIndex eye, float& diameter_mm) { return SRanipalEye_Core::Instance()->GetPupilDiameter(eye, diameter_mm); } SRanipalEye_Core.cpp bool SRanipalEye_Core::GetPupilDiameter(EyeIndex eye, float &diameter_mm) { bool valid = false; if (SRanipalEye_Framework::Instance()->GetStatus() == SRanipalEye_Framework::FrameworkStatus::NOT_SUPPORT) { valid = true; diameter_mm = 0.0f; } else { UpdateData(); Eye::SingleEyeData eyeData; switch (SRanipalEye_Framework::Instance()->GetEyeVersion()) { case SupportedEyeVersion::version1: eyeData = eye == EyeIndex::LEFT ? EyeData_.verbose_data.left : EyeData_.verbose_data.right; break; case SupportedEyeVersion::version2: eyeData = eye == EyeIndex::LEFT ? EyeData_v2.verbose_data.left : EyeData_v2.verbose_data.right; break; } valid = eyeData.GetValidity(Eye::SingleEyeDataValidity::SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY); if (valid) { diameter_mm = eyeData.pupil_diameter_mm; } else diameter_mm = 0.0f; } return valid; } SRanipal_Core.h bool GetPupilDiameter(EyeIndex eye, float& diameter_mm); SRanipalEye_FunctionLibrary.h UFUNCTION(BlueprintCallable, Category = "SRanipal|Eye") static bool GetPupilDiameter(EyeIndex eye, float& diameter_mm);
  4. @Franka & @MariosBikos_HTC thanks for the help. For reference, I'm working off of the latest version of the SRanipal Framework so had to alter some of the code. Now I'm having a little trouble understanding what I need to feed into the blueprints I'm calling and assume I have put something wrong in here as I want diameter MM as a string output rather than an input so I can write it to a csv. Here is the function in blueprints: Question: What do I need to alter to get the diameter Mm as a string output variable? > I'm going to look into this but would love a response here if someone knows how this would. - I've looked into this a little bit and it seems that the absence of the framework is what is really creating issues here. Question: Based on the updates to SRanipal SDK what do I call instead of the framework for getting the data? ---------------------------------------------------------------------------------------------------------------------------------- πŸ‘‡πŸ‘‡πŸ‘‡ Changed Code πŸ‘‡πŸ‘‡πŸ‘‡πŸ‘‡ ➑️ SRanipalEye_Core.cpp: Added: bool SRanipalEye_Core::GetPupilDiameter(EyeIndex eye, float& diameter_mm) { bool valid = false; if (SRanipalEye_Framework::Instance()->GetStatus() == SRanipalEye_Framework::FrameworkStatus::NOT_SUPPORT) { valid = true; diameter_mm = 0.0f; } else { UpdateData(); Eye::SingleEyeData EyeData = eye == EyeIndex::LEFT ? EyeData_.verbose_data.left : EyeData_.verbose_data.right; valid = EyeData.GetValidity(Eye::SingleEyeDataValidity::SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY); if (valid) { diameter_mm = EyeData.pupil_diameter_mm; } else { diameter_mm = 0.0f; } } return valid; } ➑️ SRanipalEye_FunctionLibrary.h Added: UFUNCTION(BlueprintCallable, Category = "SRanipal|Eye") static bool GetPupilDiameter(EyeIndex eye, UPARAM(ref) float& diameter_mm); ➑️ SRanipalEye_FunctionLibrary.cpp Added: bool USRanipalEye_FunctionLibrary::GetPupilDiameter(EyeIndex eye, UPARAM(ref) float& diameter_mm) { return SRanipalEye_Core::Instance()->GetPupilDiameter(eye, diameter_mm); }
  5. @Franka & @MariosBikos_HTC thanks for the help. For reference, I'm working off of the latest version of the SRanipal Framework so had to alter some of the code. Now I'm having a little trouble understanding what I need to feed into the blueprints I'm calling and assume I have put something wrong in here as I want diameter MM as a string output rather than an input so I can write it to a csv. Here is the function in blueprints: Question: What do I need to alter to get the diameter Mm as a string output variable? > I'm going to look into this but would love a response here if someone knows how this would. ---------------------------------------------------------------------------------------------------------------------------------- πŸ‘‡πŸ‘‡πŸ‘‡ Changed Code πŸ‘‡πŸ‘‡πŸ‘‡πŸ‘‡ ➑️ SRanipalEye_Core.cpp: Added: bool SRanipalEye_Core::GetPupilDiameter(EyeIndex eye, float& diameter_mm) { bool valid = false; if (SRanipalEye_Framework::Instance()->GetStatus() == SRanipalEye_Framework::FrameworkStatus::NOT_SUPPORT) { valid = true; diameter_mm = 0.0f; } else { UpdateData(); Eye::SingleEyeData EyeData = eye == EyeIndex::LEFT ? EyeData_.verbose_data.left : EyeData_.verbose_data.right; valid = EyeData.GetValidity(Eye::SingleEyeDataValidity::SINGLE_EYE_DATA_PUPIL_DIAMETER_VALIDITY); if (valid) { diameter_mm = EyeData.pupil_diameter_mm; } else { diameter_mm = 0.0f; } } return valid; } ➑️ SRanipalEye_FunctionLibrary.h Added: UFUNCTION(BlueprintCallable, Category = "SRanipal|Eye") static bool GetPupilDiameter(EyeIndex eye, UPARAM(ref) float& diameter_mm); ➑️ SRanipalEye_FunctionLibrary.cpp Added: bool USRanipalEye_FunctionLibrary::GetPupilDiameter(EyeIndex eye, UPARAM(ref) float& diameter_mm) { return SRanipalEye_Core::Instance()->GetPupilDiameter(eye, diameter_mm); }
  6. UPDATED Blueprint :: Found the Join String Array which Makes a significantly easier to work with CSV file. New Blueprint Structure: Leads to new output in csv file:
  7. I was able to get this to work now in UE4 and what it took was learning a bit about actors and blueprints. To get gaze you can create a new actor and place it in the scene, then create a blueprint on that actor. For getting gaze data I produced the following blueprint: A key feature of this blueprint was the File IO Save String Arrray which I got from a function library plugin called "Victory Plugin" which you can get here: https://www.mediafire.com/file/pgrpsf9my7tv1ae/VictoryPlugin25.zip/file Though this doesn't solve my pupil dilation problem, I can probably get that data via a functionb posted in a different forum: I'll be trying that code next and creating a new actor and blueprint for that particular function. The data I'm getting now looks like this: From the CSV file. I'd like to get this in a different format where things are columns rather than rows but I can probably work wiht this data and manipulate it to that effect in r or something I know better than C++ functions. Anyway, that is the update on the extracting data from SRanipal for eye direction, fixation, and confidence value with a unix timestamp. Would love to get a few thoughts on how to improve this blueprinting setup. -CTE
  8. :: UPDATE :: 10-07-20 :: Application: Academic Research Goals: Install SDK ----------------------------- [ X ] Get Eye Gaze ------------------------- [ X ] Get Fixation --------------------------- [ ] Get Pupil Dilation ------------------- [ ] Run Subjects & Get Tenure ----- [ ] Question: How do I reference the SDK's framework / API to extract close to real-time eye tracking data that prints either in a data.frame or CSV file. --------------------------------------------------------------------------------------------------- I've given up (more or less) on getting this to work in UE4 so tried out the sample code for _C in the SDK and had a bit more success getting data. I can now get gaze data from the sample project in some fashion but have to figure out now how to get dilation data which might mean I do a similar kind of thing to what @MariosBikos_HTC did here: I'm not sure how to wrap my head around this function library code but am assuming this is a type of blueprint class reference in unreal that I can recycle for calculating the same information and providing it to me in a string format in C. Not sure about this though since I'm not much a C programming language user but I will try to create a new header and call the function in the .cpp file such that it will show up as a keyed option "7" to start the string and "8" to stop it. Doing so, I think I can just mirror the existing gaze data keys but again, I'm not really sure I know exactly what I'm doing so appreciate any guidance in this process.
  9. Trying to work trough it now. I see the different levels in Unreal but don't know where to start with the data extraction. I examined the dartboard actor but am unsure how to use what I'm looking at here (DartBoard.h) --------------------------------- Do I need to modify this code so as to utilize it as a kind of template? - If so, where do I modify and what structure do I use? // ========= Copyright 2019, HTC Corporation. All rights reserved. =========== #pragma once #include "SRanipal_FunctionLibrary_Eye.h" #include "SRanipal_Eye.h" #include "Engine/Classes/Camera/PlayerCameraManager.h" #include "Runtime/Engine/Classes/Materials/Material.h" #include "CoreMinimal.h" #include "GameFramework/Actor.h" #include "DartBoard.generated.h" UCLASS() class SRANIPAL_API ADartBoard : public AActor { GENERATED_BODY() public: // Sets default values for this actor's properties ADartBoard(); UPROPERTY(EditAnywhere) FVector Position; // The dartboard's position UPROPERTY(EditAnywhere) UStaticMeshComponent* BoardMesh; UPROPERTY(EditAnywhere) UMaterial* ParentMaterial; UPROPERTY() APlayerCameraManager* PlayerCameraRef; private: FVector FocusPosition; UMaterialInstanceDynamic* DartBoardMaterial; protected: // Called when the game starts or when spawned virtual void BeginPlay() override; public: // Called every frames virtual void Tick(float DeltaTime) override; private: // Focus relative parameter FFocusInfo FocusInfo; FVector GazeOrigin, GazeDirection; float RayLength = 1000.0f; float RayRadius = 1.0f; // Move dart board FVector InitialPosition; bool FirstUpdate = true; bool AlreadyMoveDart = false; void MoveDartBoard(); float SignedAngle(FVector v1, FVector v2, FVector v_forward); }; Sorry if this is a basic question but I'm brand new to C++ πŸ‘Ά
  10. @Corvus would you perhaps be willing to help out with this low-level explanation of how to get at the SDK's data? I'm new to C++ and compiler languages in general so don't know how to call the API (if that is even how this works).
  11. @nbhatia how are you going about getting the data from the SDK? I'm new to this and trying to figure out how to pull that data into a writable export file of some kind (ideally a CSV).
  12. Application: Academic Research Goals: Install SDK ----------------------------- [ X ] Get Eye Gaze ------------------------- [ ] Get Fixation --------------------------- [ ] Get Pupil Dilation ------------------- [ ] Run Subjects & Get Tenure ----- [ ] Question: How do I reference the SDK's framework / API to extract close to real-time eye tracking data that prints either in a data.frame or CSV file. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Hi there, I've been able to get the VR and eye tracking set up and working per a previous thread with UE 4.25.3 and have a demo with VR functional. The only thing is I'm not sure where to go from here to get the SDK to write/print data to a csv or similar file. Presently I don't need any eye tracking interactions within the VR environment so the dartboards and mannequin head are useful in that they show this data exists but I need to get that data out of whatever loop it is in and write it to a data file for processing in statistical programs and the like. @MariosBikos_HTC , you've been a great help so far. Let me know if you or another HTC fellow are the right ones to ask about this function. Once I get something functional I'll definitely be sharing it for future folks in my position.
  13. @Franka I'm trying to get some pupil dilation data as well. I wanted to run this fix but wasn't sure what you meant by header file. I'm also thinking of extracting this data and writing to a file of some kind for processing in R or Python. Sounds like the detection isn't working great though so I'm going to try my hand at figuring it out and will let you know if I can get something. -CTE
  14. Working off of the new errors I reported above, I was able to get everything working and the dartboards and eye tracking functional. For the first chunk of similar errors: I applied the earlier fix provided in this thread where you: Replace RelativeLocation With GetRelativeLocation() Where they appeared in the code. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Upon fixing this, I was faced with "Fatal Error C1083" This was linked to the XXX_XXX_Lip.h series of files found both in "Source/SRanipal/Public/Lip" and "Source/SRanipal/Private/Lip" folders. This error was caused by the compiler not being able to locate tehse files. To fix this I simply added the exact directory location to the points where the errors were flagging. So, for example if I saw: #Include "SRanipal_AvatarLipSample.h" I would replace this with #include "SRanipal/Public/Lip/SRanipal_AvatarLipSample.h" This fix took a few iterations of building and fixign the error until I built and there were no errors. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Here is the working source folder that may contain all the fixes needed to make things work. Source.zip This can replace the existing source folder in: \Plugins After all that I was able to ge the dartboards and eye balls in the head working well enough. ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Next I'm going to look at extracting pupil dialation which I think is maybe convered in:
  15. Applied fix detailed by @MariosBikos_HTC above and additional errors were triggered. ======================================================= Fix Applied to: ~\Unreal Projects\MyProject\Plugins\SRanipal\Source\SRanipal\SRanipal.Build.cs Modified SRanipal.Build.cs Attached. SRanipal.Build.cs --------------------------------------------------------------------------------------------------- New Errors
Γ—
Γ—
  • Create New...