Eye Tracking in Linux

[Nic111]

Hi,

will there be Linux Support for the Eye Tracking SDK in the future or are there any other Linux solutions?

Thanks a lot!

@Corvus @Daniel_Y @zzy

 

[Corvus]

@Nic111 Currently the Eye Tracking runtime only supports Windows. Generally there is not much support for VR on Linux but we appreciate the feedback. If developers are requesting Linux support we will evaluate it in the future.

[HackPerception]

@Nic111, I'd add that beyond the SRAnipal runtime, Nvidia's VRS shading technology which drives foveated rendering as well as a bunch of other related VR-specific shading technologies are all Windows only. PCVR is 99% Windows based - all of the tools, rendering technologies, and runtimes are Windows ecosystem based.

[The_Fly]

@VibrantNebula this is untrue. Under Linux we have VK_NV_shading_rate_image for Vulkan and GL_NV_shading_rate_image for OpenGL, both of which I have access to under Linux. These extensions are also unnecessary when it comes to changing the distribution of ray tracing samples in the frame.

@Corvus I am requesting Linux support for the Vive Pro Eye's eye tracking features.

Additionally the hand tracking features of the Vive Pro would be very handy under Linux, @Corvus could you raise this or do I need to start another thread?

It would be a huge inconvenience if I have fragmented support for both features over Windows and Linux. With Linux being my preferred development and desktop environment it becomes cumbersome to frequently reboot to test things natively.

Linux has productivity, security and privacy advantages over Windows. The argument that "there are very few VR games for Linux" is completely missing the point and is like saying "there are very few popular VR games so we shouldn't make VR headsets".

Apart from some annoyances with Linux desktop compositors, which can be resolved in many cases, there's very little excuse not to support Linux.

LibUSB, the portable C code for talking to your firmware/chipsets, the deep learning kernels (and models) are all very portable. Vive hand tracking has Android support, so it shouldn't be much of a stretch.

If you've insufficient resources to devote time to portability then open up the source of your SDK's libraries so that the development community can have access to the over-priced hardware it pays for to support this growing VR ecosystem.

Anything that is sufficiently protected by patents and intellectual property law has very little reason to be closed-source.

Edited March 5, 2020 by The_Fly
correction / clarification

[HackPerception]

@The_Fly

Linux support for Pro Eye is unfortunately not roadmapped at this point in time within the scope of the current product and I'm not sure what Tobii's stance on this is within the context of our licensing agreement.

Our current SRAnipal SDK and runtimes are wrapped versions of core Tobii technologies that we're licensing (and that end-users also license as part of the Pro Eye HMD purchase). That unfortunately rules out open sourcing. Barring a change in Tobii's corporate philosophy, alot of that tech stack including the runtime engine is proprietary to Tobii. Vive couldn't open source it if we wanted to - we just license it.

There are actually two SDKs that can access the Pro Eye - you're also able to license the Tobii XR SDK/runtime directly from Tobii. It provides a much deeper level of API access and comes with a suite of analytics tools. I can't find any info on if that SDK/engine supports Linux currently - I'd recommend reaching out and seeing if their first party solution will fit your use case.

I've been using Linux for 15 years - I'm there with you on the benefits and I only use Windows because of VR. The unfortunate truth is that Pro Eye is already a hyper niche product in a niche industry - Linux support is an additional layer of niche meaning the userbase would be impractically small at the moment. I've only seen this question pop up once or twice - we really haven't seen many people ask for this. That said, we appreciate the feedback because it's it's a solid data point I can feedback to the HW product team.

[robo_doc]

I would like to request linux support as well.

As for the linux capabilities of the Tobii XR SDK, I received an answer from Tobii customer support that states that the Tobii XR SDK is build on-top of HTC's SDK and that they cannot offer linux support as HTC is not supporting linux. So far I assumed that HTC is using Tobii software for the eye tracking and therefore the dependency was the other way around. But now I am very confused which software depends on which here and where the  actual bottleneck is.

 

 

[xqms]

Sorry for reviving this old thread, but I've done some research myself and don't want it to go to waste. Because I'm not affiliated with HTC/Vive in any way, take everything I say with a grain of salt 😉

 @robo_doc there is some confusion around this topic, and it stems from this issue: It seems that HTC built in a standard Tobii tracking USB chip, which is directly supported by *all* Tobii SDKs. Even the free ones. Sadly, they stopped offering Linux support, but an older version is still available on GitHub. However, the Tobii chip is powered off or disconnected by default and needs to be switched on - by, you guessed it, HTC software. So the answer to the dependency question is that you need both: HTC would need to release a small utility that turns on the eye tracker and Tobii would need to continue their Linux support (although the old version works fine). That would be a minimal solution that would avoid porting and supporting the entire SRanipal software stack.

It might be possible to write such a "enabler utility" yourself and enjoy eye tracking under Linux.

Edited December 30, 2020 by xqms

[Nic111]

Little late, I kind of forgotten my question (as I now decided to use both Linux and Windows systems )... thank you very much for your answers and suggestions. I might try the approach of xqms.

To the Background why I needed it. Its not especially for doing foveated rendering (but nice to have :) ) we just want to take the eye-data for studies to measure human stress level. So in the scientific field it would be so helpful to have Linux support  as nearly our whole framework is based on Linux software and that counts as far as i know for many other institutes, too. It could be an invest in the future ;)

Thanks again!!!

[bipul mohanto]

Hello Everyone,

 

I am a newbie in this topic. I have just started with Linux (Ubuntu 20.04.4) and found there is no SRanipal Runtime for Linux. I am using `HTC Vive Pro Eye` and previously once I talked with Tobii, and they mentioned their SDK is not working for Vive Pro Eye. 

If you guys give me some solution to access real-time eye tracking data same as I could access in Windows (SRanipal+SteamVR)?

[HackPerception]

@bipul mohanto Recycling my answer from another recent thread

Pro Eye doesn't have a version of the SR Anipal runtime compiled for linux. We're only licensed for Windows.

The SRAnipal Runtime is a licensed wrapped version of a subset of Tobii's Runtime - you can also license Tobbi's SDK/runtime directly and get more features and tools. The Tobii Pro SDK seems to have a Umbuntu linux version - I'm not sure if this would work with Pro Eye and might recommend contacting Tobii to see if either their Pro SDK or their Tobii XR SDK will support Pro Eye on Linux.