The top 5 commercial VR innovations enabled by Tobii Ocumen

My position as the product manager for Tobii Ocumen has given me a front-row seat on the convergence of XR and eye tracking, and I’ve had the opportunity to get close to some of the amazing solutions that result. Since 2020, when we launched
Tobii Ocumen — our advanced VR toolkit — we’ve spent some time refactoring the code to ensure robustness, and we’ve engaged with developers from different specialties and disciplines, helping them to commercialize VR solutions that leverage our eye tracking technology and the advanced biometrics and tools for development and deployment included in the product.

In this post, I will share five of what I think are some of the most innovative commercial solutions that have been developed with Tobii Ocumen — in the hope that you might recognize the problems their innovators solved and the obstacles they overcame. At the bottom of the post, you’ll find some details about the toolkit, how the filters work, and how we’ve developed the data structure to ensure an open architecture. You will also find a link to request a free trial if Tobii Ocumen is helpful to your research, application, or solution. Enjoy!

#1 Concussion assessment — NeuroSync

One of our first customers to use Tobii Ocumen was NeuroSync (formerly SyncThink). As Scott Anderson (chief clinical officer) points out in a 2021 interview 
finding the keys to scalability, one of the main advantages of VR is mobility. NeuroSync has developed a solution — EYE-SYNC® — that helps clinicians assess traumatic brain injury (TBI). VR not only brings the solution directly to the sidelines of the football field or hockey rink, its closed environment makes it ideal for helping people ignore external influences, which can be intense at live sports events. Crucially, Anderson mentions the accurate and systematic measurement of eye movements and how they used the high-level filters included in Tobii Ocumen to ease the work of transforming raw eye tracking data into reliable biomarkers.

#2 Cognitive performance — REACT Neuro and Aperion Life

As the global population ages, the significance of cognitive performance is gaining attention as we attempt to ensure quality of life and enable therapies for neurodegenerative diseases such as Alzheimer’s, ALS, and Parkinson’s. REACT Neuro is a Cambridge-based platform developer who has been working on shifting traditional cognitive tests into VR. In a trial to determine the effects of a healthy lifestyle on cognitive performance, Apeiron Life (a Silicon Valley startup) used REACT Neuro’s VR simulation of the classic paper-based trail test.

The application leverages advanced biometrics from Tobii Ocumen to reveal saccades and fixations that cannot be observed manually. Our data robustness ensures that performance can be tracked over time.

Since the trial’s conclusion both companies have been enthusiastic about the potential for deep metrics generated by advanced sensor technologies like eye tracking, motion tracking, spirometers, EEG, and thermographic sensors and how the macro trend of wellbeing will ensure that some of these advanced technologies will trickle down to the mainstream.

You can read the full story in this blog post: Cognitive performance through decades of life.

#3 VR-based training — Cineon Training

Figures vary, but some sources forecast a global deficit of up to 80,000 trained pilots in the coming decade. The shortage is due to a multitude of factors, layoffs and early retirement during covid, fewer military pilots (causing a drop in availability for civil aviation), post-covid recovery of the travel industry, and fully booked simulation facilities, which are expensive to build and hourly rates are soaring.

Cineon Training, a UK-based company, has developed a VR-based application (TACET) to build communication competency and accelerate skills acquisition. It uses eye tracking and advanced signals from Tobii Ocumen to improve the efficiency of time spent in the simulator.

TACET uses Tobii Ocumen on a Pico Neo 3 Pro Eye — untethered VR headset equipped with Tobii’s eye tracking — connected to a Microsoft Azure cloud solution for uploading and reviewing recordings. The image shows how an expert carries out a task (the green lines) compared with a novice (white lines).

One of the main takeaways is the impact eye tracking has on the robustness of skills learned. And we’ve heard this from other partners as well. Eye tracking provides a window into the mind. It allows us to assess emotions and feelings and speeds up learning. When acquiring a new skill, being able to see how your eyes move around (what you focus on) and compare with how an expert does the job helps solve the cognitive problem. It also allows people who are already good at something to refine their solutions.

#4 Digital transformation in MedTech — Olleyes

As a digital tool that delivers insights about people, eye tracking helps to transform traditional care based on manual observation into the digital sphere. As I mentioned earlier, in NeuroSync’s concussion assessment, VR brings the bonus of mobility, often leading to highly flexible solutions.

Olleyes have used Tobii Ocumen on a Pico Neo 3 Pro Eye VR headset to deliver a comprehensive assessment of the visual field, analyzing retinal sensitivity, for example, for patients with glaucoma. They have developed a suite of eye tests that a person can take rapidly (minutes instead of hours) either lying down or sitting up — increasing the solution’s flexibility and how many people a clinic can assess daily. Like many others in MedTech, Olleyes seeks to make solutions more convenient by enabling the shift-to-the-home.

If you want to know more about our collaboration with Olleyes, I recommend this VRARA Healthcare forum talk with my colleague Amanda Bentley and Olleyes CEO Alberto Gonzalez Build more accessible and effective vision care.

#5 Pilot training – VTR

To be able to fly a new aircraft type, say a Boeing 737 or an Airbus A320, pilots need a type rating certificate (a license). To learn the position and functionality of all the instruments, they typically use what’s known as “paper tigers” — mockups of the cockpit. Tigers can be advanced touchscreen setups, but they tend to be costly and fail to provide a realistic sense of the 3D space. This situation isn’t ideal for pilots who typically have about 2-3 weeks to learn a new aircraft type and what every light, button, switch, and gauge in the cockpit means. The manual for a Boeing 737, for example, is about 2,000 pages.

To overcome the poor ecological validity of paper tigers, VTR has developed a VR simulation, accurately replicating the cockpit environment to ensure robust skills acquisition. Crucially, the company collaborated with JetBlue — one of the largest airlines in the US — to gain use case data to support ROI.

For a deep dive into VTR, I recommend this AWE talk with my colleague Johan Bouvin and Rick Parker, one of VTR’s co-founders Build disruptive pilot training with XR and eye tracking.

About Tobii Ocumen

These five highly innovative use cases are commercially viable because they leverage the closed nature of VR, its mobility, and the realism offered by the medium to deliver a cost-efficient solution. And all have been developed using Tobii Ocumen.

So, what does Tobii Ocumen bring to the table that enables the development of such applications?

Advanced signals and filters

Tobii Ocumen offers advanced eye tracking signals, including:

• Gaze vectors (left and right eye separated).
• Pupil diameter (left and right eye separated).
• High-precision system and device timestamps.
• Metadata, such as eye tracking hardware, firmware, and runtime software versions.

The filter library, which our customers seem to really appreciate, provides research-based, pre-validated filter components for detecting eye movements such as fixations, saccades, and smooth pursuit. These filters transform the data captured by our eye tracking hardware into building blocks of standard biomarkers.

Calibration and positioning

To ensure Tobii Ocumen delivers quality data and images display correctly, VR headsets need to sit correctly on the user’s head, and our eye tracking needs to be calibrated for the individual.

To ease the process of calibration and positioning, we’ve developed some cool features that you can embed directly into your application so that users don’t have to follow the awkward steps of quitting your application, starting a separate calibration/positioning process, and resuming.

Recording and replay

To facilitate development, Tobii Ocumen combines context data, such as 3D object movements and user-view video, with sensor data from advanced eye tracking and controller events into a single format.

We’ve included some ready-made recorders and templates for common research study paradigms that you can quickly integrate into your Unity project. And to help you replay, we’ve developed Tobii Ocumen Studio (see video below), which presents recorded data (context and sensor) in an easy-to-understand visual format.

Data integrity

In the recent development of Tobii Ocumen, we’ve put some effort into the data structure, optimizing for speed and file size and creating a flexible product architecture with an open I/O interface. The product supports data conversion, and you can easily import data gathered by any device or machine, regardless of the programming language you prefer.

I think our deep relationship with the scientific research community is worth mentioning. Because of it, we are keenly aware of the need for scientific-grade validation data in commercial solutions that require regulatory certification. Our validations include population coverage, accuracy, precision, and trackability performance.

Useful links

• For further inspiration and technical details, visit our
  developer zone for Tobii Ocumen.
• For more information about the product, visit the Tobii Ocumen product page.