How do eye trackers work?

Tobii has been leading the eye tracking industry for the past 20 years. With applications in fields ranging from scientific and consumer research to XR and automotive, Tobii eye trackers have advanced the world by providing unique insights into human attention, intent, and behavior. This article will provide an in-depth walk-through of how eye tracking works. We illustrate step-by-step processes of screen-based and wearable eye trackers.

What is eye tracking?

Eye tracking is a sensor technology that measures and records the position and movement of the eyes. An eye tracker is a device for assessing where or what one is looking at, also known as the point of gaze.

The point of gaze can be identified across various types of stimuli. Typically, an individual whose eyes are tracked directs their attention to a stimulus that may appear on a computer screen, in a real-world environment, or in a virtual reality. Eye trackers can be used as standalone devices or be integrated into other technology, such as XR headsets, PCs, and vehicles (as part of automotive solutions).

Eye tracking technology has a broad range of applications, including scientific and medical research, accessibility for people with disabilities, improving road safety in driving, and enhancing virtual reality and gaming experiences.

Key hardware components of an eye tracker

Video-based eye trackers, such as Tobii, typically consist of these key hardware components (Figure 1):

• Near-infrared light illumination modules
• Camera sensors
• Processor (image detection, 3D eye model, gaze mapping algorithm)

How does eye tracking work?

1. Illumination of the eyes

Tobii eye trackers use near-infrared spectrum light to illuminate the eyes and create light reflection patterns onto each eye. Specifically, the light reflection falls onto the pupil (the circular black opening in the center of the eye) and the cornea (the transparent outer layer at the front part of the eye).

The reflection of light on the cornea (also known as glint) is tracked relative to the position of the pupil center, which allows us to estimate the point of gaze. This detection method is known as pupil-center corneal reflection (PCCR).

2. Reflection detection by sensors

The eye trackers contain camera sensors sensitive to near-infrared light and capture images of the eyes and the reflections. The camera sensors are positioned in front of the user and have a clear view of the user’s eyes. Depending on the eye tracker speed, you can acquire information about specific types of eye movements. Tobii's fastest eye tracker (Tobii Pro Spectrum) takes an image of the eyes every 0.833 ms (1200Hz) and allows the tracking of microsaccades - the miniature eye movements of the amplitude less than 0.1 degree of visual angle.

3. Image processing and analysis

During the acquisition, the image of an eye is grabbed from the camera and sent for analysis. Advanced image-processing algorithms are used to estimate a 3D model of the individual’s eye and the position of the eye in space. With Tobii eye trackers, the gaze point can be calculated accurately without head restraint or chinrest. The pupil center and corneal reflection are detected during the image analysis to calculate the gaze point.

Zoom in: Pupil-center corneal reflection (PCCR) method

To estimate the point of gaze, Tobii eye trackers use a method called pupil-center corneal reflection (PCCR). The method is based on identifying and locating the pupil center and the light reflection on the cornea (known as glint). The illumination of a near-infrared light source creates a light reflection on the cornea and the pupil (Figure 2).

The image processing algorithms identify the center of the pupil and the light reflection on the cornea (Figure 3). As we move our eyes, the position of the iris and the pupil changes with respect to the corneal reflection. The eye tracking algorithms use the vector between the pupil center and the corneal reflection and make further calculations to infer the location of the point of gaze (i.e., what we look at) with great accuracy.

Zoom in: What is near-infrared light, and why use it for eye tracking?

The accuracy of eye tracking relies on a precise annotation of the pupil center and the corneal reflection. There is a need for a stable source of illumination that can withstand environmental variation and allow accurate annotation at any time. Near-infrared light is an excellent source of light that fulfills all the requirements for accurate and high-precision eye tracking:

• Invisible to the human eye

Near-infrared light is invisible to the human eye – it does not cause discomfort or distract the person in front of the eye tracker.

• Stable illumination in different conditions

Near-infrared light interferes less with ambient light compared to the visible spectrum. This assures reliable and accurate illumination of the cornea, making eye tracking results stable in different environments.

Types of eye tracking devices and how they work

Video-based eye trackers are the most used type of modern eye tracking devices and can be further divided into the following types:

• Screen-based eye trackers, also known as remote eye trackers
• Mobile eye trackers or simply, eye tracking glasses
• Integrated or embedded systems or augmented reality devices (XR, for instance)

Tobii screen-based eye trackers

As the name suggests, the screen-based eye tracker is mounted to the desktop or laptop screen (Figure 4). Screen-based eye trackers calculate the eye’s position and gaze point on the computer monitor. Screen-based eye trackers are extensively used in scientific research, as well as for enhanced gaming experience or consumer research.

Tobii wearable eye trackers

The wearable eye tracker or simply, eye tracking glasses, are worn as regular glasses (Figure 5). The wearable eye tracker has a front-scene camera that records what the user is looking at, thus allowing for first-person insights in real-world environments. The wearable eye tracker is commonly used in scientific research, user and consumer research, and training and assessment.

Eye tracking embedded in XR headset

Extended reality (XR) is an umbrella term that describes immersive technologies, the main types being virtual reality (VR), augmented reality (AR), and mixed reality (MR).

In an XR headset, eye tracking components typically include cameras and light sources placed in a ring-like structure between the user and the display (Figure 6). The core machine-learning algorithms interpret the camera feed to generate real-time data points such as pupil size, gaze vector, and eye openness. This information can be used to understand the user’s attention and cognitive state and anticipate their intentions.

Eye tracking can be integrated into glasses by either fully embedding all eye tracking components in the lenses, seamlessly integrating all components into the glasses frame, or a combination of lens and frame embedding (Figure 7).  All using Tobii technology and eye tracking reference designs.

Eye tracking in XR applications spans domains such as healthcare, training, simulation, UX insights, MedTech, EdTech, and scientific research.