Seeing through words: Navigating museum narratives with eye tracking

Zsofia Pilz is an account manager at Tobii and an external Ph.D. candidate at Leiden University. She conducts her work at Francesco Walker’s HATLab, which specializes in human interaction and technology. In this study, she partnered with the Rijksmuseum in Amsterdam, a world-renowned art museum, using Tobii Pro Glasses 3 to explore how labels guide visitor’s attention.

Art museums attract millions of visitors annually from diverse backgrounds, each bringing varying levels of art familiarity and informational needs. While some visitors may already have contextual or stylistic knowledge, others rely on additional information to fully understand the artwork. For example, this information can be provided by museum labels, which are typically brief summaries designed to guide the viewer through the key elements of a painting.

In collaboration with the Rijksmuseum Amsterdam—one of the world’s leading art museums with over 2.7 million visitors annually—this study explored the role of labels in guiding visual attention. Over a decade ago, the museum’s educational team introduced descriptive labels aiming to direct attention to specific objects or details within a painting. More recently, the museum began experimenting with new labels to encourage historical reflection, particularly regarding Dutch colonial history. These labels aim to shift the viewer’s focus toward broader social and historical contexts, inviting critical engagement with the artwork.

The experiment compared the three different label conditions: the original labels (focused on guiding attention), the new labels (focused on historical awareness), and a control condition without any labels. Tobii Pro Glasses 3 (100 Hz, Wireless) were used to record participants' eye movements during regular museum hours. The glasses proved reliable, quick to set up, and effective across diverse visitors aged 16 to 60+. In a previous study, Tobii Pro Glasses 3 were also successfully used with children.

For analysis, the researchers relied on Tobii Pro Lab software. The user-friendly interface of Tobii Pro Lab makes data preparation and analysis easy for many students. Automatic mapping tools allow mapping participants’ eye tracking data (recorded as dynamic videos) with static images of the paintings. Additionally, the AOI (Areas of Interest) tool was used, which allows users to define specific regions, such as areas mentioned in the labels, visually salient details, and grid-based divisions of the paintings. Tobii Pro Lab’s wide range of metrics allowed researchers to export and compare data across these predefined AOIs. Preliminary study results suggest that eye tracking effectively reveals how labels designed to guide attention influence viewers' gaze patterns.

The research team is discussing future collaborative studies with other museums in real-world environments.

Zsofia Pilz shares a couple of tips on using eye tracking glasses for real-world experiments:

• Whenever possible, use the 1-point calibration. This quick step (usually under 10 seconds) significantly enhances data quality by accounting for individual differences in participants' eyes.

• Plan your data analysis. Knowing what metrics you want to extract and how to analyze them can save considerable time later. Adjusting the experimental design beforehand is far more efficient than fixing issues after data collection.

Learn more about the HATlab

HATLab: https://francescowalker.com/hatlab/

Related research studies from the lab:

Walker, F., Bucker, B., Snell, J. et al. Age-adapted painting descriptions change the viewing behavior of young visitors to the Rijksmuseum. Sci Rep 14, 22880 (2024).

Walker F, Bucker B, Anderson NC, Schreij D, Theeuwes J (2017) Looking at paintings in the Vincent Van Gogh Museum: Eye movement patterns of children and adults. PLoS ONE 12(6): e0178912.