Eye tracking in human informatics

Prof. Dr. Tomoko Koda explains why eye tracking is a key research method in her human informatics course. 

Prof. Dr. Tomoko Koda is a professor in the Department of Information Science and Technology at Osaka Institute of Technology, where she runs a research laboratory and teaches the human informatics course.

Leveraging eye tracking in the human informatics course

Human informatics is an interdisciplinary field that studies the interaction between humans and information systems. It involves understanding how humans perceive, process, and use information and applying this knowledge to better design and improve information technologies and systems to suit human needs and behaviors.

We interviewed Professor Tomoko Koda from the Osaka Institute of Technology on how eye tracking can be effectively used in the human informatics course. The course encompasses a range of activities and methodologies aimed at studying and improving the interaction between humans and information systems.

At the Department of Media Science, Faculty of Information Science and Technology, second-year students learn about human informatics using Tobii’s eye tracker to understand human behavior better and apply this knowledge when designing information systems. By measuring and analyzing eye movement data and creating data-based reports, students train their logical thinking skills and develop skills for their future careers.

Eye tracking allows the visualization of human behavior that cannot be captured by observation

In human informatics, fundamental human behaviors—sensory perception, emotional processes, communication, and movement—are observed and analyzed. These behaviors are systematically examined using informatics principles, such as data analysis and computational methods, to identify and study human factors relevant to information systems.

In the first part of the human informatics course, students learn about motion capture, and in the second part, they focus on and work hands-on with eye tracking technology.

“Many students are familiar with motion capture from movies that use computer graphics, which easily captures their interest, making it an ideal first step. After students become comfortable with quantifying human actions through motion capture, we want students to take on the challenge of capturing and analyzing human gaze behavior, which is typically invisible to the naked eye, and express it in numerical terms”

The eye tracking is easy to grasp, non-invasive, and suitable for recording multiple participants

Why was eye tracking chosen for the course over other physiological measurement methods such as skin conductance responses, EEG, and heart rate?

“There are other physiological measurement methods, and it is possible to measure heart rate, EEG, and skin conductivity. However, signal processing of the measured results is required, and at the second-year stage, students do not yet have the appropriate knowledge for that. In that regard, motion capture and eye tracking are the easiest to measure and analyze. Especially with eye tracking, contactless measurement allows anyone to be measured as many times as needed.”

In the course, eye tracking measurements of 10 people are collected. If EEG were used, each person would need to be fitted with electrodes, making it challenging to measure these many individuals within the given timeframe. Additionally, the availability of the comprehensive biometric analysis software Tobii Pro Lab was a key factor in choosing Tobii eye tracking system.

Group analysis and interpretation of gaze data

In the human informatics course, students will analyze, interpret, and present eye tracking data to achieve the following three outcomes:
a) Objectively observe, record, and analyze human behavior.
b) Discuss human characteristics from an informatics perspective.
c) Make theoretical presentations on the results of the analysis.

“We present still images of four facial expressions (disgust, fear, happiness surprise) for 8 seconds each and measure the eye movements of 10 people (Figure 1). After tracking the students’ eye movements associated with reading facial expressions, students consider what type of analysis might be most suitable, focusing on which parts of the face people pay attention to when reading facial expressions. Initially, students’ viewpoints are often very intuitive and expressed by describing data with adjectives and adverbs, such as ‘It seems they are looking near the eyes’ or ‘After the eyes, they look at the nose and then the mouth and back."

Human informatics emphasizes the objective quantification of such intuitive and subjective observations. Students are guided in logically explaining the analysis of eye tracking data to verify their intuitive observations and how to use numerical values to present the results they want to verify. They also consider which graphs (bar graphs, pie charts, strip graphs, etc.) should be used to present the analysis results (Figure 2).”

Eye tracking is an effective tool for acquiring logical thinking and career opportunities for students

Professor Koda believes eye tracking is a very effective tool for acquiring logical thinking skills that will be useful in students’ future careers.

“Many students from the Faculty of Information Science become systems engineers (SEs) after graduation. When an SE creates a specification document, it must be written so that 100 out of 100 people who read it understand it similarly. Learning to write objective and comprehensible documents through human informatics report creation is also beneficial in business.”

Professor Koda also uses eye tracking in research studies in her laboratory. When students in her laboratory developed a game that used gaze to defeat zombies and showcased it at a school festival research presentation, the demonstration led to the students being offered a job placement by companies that attended the event.

“As the career support committee chairman, I interview many job-seeking students. Some students tell me, ‘I learned to measure, quantify, and logically analyze human behavior in human informatics, and I want to become a SE where I can utilize that.’ Practicing logical thinking is the trigger, turning point, or opportunity for their strong desire to become an engineer."

About Professor Koda’s research using eye tracking

Professor Koda’s laboratory developed an agent adapted to the user’s gaze

One of the research projects in Professor Koda’s laboratory used eye tracking to develop a conversational agent for individuals struggling with eye contact. For instance, introverted individuals are sensitive to others’ gazes, dislike being stared at, and avoid eye contact. For such individuals, even the gaze from a conversational agent can cause significant stress. A student in the lab who faced a similar issue proposed developing an agent that adapts to the user’s eye movement behavior.

“Using Tobii’s eye tracker to measure the user’s eye movements during a conversation, we developed a conversational agent that maintains a similar amount of gaze by making eye contact with users who look at the agent and averting its gaze from users who do not.”

The evaluation experiment discovered that the group of introverted users experienced reduced stress during conversations and an increased sense of affinity toward the conversational agent. For introverted users, the agent is initially programmed to avoid eye contact and gradually increases the gaze toward the user, demonstrating potential therapeutic effects such as alleviating gaze phobia and applications in interview practice.

Discover more about Professor Koda's research.