Why the retrospective think-aloud interview is a game-changer in qualitative research

In the search for truth at the bottom of human behavior and motivation, you may find the retrospective think-aloud (RTA) protocol a useful tool. RTA involves participants watching a video replay of their gaze, recorded while they were performing a specified task, and explaining the thoughts and feelings which accompanied carrying out this task. This method helps preserve natural behavior in the moment and gives access to a person's motivations with more accuracy than plain reflection.  

This article explores the use of the RTA interview alongside eye tracking and other data collection methods as a means to get more valuable insights from your research.  

Why the RTA interview and eye tracking is useful 

 
If you ask ten researchers to name the most valuable output of eye tracking, you'll probably get ten different answers. Heat maps or engagement metrics or impact zones or visibility benchmarks – the variety of pathways from raw data to actionable findings is nearly endless. And yet, there is one powerful benefit that I rarely see mentioned: the ability to enhance qualitative interview results. Most of the research reports you'll read from the field of eye tracking are all about fixation data, those handy X and Y coordinates that allow us to build a quantified aggregate understanding of the visual experience. The value of these kinds of results is beyond dispute, but it's also worthwhile to consider how attention data can be used to enhance more subjective, perception-based, messy methodologies. 

The two dilemmas of the qualitative moderator 

 
The traditional interview dilemma: If you're a qualitative moderator executing a study, your job is to elicit true and accurate feedback as a means of answering critical questions on a topic of interest. It doesn't sound that hard – talking to normal people about their thoughts, experiences, and motivations – but in reality it's incredibly challenging, mostly because people are challenging. Interview participants can be forgetful. They can be biased. They can be eager to please and fuzzy on details and unaware of their blind spots, and they can even be untruthful at times, without even realizing it. All of these natural pitfalls of qualitative research make it hard to collect accurate data, especially when the interview is often far removed from the context that the interviewee is asked to discuss (e.g. sitting in a bland, taupe focus group room trying to vividly describe the experience of filling your cart at the grocery store). 

The think-aloud dilemma: Part of the dilemma described above has been solved by incorporating "Think-Aloud" interview protocol, in which the moderator elicits feedback in real-time as the interviewees complete the activities that they are being asked about (e.g. ditch the focus group and ask the shopper what they are doing while they are actually shopping for groceries.) This type of concurrent interviewing cuts down on forgetting and captures more detailed contextual insights, but there is one major downside. The Observer Effect is a concept from physics that suggests that the mere act of observing a phenomenon changes the phenomenon being observed. Admittedly, qualitative research is a long way from physics, and yet the Observer Effect is an apt consideration in this case. For a participant, the act of verbalizing an experience to a moderator during a task (i.e. thinking-aloud) will always make the observed task less natural. So in effect this technique changes the behavior of interest while simultaneously capturing feedback on that unnaturally changed behavior. Obviously, this is not ideal for building results that may be submitted to a journal or used to inform critical business decisions. 

The retrospective think-aloud interview 

As described above, two of the main drawbacks to qualitative research are (1) out-of-context traditional interviewing can be unreliable and (2) in-context "Think-Aloud" interviewing can be misleading. But what if there was a way to capture all of the detailed, second-by-second insights from a "Think-Aloud" while maintaining the unobtrusiveness of a traditional interview? This is where eye tracking swoops in and saves the day. The Retrospective Think-Aloud Interview (RTA) is a technique that utilizes post-interaction replay of gaze video as a visual cue during a qualitative interview. This may sound complex at first, but actually it's a simple, straightforward method: 

• Step 1: A research participant is given a natural task to complete while an eye tracker is used to accurately record the interaction through the participant's own eyes. 

• Step 2: The research participant completes the task of interest without interruption (e.g. using a smartphone, browsing a site, shopping at a store, completing a job task, navigating a VR world). 

• Step 3: After the task is completed, the moderator cues up the eye tracking recording for playback, which provides an unparalleled vantage into the full experience from start to finish. 

• Step 4: The moderator then plays back the recording and guides the participant through a detailed discussion, allowing her/him to narrate in retrospect critical moments from the task and answer the moderator's direct questions about interesting behaviors and reflections 

Moderating an RTA requires some training, but you don't need to be a vision scientist or computer wizard to run this type of interview. Once learned, the technique quickly begins to feel automatic, even for old-schoolers and technophobes. The learning curve is quite steep, which makes perfect sense when you think about it. Humans are intensely visual creatures. Our experience is built sensation by sensation, and none of our senses account for more of these building-blocks than our sense of sight. For a qualitative moderator charged with understanding people as deeply as possible, the ability to see what they see is a huge advantage. The further step of closing the loop and engaging in a detailed retrospective discussion of an activity feels quite natural to both the participant and the moderator because – to paraphrase William James – Attention is experience. 

That's why the learning curve is steep. The RTA is not an example of technology disconnecting researcher from subject; it's quite the opposite, in fact. The use of eye tracking in qualitative interviewing creates a level of connection between moderator and participant that is not achievable through any other means. As researchers, the closer we can get to living the experience of real people, the better we will be able to understand people, explain behavior, and reveal truth.