A VR study of human behavior

The use of eye tracking in virtual reality for training and development or to research human behavior and decision-making processes is highly effective because visual attention is so strongly correlated with cognition and performance. The study discussed by Dr Tim Holmes below, explores how behaviors that are observed in VR environments mirror those in real-world ones. This validates the use of VR as a tool to aid professional training and development, and expand research possibilities relating to product design, wayfinding, safety and much more. 

 When art imitates life  

For the past 14 years I have been researching how to best use technologies like eye tracking to understand and predict people’s responses to art and design. This is a challenging field of research because the behavioral responses we capture with an eye tracker are affected by the context in which we observe them. 

We have seen for example, that the viewing patterns of large artworks are quite different when viewed on a small screen as opposed to a full-size one, and that environmental factors such as lighting, proximity to other artworks, and even the presence of a title or not can all change the way in which the artwork is viewed. Virtual reality presents a unique opportunity for research in this field since it allows eye-movement data to be collected whilst participants view works at scale from a comfortable viewing position in a controlled environment. 

Of course, VR can do more than simply mimic real-world environments, it can also facilitate research into real-world scenarios without any dependence on the presence or existence of a physical environment - allowing design alternatives to be tested in context, non-longer existing works to be recreated, and even completely original and sometimes fanciful environments to be presented - as is the case in an ongoing project at Royal Holloway, University of London, being led by my former PhD supervisor Professor Johannes Zanker. 

You are probably already familiar with the artist Piet Mondrian, who is best known for his abstract artworks comprising horizontal and vertical lines forming rectangles which are filled in white, grey, black, red, yellow, and blue.  

However, you might be less familiar with his one, and only, foray into three-dimensional architecture, and for good reason - the design he created for Ida Bienart for an entire room in this same style was never actually realized in his lifetime. The reasons for this are debated but might include the way orthogonal lines no longer appear to form rectangles when viewed in a three-dimensional perspective. This of course, is not the case once immersed in such an environment, where the perspective is no longer forced and changes with movement. 

How effectively does VR solicit behaviors that would be present in the real world?  

A few museums attempted to realize Mondrian’s design after his death, but these attempts were met with limited success. In fact, the creation of a 3D model based on his original plans requires some difficult decisions to be made - since Mondrian was no architect, his walls do not join up! 

Perhaps more surprisingly, whilst the relative size and color of each rectangle is clear in the plans, the precise tint of the red, yellow and blue “paint” is not, and as any designer knows, color REALLY matters. It certainly did to Mondrian, since he believed his designs used color and shape to create a “universal harmony”. 

It is here that research in VR really comes into its own, because the Unity 3D model was developed and tested iteratively to create a final design that felt both natural when viewed from ‘within’ it and also remained true to Mondrian’s aesthetic. 

Virtual learnings for real-world application  

The research confirmed several expected results, such as: 

• The red patches in the room receive more attention than the other colors, confirming the use of red for high-impact, standout messaging and signage. 
• Participants rarely looked up at the ceiling, and even if they did, they didn’t do it for long. In general, looking up is not comfortable for humans and results like this highlight the importance of placement and advantage of eye-level presentation for attention. 
• The room contains solid 3D furniture which project out into the room, and these attracted more attention than the flat walls or the “view” through the open windows in the room. 3D objects create visual interest in an environment that would otherwise be just flat walls. By protruding from the walls, they also impact movement through the space, which is a well-known method for attracting attention. 

Results like these confirm aspects of behavior we see in more natural environments such as supermarkets for example and highlight the potential for crossover research in the art and retail sectors. 

Other data collected in this environment has confirmed some eye-movement behaviors that are well reported outside of VR, such as a reduction in fixations when performing a targeted search task such as counting the colored the squares, and an increase in fixation duration when participants are allowed to ‘free-view’ in the Mondrian Room. Such results provide strong support for the use of VR to study naturalistic behavior, since similar results from real-world research can be seen within VR. 

By comparing the physical build of the Mondrian Room in Dresden with the VR version, we observed an important result for anyone looking to use VR as a proxy for the real world. Overall viewing duration, fixation duration, and fixation counts were all slightly higher in VR than in the real world, and this is something we need to research further to fully understand. This might turn out to be an artefact of VR itself since it is still a novel technology for most participants and so there is a tendency to look around in wonder! For researchers, this highlights the need to familiarize participants with VR to allow the “wow” factor to fade before commencing data collection. 

What this means for research in VR  

The flexibility of VR is one of its biggest advantages because we can easily change elements whilst keeping everything else constant. Moreover, the behavioral evidence confirms the value of eye tracking within any VR environment where the behavioral impact of design is of interest, be it architectural, educational, or retail. 

Perhaps more importantly, the results described here confirm that eye movements recorded in VR are as informative as those we have been recording and analyzing for years using screen-based, and more recently, mobile eye trackers. This is important because it paves the way for VR applications that use the gaze data to learn about the viewer and adapt the VR experience to them. 

VR is still a relatively new methodology for research using eye tracking, but results like those reported here, show that it is a good proxy for real-world research or training environments since the behaviors exhibited are so similar. For those looking to avail themselves of the scalability, cost, and time saving that VR can provide, the results from these studies should provide the confidence to dive-in and take a closer look at eye tracking within VR. 

Related information

• Mondrian Room VR project