The concept of the ‘digital twin’ and modeling the real world has been driving innovation in civil engineering for over 20 years, and we have leveraged technology to build, shape, and maintain the physical world around us. Spatial computing, simply put, is a word to describe further reducing the boundaries of where our digital and physical worlds meet and how we interact with them.
Imagine if your computer or phone could clearly understand and seamlessly interact with the space around you, like your room or the street, to provide information in the context of your environment without forcing your attention away from the real world to make your life easier. It's all about mixing virtual and augmented reality (VR and AR) with what's real. So, you might put on special glasses or look through your phone, and you can now see digital overlays- like information, images, or animations - right on top of the actual world.
It’s technology working in harmony with humans on our terms, which is closely tied to attention computing and exactly what we’ve been working toward at Tobii for the past 20 years. And the most exciting aspect of spatial computing? When technology has the intelligence to understand what you are paying attention to, devices have the information needed to react to gaze or gestures in a manner that augments your reality with information that may otherwise leave you wondering or digging for your phone or some other device order to find the answers you need. We progressed from the inefficiencies of pen and paper to digitalization and are finally lifting our heads up from the small portal into the wealth of digital information and bringing it into our three-dimensional space.
The applications for spatial computing are limitless. My experience in civil engineering allowed me to experience the power of spatial computing in its most practical sense. The value of both virtual and augmented reality in allowing me to visualize complex digital information was what sparked my interest in the technology and its potential.
Gaming and entertainment
Just by looking at what the latest AR and VR products have done, including the
Sony PSVR2, which uses Tobii eye tracking technology, we can see how spatial computing can bring fun to a whole new level. Spatial computing already shows promise in facilitating remote collaboration and enabling shared virtual experiences. It will only get better as technology and content mature. The sense of presence that spatial computing provides is just so much more engaging than traditional communication tools.
Education
Think about how students could access experiences and information which was previously almost impossible. Now, they can go from visualizing and experimenting with a circuit board to experiencing the immense scale and proportion of the Great Pyramid of Giza without leaving the classroom. There are many ways in which presenting information within a spatial context helps us better understand concepts, especially for people with a more visual learning style. It is not visually enhancing either, as spatial computing introduces the opportunity to include more senses.
Healthcare
Spatial computing is already used in the medical industry, from helping doctors diagnose illnesses like concussions to helping surgeons not only train but also visualize what they cannot see on the surface to enable them to perform complex procedures safely and effectively. This, of course, can be applied to other industries with similar levels of complexity, as well as step-by-step instructions for even the simplest of tasks.
Marketing
This technology allows you to experience product visualization and customization in nearly the same way as holding it in your hands. This enables virtual trialing, customization, and selection of products with the help of the same senses you would use in a store, eliminating uncertainties and helping drive a buying decision. There are endless opportunities for context-specific advertising, interactive marketing content, and immersive branding that connect much more deeply with customers.
Eye tracking paired with spatial computing is a match made in innovation heaven. It is inevitable that there will be a point in time when technology can achieve the vision of spatial computing that was sparked decades ago.
Natural interaction
When eye tracking is integrated into the device, enabling spatial computing, the device understands where you are looking. Fundamental to artificial intelligence being useful is understanding human attention and intent. Simple tracking of a user’s gaze enables a more seamless and natural user experience. Immersion puts the stakes much higher as anything ‘unnatural’ will be annoying at best but generally brings on physical symptoms such as nausea and headaches. Hangover-inducing technology is far from natural.
Foveated rendering
It plays a critical role in enhancing visual clarity for users experiencing digital virtual content, whether overlaid in the real world or in a virtual environment. Nobody wants to carry around a bulky GPU attached to their head. Foveated rendering optimizes processing power utilization, maintaining high frame refresh rates to prevent skipped frames and ensure a smooth user experience.
Contextual awareness
This is crucial for spatial computing to work. When a device is aware of where you're looking, it can offer valuable feedback in various scenarios: social interactions, menu navigation, content browsing, and context-based information provision. This capability is significantly amplified with the integration of artificial intelligence.
Adaptive interfaces
Eye tracking enables adaptive interfaces that respond to the user's gaze. It allows for contextually relevant information to be displayed or hidden based on where the user is looking. This can improve usability and reduce cognitive load by presenting information precisely when and where it is needed, enhancing the overall user experience.
Behavioral analysis
Eye tracking data can provide insights into user behavior, attention, and preferences. Spatial computing systems can analyze gaze patterns and use that information to personalize experiences, optimize content placement, and improve user interfaces. This data can be leveraged for user research, usability testing, and further advancements in spatial computing technologies, as well as simply digitizing long-established eye tracking research practices.
Tracking and localization
Accurate tracking and localization of the user's position and movement in the physical world are crucial for seamless spatial computing experiences. Robust algorithms and sensor fusion techniques are needed to ensure precise and real-time tracking, in complex and dynamic environments. This is quite different than walking around or moving in front of a display panel and positioning of the user’s eye is critical to the user experience beyond the simplest of notification overlay use cases.
Environmental mapping
Creating and maintaining a detailed digital representation of the physical environment is essential for spatial computing. Challenges include accurately mapping physical objects and surfaces, handling changes in lighting conditions and occlusions, and ensuring consistency between the virtual and physical worlds. Eye tracking aids in accurately rendering digital content and aligning objects with the user's perspective.
Interaction and input methods
Spatial computing requires intuitive and natural interaction methods. Developing input mechanisms that allow users to interact with virtual objects, perform gestures, and provide input commands in a comfortable and precise manner is a challenge. It involves designing effective user interfaces and leveraging technologies like hand tracking, voice recognition, and gesture control, which are all dependent on eye tracking as a key input for object and menu selection without being overly dependent on complex or awkward movements. This is particularly true as the duration of use increases and repetitive movements become increasingly tiresome, and there are also many handsfree use cases in the real world.
Computational power
Spatial computing involves processing large amounts of data in real time, including sensor inputs, environmental mapping, rendering virtual objects, and delivering seamless user experiences. Ensuring sufficient computational power and reducing latency while optimizing performance are significant challenges, especially for mobile and wearable devices. Foveated rendering and data transport are some ways the processing load can be reduced to enable higher visual quality and comfortable performance.
Content creation and standards
Developing high-quality content for spatial computing requires new tools and workflows. Designers and developers need to adapt to the unique requirements of creating 3D virtual objects and environments. Additionally, establishing interoperability and standardization across different spatial computing platforms and devices is crucial for widespread adoption. Tobii has a wealth of knowledge in this that we make readily available to developers through our website and continue developing our platforms to deliver core signals compliant with the latest standards.
Privacy and ethical considerations
Eye tracking may be leveraged for proper positioning, fit user profile management, and to enable convenient sharing of devices. However, a critical concern is user privacy, data security, and ethical considerations. Capturing and processing real-world data can raise privacy issues, and it's important to establish clear guidelines and safeguards to protect user information and ensure the responsible use of spatial computing technologies. Tobii works closely with all of our partners in order to ensure our data transparency policy is enforced to protect the data integrity and data privacy of every person who interacts with our technology.
The concept of spatial computing has already been gradually changing various industries, which have been limited more by technology than human imagination.
Integrating eye tracking technology, especially with Tobii’s solutions, enhances the synergy between human-machine interaction, making the overall experience more natural, comfortable, and insightful with smaller and more efficient devices and smart glasses. Whatever words we use to frame the progression of how we interact with virtual content, people will only be interested when the technology is mature enough to support meaningful use cases, as the pure novelty of wearable displays alone was exhausted long ago.
Whether the entry point is a VR/MR headset or AR smart glasses, considering an eye tracking supplier is a strategic move toward unlocking the full potential of spatial computing. At Tobii, our expertise in eye tracking, combined with its commitment to overcoming technical challenges while providing low-cost and scalable solutions, positions us as a reliable partner for overcoming the current challenges to innovation in the XR industry. Explore the transformative possibilities of
eye tracking and spatial computing with Tobii to create cutting-edge products that redefine user experiences as the gap between virtual and real worlds continues to narrow.
Want to find out more about XR integrations? Talk to one of our eye tracking integration specialists today.
Written by
In my role, I get to work directly with headset manufacturers, helping them integrate eye tracking into their hardware. My focus is the introduction of eye tracking for effortless interaction and immersion in virtual and mixed reality as well as enabling more capable devices with solutions such as foveated rendering and analytics. Personally, I am excited about the future of spatial computing, which helps me greatly in my role because I am passionate about working closely with our customers and engineering team to drive the widespread adoption of eye tracking in XR.
In this first post of our spark series, Tobii's Johan Bouvin talks about the healthcare use case — looking at the convergence of VR and eye tracking, the benefits for healthcare, and some of the tools available to accelerate innovation.
In this XR developer focus post, Johan Bouvin summarizes the impact of our eye tracking in upcoming VR headsets.
Since Tobii Ocumen launched in 2020, we have collaborated with a multitude of companies working to commercialize their VR innovations. Here are our top five.
Subscribe to our stories about how people are using eye tracking and attention computing.
