5 factors shaping the future of XR

In this post, I wrap up our series on the future of XR in 2020, sharing my thoughts and some of our partners’ ideas on what’s cooking and the role of eye tracking as a foundational technology in AR and VR.

In the coming years, I believe we will see a significant rise in the adoption of VR and AR technologies, both in the enterprise sector and in the consumer world. Smart sensor technologies like eye tracking, will play a fundamental role in this uptake. Why? Because eye tracking can improve device performance and greatly reduce bandwidth and processing requirements. But also, because it leads to the creation of meaningful user experiences. Application developers, solution providers, and users are beginning to expect eye tracking as standard — an expectation reflected by OpenXR announcing the first extension for eye tracking, which will ease cross-platform adoption and cross-vendor development, reducing time-to-market.

In this post, I summarize the top five factors shaping the future of XR based on current innovation and collaboration with our XR partners.

The future of remote collaboration

Let’s start with HTC, a Tobii partner who brought the first commercial VR HMD with built-in eye tracking to the market. In his post Eyes on HTC: Enterprise use of XR is expanding — will XR also be the new home office interface?, Drew Bamford, takes a look at the future of work and how businesses are rapidly evaluating XR to keep workforces connected and trained. It's clear that the pandemic has accelerated the need for effective remote collaboration solutions, especially ones that can deliver lifelike experiences.

With both Vive Sync and Tobii eye tracking technology embedded in its VIVE Pro Eye headset, HTC is paving the way for deeply realistic virtual collaboration. Applications running on this headset can include natural communication features such as natural eye contact, because the native eye tracking supplies the necessary data to accurately emulate a person’s eye movements in real-time.

Eye contact is central to human interaction, which is why eye tracking will be standard on all VR headsets and why features developed with eye tracking will become foundational in professional and consumer applications. After all, whether you are meeting with colleagues or competing against a virtual opponent, there is something deeply satisfying about being able to look at each other and not appear to be staring into middle-distance like a zombie.

Accelerating demand in healthcare, simulation, and training

In 2020, Pico announced the Pico Neo2 Eye, the world's first commercial standalone VR headset with native eye tracking. In Karen Zu’s post Eyes on Pico: training and healthcare will accelerate enterprise XR adoption, she points out how headsets equipped with sensors technologies like eye tracking create new opportunities within enterprise and healthcare — including pain management through distraction and therapies such as phobia treatment.

Certain kinds of healthcare services do not need to take place in person and deregulation in some regions has led to a growth in telehealth services. But it was the need for social distancing in 2020 that accelerated the uptake of remote healthcare. It’s also likely that patient interest for online care will continue post-pandemic because it’s convenient and delivers rapid access to care — relevant factors particularly for people with disabilities and people living in isolated regions and congested cities where eye tracking is perfect for enhanced guide procedures and assistance.

XR is a telehealth enabler because it promotes care at the point need of (wherever and whenever) and can deliver continuous feedback to healthcare providers. Equipped with eye tracking, XR devices provide unbiased observations based on eye health, creating opportunities for services related to neurodegenerative disease and trauma detection and monitoring, ophthalmology, and therapy.

Personalization and intuitive devices

Tobii's most recent XR partner is HP. Back in September 2020, they announces the release of the HP Reverb G2 Omnicept Edition. With integrated Tobii eye tracking, this headset enables hyper-personalized, engaging, and adaptive VR experiences for enterprise use. In his post Eyes on HP: better HMDs, social interaction for work and life, and BYOD, Peter Peterson emphasizes the importance of personalization. He predicts that XR headsets will become more customizable and that we're going to see device manufacturers catering for specific markets based on how you want to wear a headset, face size, shape, and individual needs.

Personalization creates an opportunity for application developers to build new services and unique content that targets XR devices. He further talks about more XR technology, and how software companies and independent developers will work together to ensure platform-agnostic solutions and homogenize the ecosystem — which is in line with Tobii's work to standardize the core part of eye tracking in OpenXR.

Boundless connectivity

Qualcomm, one of our long-standing XR partners, is one of the world's most recognized semiconductor tech companies. They have integrated Tobii eye tracking into multiple generations of their reference designs — including the most recent (2020) Tobii Eye Tracking in the new Qualcomm Snapdragon XR2 5G Reference Design.

In his post, Eyes on Qualcomm: What will the future of XR bring? Brian Vogelsang talks about the importance of a concept they call Boundless XR — where on-device processing can be offloaded to, for example, an edge or even a cloud server over a low-latency 5G link or to a PC over Wi-Fi. This kind of seamless connectivity with a split-rendering architecture is critical to VR and AR adoption because it supports high-quality graphics and off-device processing.

Tobii Spotlight for Foveated Transport leverages gaze, optics, and visual points of interest to enable a massive reduction in bandwidth requirements in the graphics rendering pipeline. The expected level of gain will ensure the delivery of high-quality graphics across the entire headset display (not just in the center), improving user comfort and ensuring quality user experiences over mobile networks, contributing to the adoption of mobile VR.

Enhanced devices, enriched experiences, and actionable insights

Looking ahead 1-3 years, I envision a massive increase in XR device performance with always-on eye tracking as a core design element. Enabling solutions like Tobii Spotlight Technology™ will enhance devices because they support efficient use of graphical capabilities and band bandwidth utilization in boundless connected HMDs as well as supporting natural interaction such as eye contact and gaze enhanced UI and interaction paradigms. Tobii provides eye tracking platforms for all optical designs of XR hardware, be it a standard VR Fresnel lens, compact (pancake) lens, or AR designs, to adapt to different needs in various markets.

Native eye tracking delivers a set of analytics that can be leveraged by applications to solve previously unsolvable problems. Because of this, I expect strong growth in healthcare, training, collaboration, and research applications. To deliver on this expectation Tobii recently introduced Tobii Ocumen, a premium eye tracking solution that brings two decades of experience and expertise together to optimize for the development of specialized VR applications. Because Tobii Ocumen delivers high-level observations, it saves development time, cutting time-to-market for applications, and enabling developers to materialize their innovations in an affordable way.

The current level of development, ISV innovation, and investment by headset manufacturers that I’ve mentioned in this post are just a few indications of the rising adoption of XR technologies, with enabling technologies like 5G and edge cloud now becoming part of the XR ecosystem.