Cloud is the future of AR by Jérôme Royan

b<>com is involved in many areas related to augmented reality (AR) and its future. What can the cloud do for AR? How will it transform our lives? Jérôme Royan, Senior Architect at b<>com, is here to help us better understand these issues.

What is the AR cloud?

Augmented reality-based systems already make it possible to view digital content that fits seamlessly into the real world. They do this by relying on knowledge of the world (a 3D map of their environment) so that they can find their bearings and accurately align the digital content with reality. As a result, enabling augmented reality systems to work anytime, anywhere requires 3D-mapping the real environment, at a very large scale. And the very promise of the AR cloud is that we can build and maintain a digital 3D twin of the real world, updated continuously and in real time, to offer a shared and large-scale augmented reality experience. Additionally, the complexity of the computation needed to create and update this digital 3D twin for multiple users requires offloading those processing needs into the cloud, especially when you want to reduce the form factor of goggles and other augmented reality devices while saving battery life. For this reason, 5G and edge computing will play a critical role in the deployment of AR cloud technologies given the very low latency needs (a few milliseconds) and the bandwidth needed to offer an interactive experience.

Multiple use cases for industry and consumers

There are many use cases for augmented reality that require scaling up and providing a collaborative AR experience. In one example for industry 4.0, augmented reality systems connected to an AR cloud will enable operators to view a digital twin on-site that incorporates the plant’s 3D models, or data from IoT sensors, that is perfectly aligned with the real world. Whether for plant-level guidance, assistance in assembly or maintenance, quality control, or logistics support, AR cloud technology will offer offloaded large-scale augmented reality services. What are the goals? Productivity and quality gains and improved operator safety at the site, which can greatly increase companies’ competitiveness. And it won't just be industrial applications that will benefit from AR Cloud technology: Consumers will also be able to benefit from large-scale augmented reality services through dedicated platforms, whether for navigation assistance, city-wide games, tourism, social gatherings, sales, etc.

In the field of health care, the real revolution won’t come from scaling up augmented reality, but from 5G connectivity, which will allow AR processing to move to a private cloud or other remote option. This will enable the flexible surgical operating room of the future, based on wireless communications. This is another area where b<>com has been working as part of the European 5G-TOURS project, whose initial demos will be on view early next year.

The use cases that can flow from a technology like this are nearly endless, and right now we are only seeing the tip of the iceberg, just like when smartphones had just come out. Watch the speech by Stéphanie Beaufort, Marketing & Sales Manager, who presented multiple use cases at a panel at VIRTUALITY EXPERIENCE 2020 “CloudXR: Challenges & perspectives for XR streaming and massmarket!” » (click here for the replay)

The ARtwin project for Industry and Construction 4.0

b<>com is backing the European project ARtwin whose goal is to set up an AR cloud platform for industry and construction 4.0 running on a 5G network.

The aim of this platform is twofold:

Enable the development of augmented reality applications that can be deployed at the scale of a plant or construction site using a 3D mapping service collaboratively created and updated in real time by the users of augmented reality devices.
Update the plant or construction site’s 3D digital model in real time based on that 3D map.

Multiple use cases proposed through a partnership with Siemens and Artefacto (one of b<>com's shareholder member) will be tested during this project, including a plant-wide monitoring application that employs data reported by IoT sensors, an application to plan out a new assembly line using the digital twin updated by the platform, and finally an application to prevent and detect building construction defects.

Challenges of the AR cloud

These augmented reality systems must be able to operate anywhere at any time, but most importantly, they must show meaningful information in the right place, at the right time, to the right person in order to avoid the phenomenon of “dirty data”. It is therefore very likely that these devices will serve as personal assistants to guide users throughout the day. The devices will be aware of the environment and what their users are doing, thanks to cloud-based machine learning algorithms that draw on data from embedded sensors.

The next issue is about data confidentiality and privacy protection in both the professional and personal realms, especially when few if any locally hosted solutions are currently available.

With this in mind, b<>com is working in four promising areas:

Improving the interoperability of AR/AR cloud systems as a driver of normalization actions within the Augmented Reality Framework industrial specifications group at ETSI. Improvements to interoperability would reduce reliance on a single AR/AR cloud solution provider and make it easier to switch to solutions that do more to protect data privacy and that are locally hosted.
Developing an ecosystem as a partner on the XR4ALL project whose goals are to develop an XR community and define a 2030 research agenda at the European level.

On these first two items, we would like to thank The AREA, an essential alliance on the global stage, with whom we recently organized a science conference entitled “Engineering XR for the future” in collaboration with ETSI and XR4ALL.
Open-source development with the SolAR framework enabling the implementation and deployment of custom augmented reality and AR cloudservices that meet the specific needs of each use case and offer algorithm transparency and control while also restricting the use of confidential data.
Our expertise and development of innovative components to assist our members and partners in implementing effective custom solutions to address their specific use cases.

Thanks to the emergence of 5G and edge computing, the AR cloud is one of the most promising technologies for the coming years, not just because of the various services that could come from it, but also because of the value earned from the data that is collected, which when used transparently, will help make our everyday lives easier and gives us new ways to understand the world around us.

Feel free to contact us to learn more about opportunities to innovate with b<>com on your projects related to augmented reality.