Qualcomm has introduced the idea of extended reality (XR), combining and going beyond the disciplines of augmented (AR), virtual (VR), and even mixed (MR) reality technology. More than this, it is talking about ‘boundless XR’, which leverages industrial 5G and distributed AI, when powerful AR and VT headsets will lose their tethers and roam completely free.
In research for a new report on industrial AR, Enterprise IoT Insights caught up with Brian Vogelsang, senior director of XR product management at Qualcomm to discuss the challenges with contemporary AR (and VR) setups in the industrial space. Here is the interview, in full; the new report – called Industrial AR, and the rise of the workers – is available to download here.
How are AR and VR being used today in manufacturing and production?
“AR and VR are being used in manufacturing to optimise performance and get jobs done faster, more accurately, and safer by connection people, information, an equipment.
“AR is a spectrum from assisted-reality monocular glasses such as Vuzix or Google Glass that act as a heads-up display to fully immersive binocular augmented reality devices like Hololens 2. Assisted reality typically provides a heads up display where a user can see video / documents that act as instructions or view workflow document critical steps, create audit trails, and even teach the steps of a process.
“Ultimately AR is being used to get the job done safer, faster and with greater accuracy. This could range from safety checklists, detailing instructions for complex assemblies, quality control, or for in-service inspection. AR can allow for documenting critical success so there are more audit trails for manufacturing. The device would walk through a process from the AR glass via a digital overlay, allowing the workflow to go quicker since the user is hands free and there’s no need to look at a computer or piece of paper.
“This is also a great way to troubleshoot equipment with a remote expert or mentor. That expert can guide through maintenance or repair tasks remotely, without the worker physically there.”
“VR is more about simulation and immersive training. It can simulate an environment, such as a manufacturing plant or an assembly line to train workers in a virtual format instead of the real world. The cost of dedicating a factory line solely to training, or taking one offline to train, is very high. Plus, you likely don’t want to train someone on a live production line for many reasons including safety concerns.
“Putting someone in VR allows you to train in simulation, which is a lower cost, and safer. With VR, companies can train more employees simultaneously, increase their retention of the task, and do so with greater employee satisfaction and engagement.”
What is the potential – how will AR and VR be used in the future?
“Some of greatest potential for AR and VR is the technologies’ ability to upskill people quickly and make workplaces safer, especially in manufacturing. For example, if a company had a seasonal workforce where manufacturing ramps up for pique periods time period, they would need to bring on more staff and get them up to speed quickly.
“This technology could speed up the on-boarding process immensely, enabling interactive training sessions with a higher degree of employee engagement. Realistic training scenarios that would otherwise be expensive or dangerous can be simulated, and training can be run on-demand without the need for an instructor on-site.”
How is the experience, today? Does the technology work well and easily? How clunky are headsets, and how prohibitive is the pricing?
“The experience today is good and continuously improving, but there is still room for growth. Headsets need to be lighter, more comfortable and not produce any strain so they can be worn for a full shift. As the technology begins to scale in its deployment the costs will come down.
“Today, both AR and VR are benefiting from sourcing components and technologies used in Smartphones, and the prices are coming down each year as the market begins to scale. Enterprise VR and AR are benefiting from the strong demand for Consumer VR as it scales up.”
What are the challenges / barriers from a technology point of view?
“The technology and form factors need to become smaller, lighter weight and have longer battery life. The challenge is to make the technology more power-efficient so it can be used 8-12 hours without a charge, but not add weight which requires all the components to be more power efficient.
“Display and optics also need improvement, particularly in AR, we need to see higher resolution displays with wider fields of view, they need to be more thermally efficient and lower cost.
“We take a systems approach to designing chipsets. Our heterogenous compute architecture is designed to be power-efficient and smaller, which helps reduce the size of the headsets and pack more technology into smaller footprints.”
What are the challenges from a behavioural point? Do staff like the tech?
“Many roles within manufacturing have not traditionally used computers, so this technology can be met with resistance. The technology must not be obtrusive, or feel like it’s getting in the way, which is why it has to be small and lightweight.
“To showcase the value and promise of the technology, it’s important to illustrate how it’s improving efficiencies rather than presenting it as a measure to track performance. If it’s seen as an obstacle to producing quality work, people will be less likely to embrace it. The perfect scenario is that it works cohesively with workers to present itself when needed, and is able to be perceived as invisible when it isn’t needed.”
What are the requirements on connectivity? Is a wired connection preferred? Will LTE suffice? Will 5G make this more viable?
“Wires are used in both VR and AR today, but these are going away. A wired connection is something we see as an interim solution to help distribute processing between a lightweight AR glass that a worker is required to wear for an extended period of time and an external processing or compute device like a smartphone or a compute pack.
“The wired connection between the AR/VR device to a processing puck or smartphone allows heavy workloads to be offloaded. However, since a wired connection in a manufacturing space could potentially be hazardous, we need to go wireless. We see 5G connectivity as a transformative technology for AR/VR. With the mass deployment of 5G networks, many manufacturing facilitates will likely set up private 5G networks.
“5G enables lower latency and high throughput connections, and with a 5G modem in an AR/VR device, it can unlock what Qualcomm calls Boundless XR. With Boundless XR, an AR/VR device hosts some of the processing on-device and distributes heavier computing in edge computers located in the on-premise network. This will allow for more sophisticated and photorealistic graphics that some manufacturing facilities would require.”
What are the requirements on edge processing? Where is the edge for AR and VR – on the device, on site, in the mobile network, in the cloud?
“With the deployment of private 5G networks, the edge is the factory’s data centre on premise. With Boundless XR, manufacturing workers can run an app off the headset and distribute the workload to the edge, allowing the GPU in the mobile edge to do a significant amount of the computation (that requires a bigger GPU), then stream back to the headset for display.
“The headset and network work together to deliver a high-end experience. Splitting the processing, but working together, as there is some processing on the device.”
How should the AR/VR headset develop? What needs to happen?
“I briefly touched on this, but additionally we need to make higher resolution displays and optics, improve field-of-view (FOV) to increase the area being augmented, enhance processing power in the headset, add more AI, increase the sensing capabilities to better understand the environment, the context of the user, and their tasks in realtime…needless to say, there is a lot of opportunity for AR/VR headset development.”
Who are you working with on this? How are they using these solutions?
“Qualcomm has a unique role in the ecosystem, we of course supply the SoC, but we work across OEMs and solution providers that enable software that enable software solutions on top of the hardware.
“With our XR Enterprise Program (XEP) initiative, we work with different solution providers who understand how to apply technology on the hardware and sell solutions to solve specific enterprise problems such as training, design & collaboration, data visualisation, remote expert, or guided work instructions.
“We created XEP to connect turnkey products and applications that work with the hardware. Large manufacturers aren’t simply buying the hardware, they’re buying the solution that makes their technician more efficient. At Qualcomm, we are committed to delivering the best technology to enable customers and enrich lives, and we will continue to help support mass enterprise adoption of XR and manufacturing is no exception.”
Check out the editorial report on industrial AR, Industrial AR – and the rise of the workers. The report is available to download here.
