Mixed Reality meets the workplace

Microsoft's mixed reality solutions are empowering front line workers, bridging the gap between physical and digital worlds

Tags: Augmented realityKazendi Ltd (www.kazendi.com)Microsoft CorporationMixed realityPTC (www.ptc.com)ThyssenKrupp GfT Bautechnik GmbHTrimble
  • E-Mail
Mixed Reality meets the workplace Martine: Microsoft is building the MR ecosystem and platforms so that businesses can adopt the technology more easily.
By  Mark Sutton Published  June 19, 2018

Virtual reality as a concept has been around for decades, but until very recently, the technology hadn't achieved much beyond inducing eyestrain and nausea in more sensitive users in development labs and games arcades. Now however, VR, AR and in particular mixed reality (MR) are finding their way into real world use deployments in the workplace, in environments ranging from construction sites and factory floors, to architect's studios and hospital operating theatres.

Microsoft is one of the leaders in this drive for business adoption of MR, initially through its HoloLens device. HoloLens, a mixed reality, stereoscopic headset with an onboard 1Ghz computer running Windows 10, began shipping to developers in March 2016, and has since been adopted in many pilot projects. The headset is just part of the proposition however, with Microsoft developing a Windows Mixed Reality platform and MR applications alongside of it.

Leila Martine, Mixed Reality Product Marketing Director, Microsoft UK said that while HoloLens is an important part of the company's MR play, particularly in creating experiences that deliver the ‘wow' factor to gain initial interest, delivering the platform and an ecosystem is the main goal.

"What we are really focused on right now is using HoloLens as the highwater mark of innovation, for getting into the vision of true mixed reality, and the other element is the platform," she said. "With Windows, we created a great technology platform, and our partners sit on top of that, creating ecosystems - this is exactly the same model - Azure is our platform, and HoloLens is a smart edge device, it is the same model just updated for the future of computing technologies"

MR, along with quantum computing and AI, is very much at the centre of how Microsoft sees computing developing beyond the information era and into the digital era, Martine told itp.net. Digitisation has created a proliferation of data, digital content and IoT-enabled devices and objects. Systems like Building Information Modelling (BIM) and CAD applications are creating digital models and ‘digital twins' of objects in environments like design and manufacturing, but there is often a disconnect between these tools, how we interact with them on a 2D computer screen, and with the people working with real objects in the physical world. MR has the potential to integrate the entire workforce, including first line workers, by giving them new ways to connect to the data.

"Previously we have been digitising by making a paper checklist into a digital one, on a phone," Martine explained. "That is some degree of digitisation, but the reality is that you are still requiring that mechanic to take their hands off the tools of their trade, to be able to pick up a device and use the checklist."

Over the two years that HoloLens has been in the hands of its customers and partners, Microsoft has seen a wide range of test projects and different types of ‘first line' workers getting connected and becoming empowered to do more tasks through on-the-spot information, connectivity and hands-free operation.

Sectors as diverse as construction and healthcare are developing projects to empower workers, and the scope of MR is expanding as organisations try out new ideas.

Trimble has developed a number of Mixed Reality solutions for use in construction, architecture and related fields.

Surgeons in the UK are using Mixed Reality to plan delicate surgical operations

Elevator manufacturer ThyssenKrupp was an early adopter of HoloLens, Martine noted. As part of its digital transformation program, the company first began deploying IoT sensors for its installed elevator fleet. It then developed a solution on HoloLens which supports its elevator technicians, with an MR visualisation layer and remote assistance, so the technician onsite who is repairing an elevator or carrying out maintenance, can work hands-free, while referencing real time data or connecting to an expert back at head office, for indepth support.

The success of the solution in empowering the technical team has now led to the development of a solution which will be shared by technicians and by customers. The company also supplies stairlifts, which are installed in the homes of elderly and disabled people, to help them get up stairs. The new solution will enable a technician on site at the person's home to measure up and design the lift, and then show how it will look in place to the homeowner, using HoloLens.

Microsoft partners are also addressing the uses of the technology. UK startup Kazendi began developing MR and VR meeting solutions using Google Glass, but switched to HoloLens because of its greater technology capabilities. The company has developed a HoloLens-based meeting solution which allows meeting participants to meet in an MR environment, and share, view and collaborate on popular types of digital files. Kazendi has worked with a number of FTSE companies on the technology.

Max Doelle, chief holographic officer said that the HoloLens technology frees meeting participants from the constraints of a 2D screen, to have as much 3D space as they like. HoloLens now supports GPU-based streaming of 3D renders from Azure, without processing on the device, increasing the depth and power of the simulation. Combined with the freedom to move around a design and interact from any angle, this creates a highly interactive meeting environment, that he says much like telepresence, can save companies a huge amount of time and money compared to physical meetings.

Boston's PTC has added another important component with the release of a HoloLens content creation tool. The PTC ThingWorx Studio MR authoring and publishing tool allows users to take any 3D CAD model, and drag-and-drop it into the application to instantly create a VR experience. The 3D model can be shared, and edited.

Chirag Mehta, director of IoT Strategic Alliances PTC, said that the solution saves time against the normal three to six months to develop an MR model, and also removes the need for coding expertise. The solution has a range of uses, for example, the company has developed a training solution for BAE which helps workers on a complex assembly line task, assembling batteries. Because workers are able to see the task in front of them in the real world, and follow the steps for assembly in MR, they are able to significantly speed up training and actual production time, and reduce errors.

PTC is also applying its IoT expertise to these MR models, Mehta added, to create things like procedure templates for servicing. An IoT-connected machine, pushing out real time sensor data, can be viewed on site in MR, giving a technician an up-to-date view of how the machine is functioning.

The MR ecosystem is growing and expanding, with more partners developing new use cases, and more support through things like ready-to-use applications. Microsoft launched two new HoloLens applications at its Ignite Conference in May, Layout and Remote Assist. Layout enables users to design physical spaces in a live virtual environment, and to move 3D objects around the environment. Remote Assist allows a technician on the ground, wearing a HoloLens, to be guided by a remote expert, to complete tasks like maintenance. The solution enables collaboration, live instructions and so on, so that field staff can be assisted at every step of a task. Microsoft has also announced integration of MR into SharePoint, so that companies can share and collaborate on SharePoint content.

Building these apps is part of helping organisations to get to grips with MR, and saving them the effort of having to build functionality for themselves, Martine said. Applications like Remote Assist have an obvious potential in areas like field maintenance, while other MR applications may require more time for adoption, but what is important is that the MR apps and tools empower the business user. The ecosystem is focused on helping companies to go from pilot to production, she said, and to find where the technology can bridge the gap between digital and physical to address the needs of the organisation.

"Where we are at right now, you can get stuck in the hype and hyperbole about what is possible, but really, this is 'where the rubber meets the road' - when you are coming up with real use cases, to solve real challenges," she said.

"The device is not where the story begins and ends. It is around how does the technology disappear into the background - what is the right tool, how does the platform enable that, and how do all of those associated technologies fit naturally within the workflow - that is where the magic happens."

Add a Comment

Your display name This field is mandatory

Your e-mail address This field is mandatory (Your e-mail address won't be published)

Security code