During the last two days, AR and VR professionals from the industry gathered in Munich to present and discuss current state of the art in the business of mixed reality in their fields. Attendees had the chance to learn the from the pros that use this technology from the very beginning.
The VDI-Wissensforum organizes (German-speaking) conferences and trainings for engineers and technical management. This conference was titled “AR and VR as smart-assistances – virtual technologies in industrial applications”.
Industrial Mixed Reality History
Typically the biggest industrial players in Germany are the automotive companies plus some other heavy industry as aviation. The industry here had the money to research on AR and VR long time ago (and that´s why I also joined into that circle a long time ago – to be able to research professionally in mixed reality). They have very high costs and big pain points to address. For example, an Airbus is never coming from a standardized production line – every single one is unique. There is no prototype. Basically, every client also wants it customized, too! So, you can imagine how expensive planning errors or late adjustments would get. Here you just need as much work done in bits and bytes – before you spend money on physical productions!
Therefore the German engineering started off with AR/VR very early. Many government-funded research projects were sponsored as well (like ARVIKA, ARVIDA, AVILUS, …) and are still running. Germany`s AR company metaio (now Apple) was also spun off by one of these initiatives in the very beginning. The industry joined forces very early to push technology here.
So, during this two day event companies like Audi, Opel, Airbus, Lufthansa, MAN, Porsche or Bosch presented their works (see full schedule and listing here). Research institutes like Fraunhofer IGD showed off their latest progress in tracking technology. Well-known AR-company Re´flekt hosted the event (thanks, guys! Great show!).
The industry use cases differ a lot from what VR enthusiasts think of when they talk about Riftviveplaystationgear-VR these days. Therefore I wanted to summarize what the industry does with AR and VR today (well, they do more, but let´s keep it at this today – I will only cover parts as examples):
Industrial Use Cases for Mixed Reality
Training & Maintenance
To start off, you need to train your own people. AR/VR is being used to train engineers, e.g. if a new model of a car is launched and the staff needs to know about all parts, screws, how to assemble it, how to repair it. Typically a professional training can last 2 hours and motivation might be limited when browsing through paper descriptions or powerpoints. VW developed a new training tool that uses AR glasses for the trainee (and a supervisor tablet for the trainer). The trainee gets the next step or object to work with projected into his or her glasses and gets a correct overlay of where to perform the next action (like fixing a screw). The trainer has control over the activity and can change difficulty levels (probably give more or less hints). Their results showed that training time went down significantly and the cost of transfer of learning is reduced – making it all more efficient and less cumbersome. Trainees can directly work at the object – without twisting their head to a second screen or poster. They can fully focus on their primary activity. Easy multi-user and glasses-free operation is also being tested with a projection-based system.
Maintenance can work the same way. The engineer gets step-by-step instructions projected to the glasses and additionaly needed information (measured values from the car) can be presented – without the need for a second screen somewhere behind the car. Audi showed their well-known and constantly updated eKurzinfo (we covered it ages ago). It lets end users experience an augmented manual for their new car and also get maintenance instructions. The user can for example see where to fill in the cooling liquid (an AR arrow pointing to the right spot when the smartphone camera is pointing at the engine).
Audi mentioned their analytics results and that many customers just don´t know what their car (or any product) is capable of doing. An augmented view can help to learn more about your product. The client demands just in time and just in place information – not browsing for information in a manual in your favorite armchair at home. 83% of all innovation are missed by the client (“rollt vorbei”) because they just don´t know it´s there. AR/VR is supposed to help here as well.
Planning & Analysis
Especially virtual reality became a standard tool for initial planning and car development. You design and build your hardware in 3D and are enabled to see problems early or have faster reviews and much shorter development cycles – without spending money yet on physical models. This goes from the digital mock-up (DMU) to final CAD model that will be build. Interestingly, Opel stated, that most VR work is still being done in CAVEs and COVEs to present the results. VR-HMDs are not wide-spread yet: the quality is too poor and more importantly: you need to socially interact with many people during a review. A blocked vision, fully immersed, will not work – until everybody shares a common VR space that works flawlessly.
Augmented Reality on the other hand is often used during car development integration tasks on an every-day basis. Engineers will use AR to augment the physical mock-ups (PMUs) with extra parts or use AR for guidance how to assemble additional physical parts on the overall (physical) model. You can get proof if the car can be assembled at all and get support during the task itself. When problems need to be solved, AR will help. E.g. a tube does not fit into the engine space, though it did in the DMU. Then an AR view will help to identify differences between real and virtual (“Soll-Ist-Vergleich”) – between planned and actual (see image). You can see the deviation from the 3D overlay towards the real tube. Easy measurements and comparision tasks like this can already help you out a lot. This is just one simple example, but it scales up a lot and enables discussions and decisions way quicker.
Virtual Design Reviews
Image no confirmed renderings from Airbus, image taken from AirlineReporter.com
As already mentioned during the introduction and the Airbus example, VR is typically used to previs any product yet to produce. Especially for expensive unique productions (like an Airbus) it´s worth investing the time into a virtual representation, that not only looks technically correct – but also visually appealing to show it to designers in-house but also to the client before anything got build. You try to get feedback and clearance from the client as early as possible. Walking through the product in VR obviously helps to seal the deal and avoid surprises.
During the conference you could also see different exhibitors presenting their technology. Among them was Extend3D with their professional projection based AR system. I`ve reported on it multiple times before. A rather new use case they have established includes projecting the future to-bre-created design on the exteriour of planes, boats, ferries and yachts. The workers who do the paintjob of the custom design use it as a stencil to color the complete exteriour. Before using projected AR it took many more hours and hardware effort to create a convincing design. A simple but great use case for projected augmented reality from Extend3D!
One more thing to mention that might catch your interest if you work in the industry. My partners from AugmentedRealityExperts started and presented a new program for Mixed Reality HMDs. Currently many different brands throw new HMDs onto the market. Noone really knows if they can keep up their promises. Pretty sure, that some won`t. It´s also pretty sure that you could be stuck with the wrong HMD for your use case. With tight budgets it might become a problem to find out the right one for your scenario without spending a lot of money. This is where the new initiative from AugmentedRealityExperts kicks in: you can join the partner program to get a loaner, tips, demos and the right network to exchange experiences. Check out the campaign that just launched today.
Conclusion and Take-Aways
It became obvious once more that AR and VR are already deeply routed within the industry and that all teams are hoping to get more things done in the mixed reality space. It optimizes efficiency, gets down training times and helps to visualize unbuild objects to discuss, review and redesign. Noone questions the big advantage. Although – especially 2016 – it became also apparent how still lacking standards or a bigger palette of product options often hinders AR/VR. Everybody was moarning about the shutdown of AR company metaio (that historically served as the base for most of AR activities in German industry). Fraunhofer IGD might step up into the light with their upcoming tracking solution focussing on model based tracking and sensor fusion – but it`s not there yet.
Overall the major pain points stay the interoperability and the integration into an existing IT landscape and production pipeline. Plus: a “keep it simple” user experience. You want to cross out the operator. You want to be able to use it yourself and quickly switching between devices and fully integrated with the company`s data. That´s still a challenge that is less funky but today more important than all the AR technology`s problems. Let´s get easy access to the content! As always, content is king or queen!