Has Augmented Reality reached Inflection point?

Sep 29, 2016 • Features • Augmented Reality • Future of FIeld Service • Scope AR

Augmented Reality has the potential to transform the way we approach field service but is the technology ready to go mainstream? Kris Oldland talks exclusively to Scott Montgomerie, CEO of Scope AR...

Any regular readers of Field Service News will know that we’ve been fans of the potential use of Augmented Reality (AR) for some time here at Field Service News towers.

However, in the past it has sometimes felt that we were something of a lone voice in championing the potential of AR to possibly change the way that a number of companies operate their field service operations, with AR seemingly get far less attention than other emerging technologies like Cloud Computing, Internet of Things or even Wearables.

Yet there is a definite sense that things are beginning to change.

In the past it has sometimes felt that we were something of a lone voice in championing the potential of AR to possibly change the way that a number of companies operate their field service operations, with AR seemingly get far less attention than other emerging technologies like Cloud Computing, Internet of Things or even Wearables.

At the recent Field Service Forum held in Amsterdam, I hosted a series of round-tables on the topic of AR and whereas just a year ago the conversation was very much around understanding the technology and its application within a field service deployment, now the talk has shifted to how such technology can be implemented.

Indeed, the busiest booth in the exhibitor zone at the conference was that of Swedish AR provider XM Reality who had a constant group of Service Directors around them as they demonstrated their approach to the tech.

Similarly at both Field Service Medical and Field Service USA earlier this year Help Lightning, whose own AR solution evolved from an emergency healthcare solution, were the darlings of both events.

And another AR provider that has seen the potential of AR in field service is Scope AR, a Californian based company that have been putting together an impressive list of clients and partners across the last five years which includes the likes of Phillips, Toyota and Boeing.

They’ve certainly built up a decent pedigree within the fledgling sector - much of which can be attributed to the bold view they hold that AR not just about enhancing existing communications channels but in fact a completely new form of user interface that can transform the way we utilise the ever growing computing power available to us on the move.

As CEO Scott Montgomerie explains “We’ve been doing augmented reality for quite a while, we started in the space about five years ago and when we started looking at use cases we realised that AR wasn’t just a new fun technology but a user interface that could allow people to interact with the real world in a way that had never been done before.”

When we started looking at use cases we realised that AR wasn’t just a new fun technology but a user interface that could allow people to interact with the real world in a way that had never been done before

“If you think about the way that people have been interacting with computers and all the benefits they bring such as analytics and the computations they can give us, we have been interacting with them ever since inception through a two dimensional screen - whether it be mobile, tablet or desktop. Augmented Reality really provides you with an opportunity to interact with the real world in the way that we have evolved - with our eyes and hands. Seeing things positioned in 3D just like we are meant to.”

Scott and his team were able to take this thinking and hone in very quickly on the areas of industry that could benefit from AR, including of course field service.

“We realised there is a whole segment of industry that hasn’t really had the ability to take advantage of the amazing power of the computers they use,” he continues.

“And that’s really where we see Augmented Reality providing huge benefits - by bringing massive computing power in a brand new user interface and allowing guys that use their eyes and their hands out in the field to take advantage of this new technology..”

In fact, the Scope AR story essentially begins with a piece of work they did for a client working within the mining and aggregates sector.

Having been approached to put together a step-by-step training solution that harnessed the power of AR, the project evolved quickly to a proof of concept that their client liked so much they decided to showcase it at MineExpo in Las Vegas.

What was supposed to be relatively low key three demonstrations a day across three days snowballed into being one of the event’s main highlights, with Montgomerie and his colleagues eventually giving 110 presentations with crowds of up to a 100 people at each.

This was the tipping point for Montgomerie where he realised the potential of the application and just how pervasive it could be.

“From there on we were like wow - I think we’ve found something!” He admits

Soon Scope AR were working with the likes of Boeing and Toyota building out more step-by -step training and maintenance procedures. A period which Montgomerie explains as “just being a service company trying to work out how best to use Augmented Reality.”

The work that was undertaken at this point was what ultimately led to the development of the WorkLink platform that Scope AR have recently launched.

And whilst the primary purpose for WorkLink is as a training tool, having watched the demonstration I immediately could also see an application for health and safety compliance also.

Many field service management solutions currently offer a check-list functionality in order to ensure workers are not only meeting compliance standards but working in a safe manner. Things like switch off the mains, then remove the cover plate, then remove screw ‘a’ for example - with each step only being revealed once the current one is completed.

Augmented Reality - how big an impact can it have in Field Service?

Through the WorkLink platform, each of these steps can be both visually demonstrated and also confirmed as completed via built-in analytics that capture checklist verifications and metadata such as how long it took to perform each step.

In short the content creation platform allows the rapid development of AR work instructions, allowing you to use converted 3D models, add animations, text, images, videos, check-lists, etc. in a branched workflow, to give intuitive, visual instructions.

Another useful benefit is that once ‘smart’ instructions are deployed into the field, the digital instructions automatically start collecting valuable data such as time per step, user and usage information, geo location, etc which can lead to greater insight to how your engineers are performing.

Part of the problem, which is faced by many AR providers not just Scope AR, is that computer vision right now is still fairly rudimentary and in the field you ideally want a system to recognise parts automatically.

However, whilst this WorkLink has clear applications as either a training or compliance tool it does have limitations in-terms of being a communication platform for connecting remote engineers to more experienced colleagues.

Scope AR like many similar tools relies primarily on fiducial markers and in the field these can get ripped or damage fairly easily.

So whilst Scope AR do offer their customers durable ‘beer coaster’ sized markers to help over come this issue, their alternative offering ‘Remote AR’ is a simpler use of AR technology that whilst still making use of markers, is more focussed on the use of annotations and drawings that can be added by the both users and which once added will stay fixed to relevant device section regardless of whether or not the video angle changes.

The solution also has a low band-width option which reduces the video feed to every 5 frames which again is a simple but highly effective and practical way of dealing with one of the other biggest challenges of AR - connectivity.

And as with WorkLink, Remote AR was also a solution that was born out of a real life working scenario.

“We were working with a client in Brazil who were facing a major problem where they were manufacturing equipment in Brazil and distributing it in Africa and they were having a lot of communications problems - the local Africans didn’t understand Portuguese and the Brazilians didn’t understand the African dialect, so they had to send people from Brazil over to Africa to fix the equipment at enormous cost,” explains Montgomerie.

“They told us was ‘what we really want to be able to do is get on a video call with these guys and be able to draw and annotate and show what they are doing an create instructions in real time.’ So that was the idea behind Remote AR - that the technician in the field could transmit by video what he is seeing to an expert and that expert can draw annotations in Augmented Reality and as the technician moves around those annotations can stick to what he is looking at”

“It is really a fairly basic use of Augmented Reality but it’s very useful, it’s very practical and it’s usable now.” He adds.

In fact to add further weight to this assertion Montgomerie is able to cite another of their clients, with a field force of 10,000 engineers who have been able to use Remote AR to reduce the amount of time spent on support calls within a contact centre by over half.

Despite case studies such as these coming to the fore, there still remains in many corners the perception that whilst AR is undoubtedly a technology with huge potential within field service, is still has some way to go in terms of maturity before it becomes pervasive.

However, despite case studies such as these coming to the fore, there still remains in many corners the perception that whilst AR is undoubtedly a technology with huge potential within field service, is still has some way to go in terms of maturity before it becomes pervasive.

Yet, Montgomerie believes that this is now starting to change.

“We were definitely around in the early days when AR was just starting to get accepted as a tool for the enterprise, but I think this year we have started to see an inflection point. We have now reached the point where people are aware of Augmented Reality and they want to use it, they now know it’s viable,” he comments.

Indeed, the technology has developed at a rapid pace across the last half a decade - although as further developments in hardware appear on the horizon there is a lot of potential for further refinement.

“As far as where the technology is, just within the last couple of years we’ve started to see cameras that are good enough to pick up those markers in challenging conditions and 3D cameras and Google’s Tango is really exciting for us . We are really looking forward to that getting into wider circulation”

“We really like the partnership that they did with Lenovo we expect to buy a whole load of those Lenovo phones (the Lenovo Phab 2 Pro which features depth sensing camera technology) for our customers. I actually expect that depth sensing technology to be in our all of our phones in the next few years - there is just too many cool things you can do with it to pass it up.”

“Apple acquired a company a while ago called PrimeSense, and then they acquired a software company last year called Metaio and with the two of those they are in an extremely good position to bring out a depth camera as well.”

“So I think that the hardware manufacturers are really going to solve that problem for us a. With depth tracking cameras a) we will be able to do marker tracking so much better and b) the level of interaction you can have between the two users is much, much better.”

Arguably the introduction of depth sensitive cameras as a commonplace inclusion in smartphones and tablets could just be the final breakthrough that AR needs to cement itself as a regular tool amongst field service organisations.

Indeed arguably the introduction of depth sensitive cameras as a commonplace inclusion in smartphones and tablets could just be the final breakthrough that AR needs to cement itself as a regular tool amongst field service organisations.

And Montgomerie thinks this will happen sooner rather than later.

“I think that we will see this in about two years,” he states.

“I believe the Lenovo phone is set to launch in September and there are a couple of other OEMs lining up there announcements as well. Then what we typically see with phone manufacturers is when a very advanced technology comes out within the next year many other manufacturers pick it up and then within the following year it becomes lower cost and quite pervasive. So within the next couple of years I think we will see this technology being on all phones.”

So does Montgomerie see his client base as early adopters?

“Its a little bit of both,” he explains. “We’ve definitely got a lot of companies that are forward thinking - they are actively looking for the latest and greatest technologies and they want to be on the leading edge, but we’ve also got some of our companies, and actually they are some of our best customers, that are not exactly forward thinking.”

“Some of them weren’t even sure what Augmented Reality was, they just contacted us after seeing our videos, and seeing the utility of Remote AR and they saw that they could communicate much better than they were capable of doing currently.”

“In that sense they weren’t looking for the technology but they actually saw that it solved a real problem for them and they were ready to adopt it. I’m not really sure that they even fully understood what Augmented Reality is - they just know this is a product that works.”

This is perhaps the greatest selling point for both Scope AR and Augmented Reality in general. As with any technology that eventually takes root and becomes part of our day-to-day operations it is the actual benefit, and the methodology of improving business processes that companies want to buy - rather than the technology itself.

Of course it is impossible to have any conversation about Augmented Reality today without inevitably touching on Pokemon Go.

For those that have just returned from Mars, Pokemon Go is a mobile gaming phenomenon that uses Augmented Reality to bring one of Nintendo’s best loved franchises into the Twenty First century. In fact it’s hard not to bump into someone that hasn’t at least heard of Pokemon Go such is the phenomenon - and it is putting AR directly into the hands of the man on the street.

But is that a good thing for AR in the enterprise? It certainly does give a clear, if somewhat simplistic understanding of what AR is to the average person.

But is there a danger that the power of AR as a genuine productivity tool within field service and wider enterprise could be dismissed as a gimmick if it is too heavily associated with a mobile game?

Given Montgomerie’s position as a leader within the sector I felt he was the perfect man to address this question.

“I think it’s good for AR in general in that it’s bringing awareness of the technology to people - at the very least I won’t have to explain what augmented reality is anymore!” He quips.

“I think its proving that AR is viable as a technology. Even though it is very simplistic use of it - I was playing with that type of use of AR five years ago - maybe longer. We do much more sophisticated applications of AR in the sense that aligning content precisely on top of equipment is much harder than putting a fuzzy animal at some unspecified location in front of you. But still I think it’s great for the industry and that can only be good for us.”

So whether it be depth sensitive camera’s, Google Tango or the ability to catch a Charmander while out doing your shopping, it seems it is just a matter of time before Augmented Reality seeps into the mainstream consciousness.

And given it’s potential to have a profoundly transformational effect on how we undertake field service operations, the likelihood is it will become a pervasive part of our industry soon enough as well.