The Digital Supply Chain podcast

Augmented Reality in Supply Chains - a Chat With Librestream VP Jon Newman

July 12, 2021 Tom Raftery / Jon Newman Season 1 Episode 147
The Digital Supply Chain podcast
Augmented Reality in Supply Chains - a Chat With Librestream VP Jon Newman
Show Notes Transcript

I've not done an episode of the podcast on the topic of Augmented Reality's usefulness in Supply Chains before now, so it was about time to put that right.

Librestream is one of the leading companies in the industrial AR space so I invited Jon Newman, their VP of Product Development to come on the podcast to talk about how AR can help alleviate skills shortages, improve worker safety, and the challenges for businesses when developing an AR strategy.

Jon mentioned the WBR Study at the end of the podcast as an interesting resource for folks to look into. You can check that out here.

We had an excellent conversation and, as is often the case, I learned loads, I hope you do too...

If you have any comments/suggestions or questions for the podcast - feel free to leave me a voice message over on my SpeakPipe page or just send it to me as a direct message on Twitter/LinkedIn. Audio messages will get played (unless you specifically ask me not to).

To learn more about how Industry 4.0 technologies can help your organisation read the 2020 global research study 'The Power of change from Industry 4.0 in manufacturing' (https://www.sap.com/cmp/dg/industry4-manufacturing/index.html)

And if you want to know more about any of SAP's Digital Supply Chain solutions, head on over to www.sap.com/digitalsupplychain, and if you liked this show, please don't forget to rate and/or review it. It makes a big difference to help new people discover it. Thanks.

And remember, stay healthy, stay safe, stay sane!

Jon Newman:

That's our vision around the AI connected expert where, you know, the worker will have some form of advanced AR device that they'll be wearing when they're when they're working. And just like with Iron Man, where Jarvis is kind of like you're the mentor or the expert that's coming along for the ride. That's sort of where we see in terms of the workers experience in the workforce experience. That's kind of where we see this ultimately going.

Tom Raftery:

Good morning, good afternoon, or good evening, wherever you are in the world. This is the digital supply chain podcast, the number one podcast focusing on the digitization of supply chain. And I'm your host, global vice president of SAP. Tom Raftery. Hey, everyone. Welcome to the digital supply chain podcast. My name is Tom Raftery with SAP and with me on the show today I have my special guest, john. JOHN, would you like to introduce yourself?

Jon Newman:

Hi, Tom. Thanks for having me. Hi, folks. My name is Jon Newman. I'm the VP Product Management at leadership technologies responsible for product strategy, roadmap and execution. Librestream is a company that is the leader in remote experts solutions globally. And so we we sell to the largest the largest defense oil and gas test inspection companies around the world in order to allow for their remote inspection and remote expert capabilities.

Tom Raftery:

Okay, so for people who might be unaware, Jon, what's remote expert,

Jon Newman:

so remote expert is the ability for companies that that have work workforce in locations that may be more challenging and more difficult to, to communicate it. So sort of non office environments. So it could be things like military bases, oil rigs, ports, remote farms, and things like that. And it's allowing people out in these remote locations to be able to talk to people in other facilities may be in head office, or other parts of the supply chain, to be able to communicate with them to get guidance, and expert advice in terms of what they may be able to do or what they want to do, or to perform things like inspections and have supervision or to be able to for maintenance tasks, and have more senior folks kind of watching and talking them a lot. So it's allowing for these more industrial, industrial use cases, and allowing them to happen in challenging environments where maybe bandwidth is not as strong or maybe there's other security or or more, more, more, more harsher networking needs.

Tom Raftery:

Okay, so my health insurance company this year has rolled out a new facility on their app on the phone where I can get a tele consultation with a doctor, just press a button and bang, up comes the doctor on screen and I can have a consultation with them is at its most basic level. That's the kind of thing you're talking about.

Jon Newman:

Yes, exactly. So the medical consultation is is definitely one usage of this, you know, remote expert capability. And that's something that we do do. But our sort of particular focus and our particular leadership expertise is in these more challenging environments. So defense companies talking to, you know, militaries around the world in remote locations, or oil and gas folks communicating with their employers on rigs, and inspection folks like that. So it's more the rugged environments that we specialize in. And we're the leaders in but yes, the medical concept is, is one example of a remote expert use case. Okay, and,

Tom Raftery:

I mean, I can see how technology is enabling this because we get devices which have better connectivity, better processors, better cameras, etc. So, yeah, the technology is improved to allow this to happen. But what about the demand? What's the demand for it like?

Jon Newman:

So the demand for the remote expert capability we're seeing just increasing over time, I mean, labor stream has been doing this and being the leader for the last 15 plus years and invented the remote expert category. And so we've seen the the drive for this increase, more and more, especially as you know, customers in all the verticals that we that we sell to go along their digital transformation journey and look to help to digitize and make more efficient how their workforces interact and engage in so we've just seen an acceleration of this and especially actually in the last year or so, with the global pandemic and people not being able to get on a plane to do a you know, an inspection of you know, of a salmon farm in in Norway, where their supply chain is or you know, inspecting cargo in in a port in Southeast Asia, or find getting a plan to fix military or aircraft installations in the Middle East. So the restriction on travel has only accelerate Did this this digital transformation, and also the focus on sort of the carbon footprints and how travel impacts that that's also something that folks are way more sensitive to, as well. So all these things, in addition to safety, and other aspects that have always been true, are really helping drive this, this digital transformation within these workforces. And you're driving the remote expert use case as well.

Tom Raftery:

Okay. And I can imagine as well, for example, things like the scaling of knowledge being something as well, that would drive it because no one experts sitting at their desk at HQ, for example, can suddenly be interacting with people in multiple locations at the same time, rather than having to, to your point to get on a plane, go there, spend a few days there, come back, get on another plane, go to the next place, you know, you know, it seems a far more efficient use of knowledge and resources.

Jon Newman:

Absolutely. So I mean, the example you talk about where it's more efficient for the human expert that may be interacting with people remotely in these foreign locales, that's definitely one aspect of this. And then also, as we as we look forward to where this is going and how this does converge with advanced technologies, such as artificial intelligence, and augmented reality. And then also with this with the global need of the older workforce aging out, and and, you know, not having access to these human experts, experts given given the sort of global worker shortage that's projected to be 85 million by by 2030. So that that broader trend is driving a need to, okay, we're connecting, you know, human experts with human workers today. But what we try to do now, because of this, this, this, this experts shortage that's happening is trying to start to capture this expertise that's being shared within the onsite platform, cap, capture it, and then analyze it and using advanced technologies, like artificial intelligence, be able to then surface that in a timely fashion automatically to these workers. So they require less and less interaction with a human experts. And we can more and more rely on providing them with this expertise that we've captured through through the onsite platform and then being able to surface it to the worker in ways that makes sense using augmented reality, to allow them to put this expertise that that's being captured, to be able to put it in front of their their field of view, and be able to allow them to have access to what they need and all the data and insights they need. As they going about the task, be it like a maintenance standard maintenance task or repair, or some sort of inspection and be able to do this in a safe way to allow you know, the use of AR wearable so that they can be hands free. Or they can be interacting via voice so that they can still use the tools they need still have access to that that timely information, not necessarily using a human expert.

Tom Raftery:

Okay. So just to clarify there, because you, you give me a very big answer to a very small question, I just want to try and break it out a little bit. So the the shortage of expertise I gather is happening, because a lot of experts are hitting retirement age, is that correct? And they're retiring out of the system? Or is it? Or is there some other reason for the lack of expertise? Or is there a greater requirement for expertise that just isn't bubbling up enough for how does that break out?

Jon Newman:

Yeah, absolutely. So I mean, I think I mentioned that the, the, the global worker shortage is projected to be about 85 million by 2030. And the statistics we're seeing as the baby boomers are retiring from all sectors at a rate of about 10,000 a day, in the US alone, for example. And so what you're seeing is that that generation that typically tended to stay at a job for 2030 years, which developed a vast amount of expertise and something that dramatically aging out of the workforce and the, and the people that are coming in that need to do the same jobs and fill their shoes, they don't have that same same Kenya, and years of experience under their belts. So that is huge skills shortfall, that's not going to be met by you know, experienced workers coming in, and that's where we see a huge opportunity for for technology to pick up some of this, this, this knowledge gap.

Tom Raftery:

Okay, and, and you refer to them, the two technologies, the you refer to AI and AR and if I understood it correctly, what you're doing with the AI is you're taking a lot of the knowledge that has been gleaned over the years from the experts, and you have that in some kind of back end system. And then when people come in with queries, the queries go through the AI the AI contest or can take from that kind of knowledge base at the back end and then suggest answers is that more or less what it is?

Jon Newman:

Yeah, absolutely. So at a high level what we're doing is because I'm so today is the conduit for these remote expert sessions and and bringing in as well, additional information from IoT sources, for example, or other sensory information, or even pulling in your training material and manuals from sources that all these companies have, because we have that central point today, and we're bringing in, then we have the ability to sort of analyze and collate this using artificial intelligence to extract the kind of the meaningful aspects from all these different disparate sources. And then, you know, using augmented reality devices or other devices to your phone or tablet and the like, we are able to sort of understand the context of where the worker is, and understand the context of what they're in front of them what they're doing. And we can then again, use artificial intelligence to figure out what we should be surfacing for that worker at that given time. So being able to recognize what equipment is standing in front of being able to pull in all historical information about that piece of equipment, be able to see what task they're doing, and potentially offer up steps to work through or, or just in time training information that may require and using the augmented reality set of things, to do this in a way that's intuitive and safe for the worker, because there'll be typically in difficult or dangerous situations where they'll have like, in that field of view, there'll be a lot of machinery that have one or two hands that they they'll need to use the tooling or holding onto something and the like. So augmented reality is the way we serve this information in a way that's intuitive for the worker, and AI is how we extract and figure out what to serve that worker.

Tom Raftery:

Okay, very good. Now, my daily interaction with AR happens when I get into my car, because my car has a heads up display in front of it, which I absolutely love, it is so awesome. It's one of my favorite things about a car, about my car, but it's a very limited set of data, you know, gives me navigation gives me speed, road signs, all that kind of thing. And that's by design, because you don't want to be cluttering up to your point, you know, with information that, you know, doesn't need to be there. In terms of the kind of AR that you're feeding to people who were using your system. You mentioned a number of devices, and different information sets that are being served up. First of all, I guess in terms of the devices, is it a customized device? Or it cannot be any device? Is it up to the customer to do to provide the device? Is it kind of Microsoft HoloLens devices? Is it iPads? Is it, you know, what, what's the hardware component? And and then how how do you decide what's the appropriate information to display at any point?

Jon Newman:

You know, that's? That's a great question. So what we find with our customers is they're not looking for one particular device, they're looking for a portfolio of devices to support these experiences on because they will provide different workers different tools in order to do their job. And so we work on the widest range of devices. So as we're seeing everything from you know, iPhone, Android phone, all the way up to the more advanced our augmented augmented reality, wearables and the high end of the moment being the Microsoft HoloLens application, so what we find is a customer will choose to have a portfolio of devices that they'll want to use on site on from the from from that selection. And it's, you know, it's imperative for our customers that we provide the same consistent on site experience across a broad range of these devices. So they can pick and choose also have access to that same shared information. So no matter what device they log into, they will have access to all that answered information. And then what we do is we obviously tailor the insights experience. So that's familiar across everything. But to maximize the the peak of platform it's on. So what you'll see on say, a phone will be more limited compared with what you'd see on a on an advanced AR platform like the HoloLens two, which would be a much more advanced, much more immersive AR AR platform. And you know, we see the HoloLens two is a great example of of showcasing where we see this going. And the example we always, always talk about you mentioned about your car, you know, we we always look to, you know, the Iron Man heads up display an Iron Man. And we see, I mean, that's kind of the, you know, we can talk about all the technologies, but just to visualize in people's minds. That's kind of essentially where we see this all going is, is that and that's our vision around the AI connected expert, where, you know, the worker will have some form of advanced AR device that they'll be wearing when they're when they're working. And, you know, just like with Iron Man, where Jarvis is kind of like you're the mentor or the expert that's coming along for the ride. That's sort of where we see in terms of the workers experience in the workforce experience. That's kind of where we see this ultimately going. So it's, it's always helpful we find to give give that image in people's heads of what we think the worker will ultimately have. And then it's a you know, it's a, it's a journey to want to collect all that knowledge on the back end and be able to do that in a way that's more automatic. And it's not a huge human endeavor. And then to serve it in a way that's, that's contextual, and, and timely for the workers trying to do

Tom Raftery:

interesting, I get this kind of picture in my mind's eye, not not just of Jarvis. But then I also think, if I, if I think back a number of centuries, to the idea of a master and an apprentice, and an apprentice working away with a master looking over his or her shoulder, typically his because I'm talking a couple of centuries ago, but you know, what I mean, is that kind of another way of putting it is that a good analogy as well?

Jon Newman:

Yeah, that's, that's actually an excellent one. I mean, we always, we always use the analogy of the old time expert, like Old Bill, you have a 30/40 year veteran. And when we talk to our customers, this, every customer has an old bill where they can go in, they can go in and basically just listen to the sound of an engine or machine or just put their hand on it, I'm feeling that the vibration, and their expertise has gone to such a level that they can just do that. And all of our customers have someone like that. And so it's exactly that it's trying to capture your Old Bills, wisdom and expertise, digitize it, or lives forever. And then you analogy, use of the master with the apprentice, having the virtual master that's basically, you know, an old bill that, you know, without wisdom lives forever, and can then continue to mentor, you know, younger and less experienced workers indefinitely. So that's, that's kind of the, the end game, and that how we see this evolving, and obviously, we're taking steps on the knowledge capture side. And that's what the onsite Knowledge Network vision is around. And then on the sharing with the worker side, the AI connected expert vision,

Tom Raftery:

okay, and how long how far away from our from that vision are we do you think?

Jon Newman:

So I mean, we we also, I mean, about a year and a half ago, is when we put out our AI connected expert vision to kind of showcase, you know, the, the Iron Man scenario, if you will. And so, we, in terms of executing that vision, then we have sort of taken steps to add the additional third initial foundation for that. So some of the key foundational tenants that we've added to onsite to do that, you know, one is computer vision, so we can kind of see what the worker is seeing and identify that the other part is around natural language processing. So we can hear and communicate with the worker and start to capture that expertise. The other part is around tying in with IoT or digital twin systems that our customers have. So instead of bringing in that raw data and visualize that, and then the last key pieces around the advanced augmented reality, so providing support for the HoloLens, two and the other ones that come out, so you can have that true heads up display AR experience. And so those initial building blocks, we actually have in the onsite platform at the moment, and we put them in as foundational pillars. And then what we are doing is as we as we progress along this multi, multi year journey around, you know, the Knowledge Network and the AI connected experts, then we are building capabilities that solve specific customer business needs and challenges now, but but on a journey to getting to this ultimate state. So we think, you know, over the coming years, we will get start to get closer and closer to the vision that I that I outlined. And you know, probably in 510 years, we'll be getting close to that, that Iron Man experience that that has all the information automatically collected, analyzed, and can be surfaced to the workforce,

Tom Raftery:

Super, super and, I mean, you've mentioned customers a few times, have you need good customer examples you can talk to

Jon Newman:

the customers that we find are really pushing the envelope in terms of, you know, some of the advanced augmented reality and some of the advanced artificial intelligence usage, we find, it's a lot of the aerospace and defense customers that are really pushing the envelope here and which which totally makes sense, because even some of these specialist some of these new devices are quite expensive. And some of these technologies are expensive to implement as well. But for the defense and aerospace folks, obviously, their products that they're working on are like aircraft missiles, you know, spaceships where the cost of an error can be had to be a millions or 10s of millions of dollars, so

Tom Raftery:

and lives need

Jon Newman:

lives as well. Exactly. So we find those customers tend to be on the leading edge of these things just because the products are working on are a lot more valuable. And therefore they want to invest as much as they can to ensure mistakes on is made sure that their workforce, which needs to be highly skilled to work on these things has access to everything that they need. And so we find these folks tend to be at the at the cutting edge of the adoption of this advanced technology and investing in these advanced use cases.

Tom Raftery:

Nice nice Jon, we're coming to the end of the podcast. Now, is there any thing that I've not asked that you wish I had? Or is there any topic we've not touched on that you think it's important for people to be aware of?

Jon Newman:

Ah no, I think I think that covers about everything, Tom, I mean, it's as you can tell, it's an exciting time, the transformation that the workforce is going through a large and the challenges they have, and it's a it's a perfect intersection of their tremendous opportunity from the business standpoint, and the maturity of some of these key technologies in augmented reality and artificial intelligence to really solve some of these, these large challenges that are some of the big the big problems that we're facing today with the workforce.

Tom Raftery:

Okay, great. If people want to know more about yourself, Jon, or about Librestream or about any of the topics we discussed today, where should I direct them?

Jon Newman:

So you can find us on LinkedIn, Twitter, Facebook, and YouTube. And also, we have access to a WBR report that we have around AR and the field ser ice industry that also tou hes on more of these themes as ell. And I believe your lis eners can access the full rep rt via the link next to this pod ast.

Tom Raftery:

Cool. I'll include that link in the show notes, so, yeah. So people have access to it. Great, Jon. That's been antastic. Thanks. A million for oming on the podcast today.

Jon Newman:

Great. Thanks for having me, Tom.

Tom Raftery:

Okay, we've come to the end of the show. Thanks, everyone for listening. If you'd like to know more about digital supply chains, head on over to sa p.com slash digital supply chain or, or simply drop me an email to Tom Raftery of sap.com. If you'd like to show, please don't forget to subscribe to it and your podcast application of choice to get new episodes as soon as they're published. Also, please don't forget to rate and review the podcast. It really does help new people to find the show. Thanks. Catch you all next time.