As promised last week, in an op-ed write-around of the same conversation, here is the full interview with Phil Skipper, group head of IoT business development at Vodafone, on the subject of industrial 5G performance KPIs and management SLAs – and the role of mobile operators in the developing private 5G market.
This conversation will form part of the narrative in the forthcoming Enterprise IoT Insights report on industrial 5G SLAs, out at the start of April (to be available here). A webinar on the same subject is available here, with panellists from ABI Research, EXFO, Metaswitch (Microsoft), VoltDB and Zeetta Networks. All the responses, below, are from Phil Skipper.
…
How does a network provider, of whatever sort, guarantee the private network they are putting into a factory, say, will not go ‘pop’, and the production line attached to it will not go down? How is that negotiated between the enterprise and the service provider? Clearly, this is ‘early days’, and those conversations are still being worked out. But how will that be defined in SLAs?
So it all starts with this idea that IoT is moving from IT to OT, and going from being a technology to being an operational asset. Which changes the way you think about it. Where we have previously connected the things people have made and sold in the market, we are now monitoring the assets used to make them. That is quite a change, because you have to look at the network infrastructure suddenly as part of an OT platform, which is quite different. As soon as you get your head around the fact you’re moving from IT to OT, [the negotiation] moves from how you sell and build to how you run [a network].
The question is how that run-process works. And especially with private networks, you cross between what the telco provides and what the enterprise owns. These manufacturing companies have maintenance teams on the shop floor for when things go wrong. They have people trained up, ready to go, if a piece of equipment goes down; they don’t call the manufacturer of the plant about it. And this hybrid model will gain traction [with private LTE / 5G] – where, rather than bringing someone in to fix issues, the relevant skills exist in-house, as well.
And with that, the whole concept of SLAs changes. It becomes about faultless resolution. I don’t think any customer assumes something is going to be 100 percent reliable. What they want to know is that, if it doesn’t work, they have a pathway to a faultless resolution – that they can go from down to up in the shortest time. Which is where you have this more interesting relationship between the factory owner and the network supplier, whoever that may be, within that closed-box factory.
So factories will have some resident expertise to manage the 5G network, in the same way they manage their Wi-Fi networks today. Presumably, where Vodafone provides additional support, it does not intend to put somebody on site, in every factory? Presumably you are running remotely, as a managed service provider, and can tap into that network to run fault diagnosis, and this kind of stuff? Just talk through the mechanics of that.
Yes, so we offer mobile private networks, to all intents and purposes, as a managed service, and we will monitor the performance of that infrastructure. One of the things you will see is greater granularity of what is actually viewed. Within the factory, you may go between licensed and unlicensed spectrum. These networks will typically include a bit of Wi-Fi, a bit of Bluetooth, and some LTE and 5G as well. We will monitor this infrastructure to understand where the core network services are becoming less ‘performant’, and to make adjustments to maintain performance and avoid problems in the first place.
Is that done from a conventional network operations centre (NOC), from where you also monitor the national network?
It depends how that network is set up. In an open, national network, we have access to everything, clearly. As a managed service, we would need access to the private network, as well. And it is important to understand that even a private network will typically have interconnect points with the public network. But the network will be optimised for very different things, around performance and flexibility, and not necessarily around the kind of availability or bandwidth [metrics] you will find in the public network space. It will require very different monitoring.
This is where the SLA question becomes quite interesting. Because if you are working in the open space, you basically want many customers to connect in the same way and receive the same quality of service, in terms of the bandwidth, media streaming, and everything else. So the axis that varies most is density. When you get into a factory setup, the density is fairly constant – just because you only have a fixed number of machines.
Instead, you have to monitor the performance axis, for things like virtual reality and machine control, and so on, which will change a little from one space to another. The second thing is redundancy at hardware level, to ensure fallback communication from the start. Between those things, the piece in the middle is about how to put feet on the ground if something needs to change.
So does this scenario depend on whether everything is on the edge, or whether there are interfaces with the public network? Because different enterprises want different things. A paranoid car maker or high-security defence manufacturer does not want to interface with the public network at all. Does that preclude you from running remote diagnostics?
No. Being able to run diagnostics will be essential. Monitoring the network is different to accessing the data on the network. It’s sort of like health records – they’re protected, but you don’t want to stop doctors from looking at them. It will be the same kind of thing. You will have these security domains [with restricted access]. But to maintain a network, you’re only interested in the flow of data, and not the data itself. So nothing will escape. But, like we said, the network is an asset, so it has to be maintained – and [cellular] will replace existing cable and Wi-Fi systems, which are being actively managed today.
So should we disabuse ourselves, then, of this idea which gets talked about, around whether enterprises will even allow carriers near the controls? About this perceived jeopardy for the old telecoms market; that it’s not really about the carrier on site, but about managed services and diagnostics? Is the term ‘network operator’ more fluid in this environment, because you are not really operating the network anymore. The enterprise is doing that, in the main part, and you are only providing second and third-line assurance from afar? Is that what we are looking at?
Well, actually, I think ‘network operator’ is a good name for it. Because we operate the network, and keep the network running. We make sure this magic radio-airwave stuff is working. Which is quite different from the use cases going over the top. We are not operating the factory. We are only providing a piece of infrastructure to help to run it. We are more like the electric network, or the gas network, or the sewage network – all of which are operated by third parties, and help the factory to work. It is wrong to think a telco would have any involvement in private LTE or 5G from the OT perspective. We are not interested in what’s flowing across the network. We are only interested in making sure the network is available.
Is it very different from running a Wi-Fi network? Because the market says that, in the end, this becomes – in a single site, with a simple core – not that complicated. The radio network becomes an open system, somewhere down the line. And it is not that hard; it is in a box, attached to the server, like a Wi-Fi network. Is that what happens? And therefore, for you, it’s a hands-off thing, and the network planning is key. And perhaps the key SLAs are about planning and deployment, rather than performance?
If you look at the traditional sell-build-run model, there are different SLAs for different bits. For the build-phase, it comes down to how long it takes to go from nothing to a working system, and there are traditional project based SLAs for that. It gets more interesting with the run-phase, because that is where you define this faultless resolution. Because you can’t afford the old finger-pointing, where it is because of this, or because of that. You need a mechanism to find a faultless resolution.
And, yes, there is this idea of a network-in-a-box, in its simplest form, where you can go in there and connect up, and dah-dah. But even with that, there will be a lot of bespoke configuration and reconfiguration, and resilience and redundancy to build in – all of which will also need managing. And the enterprise will have devices connecting and disconnecting to and from the network, and being swapped-out.
All of that needs to be handled – and it has to be handled from the point of view of both the network infrastructure and the business running on top of it. And if a file from one machine does not arrive in the manufacturing execution system (MES), the line of questioning goes: was the network running, was the file generated, or was it lost in the system? So your ability to fault-find is the real key to success for these private networks. And that involves multiple payers in the OT space.
So does that involve more than the enterprise and the carrier? Does that involve a system integrator, as well, whether a major multi-international like Accenture of Infosys, or a local specialist like we are coming across increasingly in the German market, for example?
It will be the guys running the factory, in most cases. If you have a faulty PTT radio on site, for example, you check-in with maintenance and get a replacement. That’s how it works. And if you press the button and it still doesn’t work, you run the diagnostics of the radio network itself, and work through and find out that, actually, it is because a transponder over here has gone down, or whatever.
So the diagnostics process is pretty established in factories, already. The difference now, with LTE and 5G, is these new technologies are unfamiliar. That is the key balance to strike, to get the shop floor up-to-speed, so the network operations team and the boots-on-the-ground team are on the same page.
So how does that change, between working with a tier-one manufacturer – which the market expects to do this on its own, to take greater charge of its networking estate – and working with a SME, which doesn’t probably have the resources, or interest, to run it themselves? What is the difference in Vodafone’s approach?
We will offer the same type of managed service. The difference is the SME, most likely, does not have capacity to create a department of network specialists. They will want that full end-to-end service. But those implementations will be quite small, typically, and self-contained. When you go to a large car plant, by contrast, there may be multiple cells, and the car maker will probably choose to invest more in its own networking capabilities.
The other thing is these networks seldom go down in their entirety. What happens is you might lose Building 17, or data traffic from a particular machine. So even though we can diagnose an issue in the network, you still need someone on the ground to have a look at it. Which means this idea about going from working to not-working is somewhat wrong; what you tend to find in a factory network is that you go from 100-percent of everything to 100-percent of most things.
Is this idea of ‘five-nines’, or six or seven or eight-nines, reliability written into the technology or into the network management SLA? Because even ultra-reliable 5G will go wrong, right? And if enterprises don’t want that, then they need to build a second network in a separate frequency, or with a different technology. Is that how to guarantee a factory network, in the end?
Yes. We have national roaming, which means if one network goes down, we can bounce to a second network. Now theoretically, if both networks are 51 percent reliable, say, then you probably won’t have a problem. Because you bounce from one that isn’t working to one that is, and back again. That’s where you also need to think about fallback comms for critical bits of machinery, and so on.
But if you really want the ultimate, then you need to think about what you want to connect, how you want to connect, and how to support all of that – so devices, network, support. If you consider only one of those three things, then you end up being limited by your 99.999-whatever percent. If you target two out of the three, you make a significant improvement; if you target all of them, then that is how you do it.
If you look at systems with ultra-high reliability and quality, they normally tackle it on those three axes – on a base-layer of infrastructure reliability, and then on redundancy, and then on resilience. Once you have that three-dimensional space – and aircraft are a fantastic example of this – then you can start to get that reliability up. By putting all three together – by deploying really reliable kit, by backing it up, and by establishing ways to work around it – you start to build [that ultra-reliability].
But presumably that, in the end, is not in the managed services contract, but in the deployment contract and the sale – in the sell- and build- phases, rather than in the run phase.?
Yes. But as part of the managed service, there is an opportunity to talk to customers about what they want from their OT systems. And most say they want 100 percent reliability. So you go backwards from there, and say, ‘Well, actually, you can’t get 100 percent on everything’. So the thing to do is to reduce exposure. And, together, you start to look at what can be done at the device level, what can be done at the operational level, and what can be done at the tech level in order to get you there.
Just going back to the question of remote diagnostics, will Vodafone set up an regional enterprise NOC in industrial heartlands in the UK and Germany, say – within a 20 minute drive if they ever do need you on site?
We [are managing] mission critical [network] services today, already. And in some cases we even have people working on site. But I think the model will be different. For us, we are going to need a NOC. And there is a good parallel for this in what we do with stolen vehicle tracking, where we have 52 secure operating centres across the world, already, to pick up stolen vehicles. But it is a good question: whether to have a dedicated enterprise NOC for mobile private networks? I don’t know. But you can imagine we will have something like that.
And just going back, again, to this idea of simplified and automated private cellular networks, running some manner of AI: do you get to the stage, at some point, where enterprises don’t need a managed service provider? Because it is all automated, and intuitive? If it’s off-the-shelf and self optimising, and running in ‘vertical’ spectrum, then why does anyone need an operator? Surely, it gets to the point where you have designed and sold the network, and the enterprise has a resilient three-axis network, with the best technology and the best back-up. So what else is there to manage?
But mobile networks are not static. They continually change, as devices are added and updated, and more routing paths are designed, and as the network interacts with the public network. So again, it depends on the customer. Enterprises that are massively invested in their own capabilities may well take on more of that service, and SMEs may just want the capability to buy more of those services. It becomes like horses for courses, as to the level of engagement each customer needs and desires.
What about this idea that the discipline to manage one network for millions of people is different to managing hundreds or thousands of networks for hundreds or thousands of different enterprises? That it requires, actually, a specialist with vertical knowhow, and not a traditional mobile operator? Is that fair, or would you take the line that it is a mobile network at the end of the day, both ways, and you know about that better than anyone else?
Yes, and that is the point. System integrators handle the build, which [covers] a distinct period of time. We are talking about an operational asset that might last for 10-15 years. So you need somebody that does networks as a business, which is what we do. So the network operator is well placed to maintain these networks going forward.
So the system integrator would not typically be engaged in the ongoing managed service of that network?
No. They fill in the middle, between a big consultancy and a network operator, if we put it in simple terms. But the underlying network is a core management issue. And this ecosystem will develop in support of it, because you need to very rapidly decompose that value chain when there is a problem, and decide where the issue is.
Are you frustrated with the way this is written about? Do you feel that, actually, the market doesn’t understand there are more layers in the delivery and management of this – or there will be more layers? And that it gets painted as a winner-takes-all scenario. Because I have just got off the phone with an analyst, saying carriers talk about ‘enterprise 5G’ like it’s a telco service, where enterprises want an enterprise service, and that there’s a fundamental language barrier. Presumably carriers might respond that they are used to talking about it one way, and that enterprises might want to talk about it another way, but the discussion is about the same thing in the end? And that when you get into the bones of it, there is common ground? Is that a fair summary?
Yes, that is the point for us. We want to talk about the run-phase, and the in-life service. If you’re just selling networks, you talk about technology and hardware and all the rest of it. But if you’re going to sell a managed service, you need to understand what the run is. And mobile private networks, in particular, are seen as a hardware thing. We look at it differently. We look at it as a managed service. Because for the customer, the most critical thing is the factory keeps running. And so when we talk about the run, we are very aligned – and, of course, if you know what’s required for the run, then you are in the lovely position, as well, to know what to build and what to sell. And then it becomes easy.
But Vodafone is something of an exception, because we combine two things: one is a very strong enterprise mindset, and the other is very clear IoT capability. So we do talk the language already. We aren’t just talking about the ins-and-outs of 3GPP standards; we are talking about manufacturing execution, real-time performance, quality-of-service, how to put AGVs on the shop floor, and all those types of things.
As an aside, how should we consider network slicing for enterprises? It seems slicing was all anyone was talking about a couple of years ago, and then it went away, and now it has started to return. It seems there is a separate opportunity for operators with slicing – which is not about selling a production line network to a factory, on a slice, but about a piece of the network for train services, or for outdoor industrial use cases, or for stadiums. Is that how Vodafone perceives slicing – as a separate and complementary thing to mobile private networks?
Yes. But the immediate opportunity is with real-time connected factories and Industry 4.0, and private networks. Because operators are still deploying national 5G, even as we are accelerating that rollout right now. What we’re seeing is these mobile private networks are being linked to [public] LTE networks so you can see goods flowing in and out. As [national] 5G deployments extend out of cities, and connect cities, and you start to get the trunk interconnects on 5G, you will start to see the conversation around slicing.
Because slicing requires a more ubiquitous access to networks. And that is happening. We are rolling lots of 5G out, as are many other operators. Which is why [talk about slicing] is coming back. These are new things in our strategy: private networks are about providing services that do not use our public network, and slicing is about creating quality-of-service channels on the public network.
