Build your Industrial Data Platform (2026 edition - 20 min free crash course)

Hello, I'm David and in this ITOT Insider video, you learn to understand and work with the second version of our industrial data platform capability map. So in 2024, we published our first capability map in an attempt to structure the, sorry to say so, mess or unclarity fogginess in the industrial data ops world. Vendors promising everything to you, people who just... want to buy a unified namespace, others they don't have a clue at all. So all that input, all that questions and all that unclarity led us to develop our first capability map. Now then came 2025 and in the spring of 25, we first did a podcast series leading up to Hannover Messe where we talked to vendors, 12 vendors on their point of view. when it comes to their questions, their ideas, their solutions, when it comes to industrial data ops. And that inputs, together with a lot of conversations that I had with end users and my own experience, then led us to publish Capability Map version two later that year. But I haven't recorded the video yet. So here we are. And in this video, I'll guide you through the second map step by step. So to understand why a data platform is so important, you need to understand the concept of scaling. What happens most often is that, and on this graph you'll see time on the horizontal axis and successful implemented data use cases on the vertical axis. So what we see most often is a scenario which looks like this. So we start, we have a first few data ideas, you know, We connect to an extra historian, we add another database, maybe some PowerShell scripts, maybe some FTP transfers. Maybe we install an open source tool, maybe we have a good historian, et cetera. But with time progressing with more data use cases being added, oh, and there's your process engineer and there's your data scientist asking for something else, et cetera, et cetera. Typically, these data initiatives, they stall. Why? Because you're... building quick fix after quick fix after quick fix, the thing becomes a monster and it's not able to scale any longer. Now, what we want to achieve obviously is something like this, where you get these exponential effects where all of a sudden, especially in the later stage, you have a platform which seems to be growing by itself. Users can do whatever they want. new ideas pop up in a Jiffy. Now, notice the fact that it does require some initial investment in the early phases. And this is obviously what we're going to talk about further on in this video. Now, the upwards trending curve, the exponential curve, is the one you would like to be on. That is the one which you can be on if you have a proper industrial data platform in place. All right, so how does this data platform looks like? We need to first explain you these first two things. You have data sources on the left, that only makes sense. So can be different types of data sources. Most often you'll see your traditional PLC SCADA DCS data flowing through an historian or direct. More and more also clouds, IOT data. But that's not all. Obviously, you have your databases, your MES systems, your maintenance systems, your engineering data, et cetera, et So those are all data sources. On the left side, here in yellow. On the green side, or in the green cloud on the right side, you have your data consumers, people, applications who want to use that data. That can be different ways and forms. In this case, we are talking about the non-AI and the AI consumers. I think most important thing here is that you have users and applications sitting on the right side needing that data. it in Excel? Is it in a data lake somewhere? Is it to build fancy AI models, which later on will be deployed on the edge? Is it to do some just on standard trending dashboarding? Is it to share data with other parties, et cetera, et cetera? It doesn't really matter for us, but the data needs to be available to them. So to do that, we obviously need something in the middle, an industrial data platform. Now, suppose that we would just be dumping all that data in the data platform, then we have built a data lake, which typically very fast becomes a data swap. Why is this a data lake? Because if we would just take data from the data sources and dump it somehow, push it or pull it into this data platform, Then the only thing we've is we've taken the data from one position or one place and placed it into another place. So the only thing we solved at this point is data accessibility. But the big problem of data projects, and we talked about it in an earlier article on scaling, is that it's not so much about finding the data. Yes, that's problematic as well. But it's also about integrating several data sources, cleaning them. aligning them, contextualizing them, and then giving that data to the data users. Because those steps, the finding, integration, cleaning, aligning, et cetera, et cetera, that's really the hard work. If that's all done, then the simple work is just using the data to make clever, to come up with clever ideas, optimization ideas, et cetera. All right, so that means that at a minimum, and this is If you follow our blog, this is the picture you probably know, we use it quite a lot. What we show here is that on the left side, when data flows in, it needs to be contextualized. And that's really, really important. What is context? Context for me is what links raw data, raw data points to the physical world. You can say context is what makes just plain data, it transforms it. into information. Context comes in different shapes and forms. The most known one, I would say, especially, and we'll talk about it later in the video as well, especially when it comes to unified namespace, is the assets context, this one. So the asset context where we typically use something like ISA95 to structure our data based on... Yeah, your site, plant, equipment, et cetera, et cetera. So the ISA 95 hierarchical model is something which typically used here. However, then it becomes there is more to ISA 95 than just the hierarchical model. But that's going a bit too far for now. Your production context is also really important. This is typically data which comes in from your MES system or sometimes your ERP system. It's typically to say, what are we producing? Which product, which recipe? Which is especially relevant when you're working in the batch or discrete industry. Maintenance context is another one which I didn't mention here, but is equally important as well. For example, to calculate a interesting or a true OEE number, you also need to know when was the plant running, when was it down, was it expected, unexpected, et cetera, et cetera. So this is the left side of the platform. Then we have the right side. This is where we're gonna extract the data to send it to the data consumers or the data consumers. get that data. So that obviously means that you need your APIs, your SDKs, you need some building connectors, maybe even some simple build in trending, MPI tooling, because for most cases, you just want to check a couple of simple data points. And more and more, we also see MCP as a protocol coming, especially when being used by LLMs and large language models. So that is the concept of an industrial data platform that doesn't give you a solution yet. If you want to build an industrial data platform, you now need to take another step. And that means stepping into our seven core capabilities. Those are the basic building blocks you need to build such a platform, to buy such a platform. And it might be the case that you have one vendor, you might have multiple vendors working together. So our first capability is obviously connectivity. And I think that's really important. Everybody in industrial world understands that you need to be able to talk to both modern systems as well as legacy systems. So it's typically not enough to only speak OPC UA or to only speak a certain API, REST API or whatever. You need really a variety of legacy protocols in order to talk to those PLCs. They and there, right? We're not gonna change the PLC for your data platform. So that's connectivity and it's fairly easy to find vendors which provides a set of connectivity options on the internet or also via our website. But I would say the first really, really, really interesting thing comes with a second capability that's transform contextualize instruction. So instead of just, I would say, almost dumping the data into the data platform, we want to add that contextual information to it, right? The metadata. We wanna add the asset structure. We wanna add information about your productivity, your quality data, et cetera, et cetera. So preparing that data, transforming it, adding the metadata, et cetera, that happens in the second step in the transform contextualize structure. That might be okay, but unfortunately, time series data or industrial data in general is notoriously messy data. So that means that the third capability is about ensuring and maybe even correcting data quality. So a typical data quality problem, I faced millions of them in my career when I was still working with data, is you have a temperature reading or whatever, all of a sudden spike. Amazing. It goes further and then oops, we have a flat line, perfect. And maybe we have at a certain point in time, some drifting data. noisy data, undersampling, oversampling, all of this, but not just once, not just twice, across the entire data set. this means that, yeah, just detecting data quality issues is something extremely important. It's something which you might still wanna skip when you're just looking at dashboards. Although it's also really inconvenient to look to a wrong dashboard where your KPI is all of a sudden 9,999,999 or whatever. But especially when we are using our data in automatic models, in machine learning models, et cetera, we want data which is correct, right? Because that one outlier can mess up the entire calculation, or especially that sticky data that's null data or whatever, yeah, if we don't know the sensor is faulty, then we will never fix it. So first finding, flagging the data quality problems is really important, but also correcting them becomes more and more important. In the IT world, this is also typically where we talk about bronze, silver, gold, so this silver store is typically the one where we have validated data. So that means we are able now to connect. transform, check the data quality, and then obviously we're stepping to the data store itself, which means that the first part, the first, I would say integral part of our industrial data platform is built with these four in mind. So having the connectivity until the data store, well, that gives us the first part of our digital data platform. Now here is a small update compared to the article which is on our website. we used to say that number four is the data broker and store. That is correct, but I started seeing the broker more as just a part of the connectivity mechanism, right? Yes, you could use an MQTT broker as a mechanism to transport your data, to make it available, but it doesn't store the data. It doesn't have the historical access, right? So for us, we removed data broker from the capability map. It is an essential part, as I say, it's an essential part of the connectivity layer or the connectivity and I would say the connectivity infrastructure. But it shouldn't sit in the middle of your industrial data platform because if it sits there, you'll always need to have a store next to it in order to access the historical data. So these are the first four ones. Then comes edge compute. Edge compute is really interesting for several reasons. First of all, in some cases, we want to manipulate the data on the edge before we send it into the platform. There might be several reasons for that. For example, if you have vibration sensors at very high frequency, you might not wanna store the high frequency data in the platform. So you're gonna prepare, so you're gonna create some statistics, for example. The same is true, for example, for computer vision data, where you're analyzing video feeds, but you only wanna store the calculated features, for example. Or just some simple calculated tags. For example, we are calculating or predicting a quality, whatever, and we wanna store that predicted value into the platform. But also the other way around, right? More and more, we wanna publish machine learning models, which we create somewhere in the cloud. So suppose my machine learning environment sits here in the cloud. I'm getting the data using my APIs into my environment. I train a model, but then I somehow need to deploy my model back to the edge. So that is where EdgeCompute comes into play. And with that, already talked a little bit about capability seven, data sharing. So obviously you want to be able to share that data, not just the data, but data in context. And then finally, the number six, visualization. This is a bit tricky. You could say that visualization should be part of your data consumers. So your trending tools, your dashboarding tools. That's right. But we do see that the most powerful, industrial data platforms, they come in with some built-in visualization and dashboarding capabilities because it's something you'll always need. So that's why we believe it might be a good idea to have some building capabilities there as well. These are the seven core building blocks. You'll probably need most of them or I would say at a minimum, you'll need your connectivity, your transform capabilities, your data store, your data sharing and your visualization. You could argue that data quality is important in every case. I would say yes, but that's a bit more advanced and there might be enough cases where edge compute is also not required. But these capabilities only bring you that far. You also need to be able to support the entire stack. And that means that as you can see here, we've added some supporting functions. I'm not going into too much detail for that. I'd like to point you to the blog. But it of course starts with the fact that you probably need at a certain point rules and workflows. You wanna define things which happen. If this happens in the platform, then that gets triggered. You need orchestration across the entire set. So that means how do we make sure that it is easily possible, for example, to deploy a change to multiple edge devices, to multiple connectivity points, et cetera? How do we make sure that we have our... master data management available across all devices. This is also where data governance comes into play. Data governance is becoming more and more important in my opinion. It's also typically the way where you can, if you say data governance, that's a way to scare people away very quickly, but that's not true because if we are unable to define those standards, if we are unable to define the context, for example, then we will never be able to hit that scaling point, right? If every plant manager, if every process engineer operator has an other idea about naming conventions, about production events, et cetera, then your entire, I would say, data platform or the entire data set will easily become a mess. So we need some governance there as well. Obviously, cybersecurity is extremely, extremely important. That brings me... to the question, but yes, David, where does the unified namespace sit? And that's a really interesting one. And again, I've published an article on the blog, which I call what I like, don't like about UNS recently. I'll happily point you to that one in show notes. But I think this diagram already shows you that the concept of a unified namespace flows through different capabilities. In a, I would say the most common way of talking about unified namespace, we typically talk about a MQTT broker. Now, in my opinion, this sits mostly on the connectivity side. This also means that I see an MQTT broker sitting, for example, on top of some PLC SCADA systems, or maybe on top of... Cloud IoT devices where the broker makes that data available for the platform. A second part of unified namespace is the naming. So making the data available in context, which means that it not only comes back in number one connectivity, it also comes back in number two, transform contextualize and structure because somehow we need to define the governance, the namespace. and then apply that namespace to all data sources, to all data flowing in. As I already mentioned, the data broker, I don't really see that as part of the platform itself. Again, the data broker might be available as data source. The data broker might also be available as data sharing option. So you might wanna be able... publish or to make the data sitting in the platform available to other users using a data broker. So yeah, means that if we talk about unified namespace and again, the definition here or my definition is that unified namespace is a design pattern to build an industrial data platform which holds contextualized structured data across the entire business. So that means that it comes back here in number one, it comes back in number two, it's of course also sits here in the middle and it might sit here a bit on a broker. on the data source side, it might also sit here on the right side as a broker to expose that data to the outside world. A data broker doesn't sit here in the middle. I guess if you look to this explanation, you probably understand that as I already said in the beginning, a unified namespace is not just a product you buy, it's a philosophy, it's a design pattern which might flow through your entire design. So with the data platform being explained and with you having some first ideas on where Unified Namespace fits into it, this is actually a perfect basis for industrial AI because good industrial AI, well, good AI in general needs not only access to all data, it needs access to contextualized data to data of high quality. So we've discussed all of these things. If we built a platform, If we build this unified namespace concepts as a design pattern to structure the data, then it's easy to share that data, capability seven, share that data with AI applications. And then the Edge computer will make it possible to run these models at the edge or close to the edge. So I'll make sure to record some additional videos to also talk a bit more on UNIs. and to also talk a bit more on industrial AI. In the meantime, if you want to learn more, go to itot.academy to learn more about our six week live online learning program for ITOT professionals, where we'll teach you to understand, cooperate and scale. This program is perfectly fit for ITOT managers, architects or alike roles. Go to our DataOps vendor list on our website. This will give you a head start in your selection process. It will also contain all the links to the interviews we did with several vendors. Or just subscribe to our blog or to our YouTube channel to get our weekly updates. Thanks for watching this explainer video and see you next time with more insights in bridging IT and OT. Until then, take care. Bye bye.