Apache PLC4X, the missing link for industrial innovation

No Comments

When reading the current media or talking directly to people in the industry, it seems the industry is currently undergoing one of its greatest revolutions. It’s all about “Industry 4.0”, “smart factories” and “digitization” in general.

Usually this implies: optimizing production, increasing product quality, reducing waste, speeding up production, eliminating downtime right down to being able to produce products individually customizable by their customers.

Digging a little deeper into what this actually means from a technical perspective, the areas mostly involved are: integration of production machinery, collecting data (can be huge amounts), using big data systems to analyze this and perhaps even use machine learning, deep learning, or artificial intelligence to draw conclusions from that.

So in general, it’s mostly about all the things we have been doing for the last decade or so. Open source provides really stable and mature systems to take on the fight. So why aren’t all the awesome people who built all of these great systems involved in this revolution?

A plethora of standards

Fact is that they are trying: just look at all the IoT initiatives at the big open source organizations. The Apache Software Foundation has a large repertoire of projects in this space. The Eclipse Foundation’s IoT segment as well as the Linux Foundation’s LF-Edge initiative are also working hard on providing open solutions for an open future in the industry.

So what’s missing? Why aren’t these technologies eagerly adopted by the industry? The answer is quite simple: they have huge problems communicating with the existing industrial hardware. While it is possible to communicate with some newer devices, most hardware on the shop floor is currently not fit to be integrated into one big system. This is due to the fact that the vendors producing this highly proprietary hardware usually focus completely on their own proprietary protocols for communication. There is a vast number of so-called standards as well as an even bigger number proprietary protocols. Unfortunately these are all incompatible with each other. So integrating machinery of multiple vendors is currently extremely difficult, if not even impossible. Beyond that, also the adoption and distribution of these protocols varies greatly, depending on where you are. In Europe, completely different protocols are dominating the market than in the US or again different ones in Asia.

What about OPC-UA?

A new standard, OPC-UA, tries to fix this. This has been worked on by almost all major players in the industry for almost a decade. However, in my opinion, OPC-UA is still far from ideal, at least at the moment. One of the main reasons for that is that for example the standard is still being worked on and some important features have just been finalized in 2018. So even if at that time all PLC vendors had been able to immediately start shipping their products with OPC-UA support – with an average machine lifetime of 15-25 years for industrial machinery – you can imagine that it will take 15-25 years for this to be fully available. In addition to this, OPC-UA support is currently a second-class citizen on many PLCs. So instead of deeply integrating OPC-UA support into the core of the PLC, often an application-server-like piece of software is run on the same CPU and internally the OPC-UA Server uses the internal protocol to communicate with an internally unchanged core PLC. This usually comes with a huge performance disadvantage.

So what options do we have till then? Well, unfortunately none that I would like to live with. At least this was the case till the end of 2017.

Enter Apache PLC4X

At this time I was able to initiate a project at codecentric’s innovation incubator which had the goal of tackling exactly that problem. For the past 20 months I have had the pleasure of being able to work full-time on implementing a solution for this connectivity problem. It was our goal to create an API for accessing industrial hardware (mainly PLCs) but not putting the burden of having to deal with all of the protocol details onto the shoulders of the developer. In general we are doing something very similar to what JDBC did in the late 90’s for simplifying access to relational databases. The only difference is that we’re not only defining the API, but also implementing the drivers ourselves. In addition to the communication itself, we also wanted to provide modules for easy integration into existing open source solutions. We called the project PLC4X.

Apache PLC4x logo

At codecentric, we’re more a bunch of tech geeks that greatly enjoy building awesome bleeding-edge software and open source is part of our DNA. Building and selling proprietary products not so much. We knew from the beginning that this endeavor should be taken on as an open source project. Myself being a full-blooded Apache Member involved in numerous projects at the Apache Software Foundation and strongly believing in The Apache Way for running communities, there was never a second thought to where this should happen.

Therefore, in December 2017, our little project moved to the Apache Incubator and became Apache PLC4X (incubating).

This has been my first time to be involved in an open source project from writing the first line of code. I usually joined when there was already an initial community. So it was new to me and I had to learn how to get people to join in. So it took about half a year of myself sending email after email to myself and replying to these on our public mailing list (dev@plc4x.apache.org), going to industrial fairs and giving 5-minute introductions to hundreds of companies, submitting talks to CFPs for every conference or meetup I could find, hoping that at least one would be accepted. Also I guess in that time I spent more time riding through Germany in trains than in the office, because I was meeting with every company that gave me the opportunity to pitch Apache PLC4X.

Mid 2018 was when my efforts seem to have started to show effect. First emails from people asking questions started coming in. Even first people actually trying out what I was working on. Also first companies started joining as PLC4X was solving big problems they were having. About that time also first companies actually gave us the chance to do first test projects with them. Each of these were a lot of work, but also tremendous success stories – unfortunately none we were allowed to talk openly about, let alone mention any names.

I’ll never forget the joy and satisfaction when voting in our first new contributors and seeing the community start growing or giving a talk and seeing first Apache PLC4X stickers on the laptops of attendees in the first row. I guess this is how parents must feel seeing their kids do their first steps.

What have we achieved so far?

Well, we managed to implement the Java API and managed to stabilize this in our last 4 releases.
The functions the API defines are:
– Reading of data
– Writing of data
– Subscribing to events and data changes

Currently Apache PLC4X supports the following protocols:
– Siemens S7-Step7
– Beckhoff ADS
– Modbus
– EtherNet/IP

First work has started on:
– KNXNet/IP
– Emerson DeltaV

And we are prepared to start work on the following protocols in the near future:
– Siemens S7-TIA
– Profinet
– BACnet
– UPC-UA

Future versions will also support functionality like:
(These are already part of the API, but currently don’t have any drivers that implement these features)
– Executing pre-defined function blocks
– Listing of resources

The project is also not called PLC4J for a reason, because we believe that the difficulty doesn’t lie in writing drivers, but in understanding the protocols. Therefore we are planning on supporting drivers for multiple languages. Work for C++ has already begun and in the next few weeks we’ll probably be adding C# to that list.

Next steps for Apache PLC4X

As we knew from the start that manually implementing drivers in multiple languages, keeping them in sync and offering support for them is an almost impossible task, we invested quite a lot of time and effort into developing a way to formally specify an industrial protocol in a machine-processable way and to start generating most of the driver code from that specification. In the end we came up with a combination of DFDL (Data Format Definition Language – Format) and SCXML (State-Chart XML – State machine). First tests with drivers interpreting this specification based on the Apache Daffodil and the Apache Commons SCXML implementations proved that this is possible. Now we are currently implementing code generators in order to provide the performance the hand-written drivers have provided and hope to be finished with this in a two or three months.

As soon as that’s done, adding support for a new language is mainly manually implementing the API module and creating a set of templates for the new language. Implementing a new protocol is just providing roughly 2 xml files.

Then we will also be able to provide different versions of our drivers. As security is a large concern to the industry, we are planning on providing stripped-down versions of drivers that for example only allow reading of data and no code for writing is included. Or even more restrictive, generating a passive-mode driver which is most certainly side-effect free as it passively intercepts all network traffic and extracts information from that without even sending a single network packet to the remote device. This will also be a unique selling point of Apache PLC4X as we don’t know of any other software able to communicate in this form.

We are also currently reaching out to other open source initiatives such as the Eclipse Foundations IoT projects as well as the Linux Foundations LF-Edge initiative, as all of these are in great need of a communication layer like Apache PLC4X.

However I guess I should have been a little more careful with the CFPs though. Looking back in 2018 I held about 17 talks on PLC4X in North America and throughout Europe. But the community has grown and we have learned how things work at Apache.

Apache PLC4X is now a top-level project

So after after about 14 months in the Incubator, first suggestions came up by our mentors on graduating into a top-level project. And now this has happened: after 16 months of incubation, the Apache PLC4X project is now an official top-level project of the Apache Software Foundation. I really want to congratulate all of us on achieving this goal. It’s a great demonstration of how a group of people with a common goal and an open source mindset can achieve big things.

Links to related Posts:
– Apache Press Release: https://blogs.apache.org/foundation/entry/the-apache-software-foundation-announces49

Tags

Christofer Dutz

Likes to swim outside the mainstream. He is really passionate about walking new paths. Some times this means making a new technology useable for the masses or by creating new combinations of established strategies. He invests almost all of his free time on the job or privately for working on multiple projects at the Apache Software Foundation. Here he is particularly involved as (P)PMC in projects like Apache Flex, Apache Edgent and now Apache PLC4X. Beyond that he is also very involved on the community side, for which he was invited to become a Member of the Apache Software Foundation in 2015.

Comment

Your email address will not be published. Required fields are marked *