Yesterday the release 0.4.0 of Eclipse NeoSCADA™ was made available. This release features a cool new feature, an IEC 60870-5-104 stack, written in Java, licensed under the EPL and available on Maven Central. See also the Eclipse Wiki: https://wiki.eclipse.org/EclipseNeoSCADA/Components/IEC60870
So it was time to update my Apache Camel component for IEC 60870 and finally release it to Maven Central with proper dependencies on Eclipse NeoSCADA 0.4.0.
The Camel components for IEC 60870 are based on the NeoSCADA implementation and provide both client and server side of the protocol. Although the implementation of IEC 60870 does not implement all message types defined, all relevant types for data transmission and control are implementation and other modules can be added by an extensible mechanism, using the core layers of the protocol.
For Camel there are two endpoint types iec60870-server and iec60870-client. These allow either to offer data as IEC 60870 or to actively request data from another 60870 server.
The client component will open a connection to the remote station and initiate the data transmission. 60780 will then send updates for all addresses but the Camel component will only forward events to connected endpoints. When the connection breaks, it will be periodically tried to re-establish the connection. All event coming from the IEC connection can of course be processed with Camel.
For the server side the Camel component will hold an intern data model which can be filled using the Camel routes. In internal state will then be published to IEC clients connecting to the server instance. It also allows the use of background transmission or batching of events when required.
Now what can you actually do with IEC 60870 and Apache Camel? Well, to be honest, if you never have heard about IEC 60870 and don’t have a proper use case or specific requirement for it, then you should probably look for something different to play with ;-) IEC 60870 is used to remotely control and monitor electrical systems and power control systems (see Wikipedia page about IEC 60870-5). On the other hand, if you do want to use 60870, then the Apache Camel component can make it pretty easy to provide a data over the IEC protocol or get data out of an 60870 based system.
As routing data with Camel is easy, you can for example create a very simple Mock device in a Raspberry Pi for testing your system with an IEC component. And you can do all of this with pure open source (EPL licensed) software. You can also extract data out of your application and offer it towards another system, which explicitly requires a transmission based on IEC 60870.
When the component will be released on the next few weeks I will hopefully find the time to provide some example, showing what you can do with IEC 60870 and Apache Camel.
This means that there is now a released version of, available on Maven Central as well, of the Apache Camel Milo component which can either be used standalone or dropped in directly to some OSGi container like Apache Karaf.
If you want to have a quick example you can clone the GitHub repository and simply compile and run an example using the following commands:
git clone https://github.com/ctron/de.dentrassi.camel.milo
This will compile and run a simple example which transfers all temperate measurements from the iot.eclipse.org MQTT server from the topic javaonedemo/eclipse-greenhouse-9home/sensors/temperature to the OPC UA tag item-GreenHouse.Temperature, namespace urn:org:apache:camel on the connection opc.tcp://localhost:12685.
The project is a simple OSGi Blueprint bundle which can be also be run by Apache Camel directly. The only configuration is the blueprint file:
If you want some more IDE integration you can quickly install the JBoss FUSE tooling and connect via JMX to either the Maven controlled instance (mvn camel:run) or the Karaf instance and monitor, debug and trace the active Camel routes:
What is next?
For one this component will hopefully become part of Apache Camel itself. And of course there is always something to improve ;-)
I also did update the Kura Addon for Milo, which provides the Milo Camel component for Eclipse Kura 2.1.0 which was recently released. This component is now also available on Maven Central and can easily be deployed into Kura. See the Kura Addons page for more information.
Then there are a few location where I used SNAPSHOT versions of Milo and for some I did promise an update. So I will try to update as many locations as I can with links to the released version of those components.
The upcoming version 2.1.0 of Eclipse Kura™ will feature an enhanced version of the Apache Camel™ integration which was introduced in Kura 2.0.0. There are various new ways on how to run Camel routes, configured either by XML routes or using the Java DSL. Apache Camel can act as a Kura application but, new in this release, there is also a way to simply configure Camel as a “cloud service”. In past releases of Kura, applications could only push data to one cloud target. The new 2.1.0 release will add the functionality of adding multiple cloud targets and one of those targets can be Apache Camel router instances.
With Camel you can have different ways of achieving this goal, but in this post I would like to focus on the “out of the box” way, by simply configuring (not developing) a set of Camel routes, which act as cloud service. Traditional instances of cloud services in Kura are only capable of delivering data to one cloud target or subscribing to one cloud infrastructure. But using Apache Camel as a technology it is possible to connect to a bunch of technologies at the same time.
The setup will be a Kura instance, running a pre-release version of Kura 2.1.0. The final version should be out in a few weeks and won’t differ much from the current version. We will be configuring a new cloud service instance which takes Kura application payload data and provide it as OPC UA, using the Camel OPC UA adapter. As payload provider (aka Kura application) we will be using the “Example publisher” from my Kura addons project.
Open up the Kura Web UI, navigate to “Packages” and select “Install/Update”. Switch to “URL” and provide the following URL:
Note: As an alternative you can also download the “dp” package with your desktop browser and deploy the file using the “file” upload instead of “URL”.
The installation may take a bit and it may be necessary to press the “Refresh” button in order to see the installed package. After the packages was installed you should be able to see the service “Camel example publisher” on the left side.
Now we need to install the “Milo component for Camel”. Press “Install/Update” again and enter the following URL:
This installation will take a lot longer and you will need to check again by pressing the “Refresh” button in the Web UI.
We will also need to allow TCP access to port 12685. If you have the network managed version of Kura installed switch to the UI section “Firewall” and open a new port “12685” allowing access from “0.0.0.0/0” (Permitted Network) and press “Apply”.
A new cloud service
By default the “example publisher” will publish to the default Kura cloud service instance. We will now create a new Cloud service instance and then redirect the data to OPC UA. The data will be available as an OPC UA server. OPC UA differs between client and server. And while the Camel component does provide both ways, in this case we want others to consume our data, so offering data as an OPC UA server is the way to go.
Navigate to “Cloud Services” and press the “New” button. From the list of possible providers select org.eclipse.kura.camel.cloud.factory.CamelFactory, enter a cloud service PID (e.g. camel-opcua) and press “Create”.
After the instance has been created select it and configure it with the following options:
var milo = new org.apache.camel.component.milo.server.MiloServerComponent();
Assigning the cloud service
Now we need to configure the example publisher to actually use our new cloud service instance. Select “Camel example publisher” from the left navigation bar and enter “opcua” (or whatever PID you used before) as “Cloud Service PID”. Apply the changes.
Testing the result
First of all, if you log in into your device using SSH, you should be able to see that port 12685 is opened:
This tutorial uses a SNAPSHOT version of Eclipse Milo. Simply due to the fact that no version of Milo is released just yet. This should change in the following weeks and my play is to update the blog post once it is available. However the functionality of Milo will not change and using the Camel component, most internals of Milo are hidden anyway.
Update: As Milo and the Camel Milo component are released now I did update the links.
Apache Camel on Eclipse Kura can provide a complete new way of communication. This example was a rather simple one, Camel can do a lot more when it comes to processing data. And not all real-life applications may be as easy as that. But of course the intention of this blog post was to give a quick introduction into Camel and Kura in combination. Using the Camel Java DSL or the Kura Camel programmatic API can give greater flexibility. And yet, the example shows that even with a few lines of Camel XML, amazing things can be achieved.
OpenTSDB is an open source, scalable, <buzzword>big data</buzzword> solution for storing time series data. Well, that what the name of the project actually says ;-) In combination with Grafana you can pretty easy build a few nice dashboards for visualizing that data. The only question is of course, how do you get your data into that system.
My intention was to provide a simple way to stream metrics into OpenTSDB using Apache Camel. A quick search did not bring up any existing solutions for pushing data from Apache Camel to OpenTSDB, so I decided to write a small Camel component which can pick up data and send this to OpenTSDB using the HTTP API. Of course having a plain OpenTSDB HTTP Collector API for that would be fine as well. So I did split up the component into three different modules. A generic OpenTSDB collector API, an HTTP implementation of that and finally the Apache Camel component.
All components are already available in Maven Central, and although they have the version number 0.0.3, they are working quite well ;-)
You can directly push Floats or Integers into OpenTSDB. The example above shows a “to” component which does directly address a metric. Using #test3/value=temp If you need more tags, then those can be added using #test3/value=temp/foo=bar.
But it is also possible to have a generic endpoint and provide the metric information in the actual payload. In this case you have to use the type de.dentrassi.iot.opentsdb.collector.Data and fill in the necessary information. It is also possible to publish an array of Data.
My first two weeks at Red Hat have been quite awesome! There is a lot to learn and one the first things I checked out was Apache Camel and Eclipse Milo. While Camel is more known open source project, Eclipse Milo is a newcomer project at the Eclipse Foundation. And while it is officially still in the Incubation Phase, it is a tool you can really work with! Eclipse Milo is an OPC UA client and server implementation.
Although Apache Camel already has an OPC DA connector, based on OpenSCADA’s Utgard library (sorry, but I couldn’t resist ;-) ), OPC UA is a complete different thing. OPC DA remote connectivity is based on DCOM and has always been really painful. That’s also the reason for the name: Utgard. But with OPC UA that has been cleared up. It features different communication layers, the most prominent, up until now, is the custom, TCP based binary protocol.
I started by investigating the way Camel works and dug a little bit in the API of Eclipse Milo. From an API perspective, both tools couldn’t be more different. Where Camel does try to focus on simplicity and reducing the full complexity down to a very slim and simple interface, Milo simply unleashes the full complexity of OPC UA into a very nice, but complex, Java 8 style API. Now you can argue night and day what is the better approach, with a little bit of glue code, both sides work perfectly together.
To make it short, the final result is an open source project on GitHub: ctron/de.dentrassi.camel.milo, which features two Camel components providing OPC UA client and OPC UA server support. Meaning that it is possible now to consume or publish OPC UA value updates by simply configuring a Camel URI.
Can be used to configure an OPC UA Client connection to “localhost:12685”. Implementing both Camel producer and consumer, it is possible to subscribe/monitor this item or write value updates.
The following URI does create a server item named “MyItem”:
Which can then be accessed using an OPC UA client. For the server component the configuration options like bind address and port are located on the Camel component, not the endpoint. However it is possible with Camel to register the same component type with different configurations.
With help from @hekonsek, who knows a lot more about Camel than I do, we hope to contribute this extension to the Apache Camel project. So once Eclipse Milo has it’s first release, this could become an “out-of-the-box” experience when using Apache Camel, thanks to another wonderful project of Eclipse IoT of course ;-)
Also, with a little bit of luck, there will be a talk at EclipseCon Europe 2016 about this adventure. It will even go a bit further because I do want to bring this module into Eclipse Kura, so that Eclipse Kura will feature OPC UA support using Apache Camel.