IoT

23 posts

Providing telemetry data with OPC UA on Eclipse Kura

The upcoming version 2.1.0 of Eclipse Kura™ will feature an enhanced version of the Apache Camel™ integration which was introduced in Kura 2.0.0. There are various new ways on how to run Camel routes, configured either by XML routes or using the Java DSL. Apache Camel can act as a Kura application but, new in this release, there is also a way to simply configure Camel as a “cloud service”. In past releases of Kura, applications could only push data to one cloud target. The new 2.1.0 release will add the functionality of adding multiple cloud targets and one of those targets can be Apache Camel router instances.

With Camel you can have different ways of achieving this goal, but in this post I would like to focus on the “out of the box” way, by simply configuring (not developing) a set of Camel routes, which act as cloud service. Traditional instances of cloud services in Kura are only capable of delivering data to one cloud target or subscribing to one cloud infrastructure. But using Apache Camel as a technology it is possible to connect to a bunch of technologies at the same time.

The setup

The setup will be a Kura instance, running a pre-release version of Kura 2.1.0. The final version should be out in a few weeks and won’t differ much from the current version. We will be configuring a new cloud service instance which takes Kura application payload data and provide it as OPC UA, using the Camel OPC UA adapter. As payload provider (aka Kura application) we will be using the “Example publisher” from my Kura addons project.

Open up the Kura Web UI, navigate to “Packages” and select “Install/Update”. Switch to “URL” and provide the following URL:

https://dentrassi.de/download/kura/de.dentrassi.kura.addons.example.publisher_0.1.0-SNAPSHOT.dp

Note: As an alternative you can also download the “dp” package with your desktop browser and deploy the file using the “file” upload instead of “URL”.

Adding packages to Kura
Adding packages to Kura

The installation may take a bit and it may be necessary to press the “Refresh” button in order to see the installed package. After the packages was installed you should be able to see the service “Camel example publisher” on the left side.

Now we need to install the “Milo component for Camel”. Press “Install/Update” again and enter the following URL:

http://central.maven.org/maven2/de/dentrassi/kura/addons/de.dentrassi.kura.addons.milo/0.2.2/de.dentrassi.kura.addons.milo-0.2.2.dp

This installation will take a lot longer and you will need to check again by pressing the “Refresh” button in the Web UI.

We will also need to allow TCP access to port 12685. If you have the network managed version of Kura installed switch to the UI section “Firewall” and open a new port “12685” allowing access from “0.0.0.0/0” (Permitted Network) and press “Apply”.

A new cloud service

By default the “example publisher” will publish to the default Kura cloud service instance. We will now create a new Cloud service instance and then redirect the data to OPC UA. The data will be available as an OPC UA server. OPC UA differs between client and server. And while the Camel component does provide both ways, in this case we want others to consume our data, so offering data as an OPC UA server is the way to go.

Navigate to “Cloud Services” and press the “New” button. From the list of possible providers select org.eclipse.kura.camel.cloud.factory.CamelFactory, enter a cloud service PID (e.g. camel-opcua) and press “Create”.

After the instance has been created select it and configure it with the following options:

Router XML:

<routes xmlns="http://camel.apache.org/schema/spring">
    <route id="opc-ua-example">
       <from uri="vm:camel:example"/>
       <split>
           <simple>${body.metrics().entrySet()}</simple>
           <setHeader headerName="item">
               <simple>${body.key()}</simple>
           </setHeader>
           <setBody>
               <simple>${body.value()}</simple>
           </setBody>
           <toD uri="my-milo:${header.item}"/>
       </split>
    </route>
</routes>

Initialzation Code:

var milo = new org.apache.camel.component.milo.server.MiloServerComponent();
milo.setEnableAnonymousAuthentication(true);
camelContext.addComponent("my-milo", milo);
Screenshot of cloud service configuration
OPC UA configuration

Assigning the cloud service

Now we need to configure the example publisher to actually use our new cloud service instance. Select “Camel example publisher” from the left navigation bar and enter “opcua” (or whatever PID you used before) as “Cloud Service PID”. Apply the changes.

Testing the result

First of all, if you log in into your device using SSH, you should be able to see that port 12685 is opened:

root@raspberrypi:/home/pi# ss -nlt | grep 12685
LISTEN     0      128                      :::12685                   :::*     

Now you can connect to your device using any OPC UA explorer to the URI: opc.tcp://<my-ip>:12685

I am using Android and the “ProSYS OPC UA Client”

Summing it up

This tutorial uses a SNAPSHOT version of Eclipse Milo. Simply due to the fact that no version of Milo is released just yet. This should change in the following weeks and my play is to update the blog post once it is available. However the functionality of Milo will not change and using the Camel component, most internals of Milo are hidden anyway.

Update: As Milo and the Camel Milo component are released now I did update the links.

Apache Camel on Eclipse Kura can provide a complete new way of communication. This example was a rather simple one, Camel can do a lot more when it comes to processing data. And not all real-life applications may be as easy as that. But of course the intention of this blog post was to give a quick introduction into Camel and Kura in combination. Using the Camel Java DSL or the Kura Camel programmatic API can give greater flexibility. And yet, the example shows that even with a few lines of Camel XML, amazing things can be achieved.

Collecting data to OpenTSDB with Apache Camel

OpenTSDB is an open source, scalable, <buzzword>big data</buzzword> solution for storing time series data. Well, that what the name of the project actually says ;-) In combination with Grafana you can pretty easy build a few nice dashboards for visualizing that data. The only question is of course, how do you get your data into that system.

My intention was to provide a simple way to stream metrics into OpenTSDB using Apache Camel. A quick search did not bring up any existing solutions for pushing data from Apache Camel to OpenTSDB, so I decided to write a small Camel component which can pick up data and send this to OpenTSDB using the HTTP API. Of course having a plain OpenTSDB HTTP Collector API for that would be fine as well. So I did split up the component into three different modules. A generic OpenTSDB collector API, an HTTP implementation of that and finally the Apache Camel component.

All components are already available in Maven Central, and although they have the version number 0.0.3, they are working quite well ;-)

[code language=”xml”]
<dependency>
<groupId>de.dentrassi.iot</groupId>
<artifactId>de.dentrassi.iot.opentsdb.collector</artifactId>
<version>0.0.3</version>
</dependency>
<dependency>
<groupId>de.dentrassi.iot</groupId>
<artifactId>de.dentrassi.iot.opentsdb.collector.http</artifactId>
<version>0.0.3</version>
</dependency>
<dependency>
<groupId>de.dentrassi.iot</groupId>
<artifactId>de.dentrassi.iot.opentsdb.collector.camel</artifactId>
<version>0.0.3</version>
</dependency>
[/code]

Dropping those dependencies into your classpath, or into your OSGi container ;-), you can use the following code to easily push data coming from MQTT directly into OpenTSDB:

[code language=”java”]
CamelContext context = new DefaultCamelContext();

// add routes

context.addRoutes(new RouteBuilder() {

@Override
public void configure() throws Exception {
from("paho:sensors/test2/temperature?brokerUrl=tcp://iot.eclipse.org")
.log("${body}")
.convertBodyTo(String.class).convertBodyTo(Float.class)
.to("open-tsdb:http://localhost:4242#test2/value=temp");

from("paho:tele/devices/TEMP?brokerUrl=tcp://iot.eclipse.org")
.log("${body}")
.convertBodyTo(String.class).convertBodyTo(Float.class)
.to("open-tsdb:http://localhost:4242#test3/value=temp");
}
});

// start the context
context.start();
[/code]

You can directly push Floats or Integers into OpenTSDB. The example above shows a “to” component which does directly address a metric. Using #test3/value=temp If you need more tags, then those can be added using #test3/value=temp/foo=bar.

But it is also possible to have a generic endpoint and provide the metric information in the actual payload. In this case you have to use the type de.dentrassi.iot.opentsdb.collector.Data and fill in the necessary information. It is also possible to publish an array of Data[].

Bringing OPC UA to Apache Camel

My first two weeks at Red Hat have been quite awesome! There is a lot to learn and one the first things I checked out was Apache Camel and Eclipse Milo. While Camel is more known open source project, Eclipse Milo is a newcomer project at the Eclipse Foundation. And while it is officially still in the Incubation Phase, it is a tool you can really work with! Eclipse Milo is an OPC UA client and server implementation.

Overview

Although Apache Camel already has an OPC DA connector, based on OpenSCADA’s Utgard library (sorry, but I couldn’t resist ;-) ), OPC UA is a complete different thing. OPC DA remote connectivity is based on DCOM and has always been really painful. That’s also the reason for the name: Utgard. But with OPC UA that has been cleared up. It features different communication layers, the most prominent, up until now, is the custom, TCP based binary protocol.

I started by investigating the way Camel works and dug a little bit in the API of Eclipse Milo. From an API perspective, both tools couldn’t be more different. Where Camel does try to focus on simplicity and reducing the full complexity down to a very slim and simple interface, Milo simply unleashes the full complexity of OPC UA into a very nice, but complex, Java 8 style API. Now you can argue night and day what is the better approach, with a little bit of glue code, both sides work perfectly together.

To make it short, the final result is an open source project on GitHub: ctron/de.dentrassi.camel.milo, which features two Camel components providing OPC UA client and OPC UA server support. Meaning that it is possible now to consume or publish OPC UA value updates by simply configuring a Camel URI.

Examples

For example:

milo-client:tcp://foo:bar@localhost:12685?nodeId=items-MyItem&namespaceUri=urn:org:apache:camel

Can be used to configure an OPC UA Client connection to “localhost:12685”. Implementing both Camel producer and consumer, it is possible to subscribe/monitor this item or write value updates.

The following URI does create a server item named “MyItem”:

milo-server:MyItem

Which can then be accessed using an OPC UA client. For the server component the configuration options like bind address and port are located on the Camel component, not the endpoint. However it is possible with Camel to register the same component type with different configurations.

Also see the testing classes, which show a few examples.

What is next?

With help from @hekonsek, who knows a lot more about Camel than I do, we hope to contribute this extension to the Apache Camel project. So once Eclipse Milo has it’s first release, this could become an “out-of-the-box” experience when using Apache Camel, thanks to another wonderful project of Eclipse IoT of course ;-)

Also, with a little bit of luck, there will be a talk at EclipseCon Europe 2016 about this adventure. It will even go a bit further because I do want to bring this module into Eclipse Kura, so that Eclipse Kura will feature OPC UA support using Apache Camel.