OpenTSDB is an open source, scalable, <buzzword>big data</buzzword> solution for storing time series data. Well, that what the name of the project actually says ;-) In combination with Grafana you can pretty easy build a few nice dashboards for visualizing that data. The only question is of course, how do you get your data into that system.
My intention was to provide a simple way to stream metrics into OpenTSDB using Apache Camel. A quick search did not bring up any existing solutions for pushing data from Apache Camel to OpenTSDB, so I decided to write a small Camel component which can pick up data and send this to OpenTSDB using the HTTP API. Of course having a plain OpenTSDB HTTP Collector API for that would be fine as well. So I did split up the component into three different modules. A generic OpenTSDB collector API, an HTTP implementation of that and finally the Apache Camel component.
All components are already available in Maven Central, and although they have the version number 0.0.3, they are working quite well ;-)
<dependency>
<groupId>de.dentrassi.iot</groupId>
<artifactId>de.dentrassi.iot.opentsdb.collector</artifactId>
<version>0.0.3</version>
</dependency>
<dependency>
<groupId>de.dentrassi.iot</groupId>
<artifactId>de.dentrassi.iot.opentsdb.collector.http</artifactId>
<version>0.0.3</version>
</dependency>
<dependency>
<groupId>de.dentrassi.iot</groupId>
<artifactId>de.dentrassi.iot.opentsdb.collector.camel</artifactId>
<version>0.0.3</version>
</dependency>
Dropping those dependencies into your classpath, or into your OSGi container ;-), you can use the following code to easily push data coming from MQTT directly into OpenTSDB:
CamelContext context = new DefaultCamelContext();
// add routes
context.addRoutes(new RouteBuilder() {
@Override
public void configure() throws Exception {
from("paho:sensors/test2/temperature?brokerUrl=tcp://iot.eclipse.org")
.log("${body}")
.convertBodyTo(String.class).convertBodyTo(Float.class)
.to("open-tsdb:http://localhost:4242#test2/value=temp");
from("paho:tele/devices/TEMP?brokerUrl=tcp://iot.eclipse.org")
.log("${body}")
.convertBodyTo(String.class).convertBodyTo(Float.class)
.to("open-tsdb:http://localhost:4242#test3/value=temp");
}
});
// start the context
context.start();
You can directly push Floats or Integers into OpenTSDB. The example above shows a “to” component which does directly address a metric. Using #test3/value=temp If you need more tags, then those can be added using #test3/value=temp/foo=bar.
But it is also possible to have a generic endpoint and provide the metric information in the actual payload. In this case you have to use the type <a href="https://dentrassi.de/iot/de.dentrassi.iot.opentsdb.collector/apidocs/de/dentrassi/iot/opentsdb/collector/Data.html" target="_blank">de.dentrassi.iot.opentsdb.collector.Data</a> and fill in the necessary information. It is also possible to publish an array of Data[].