opentsdb #0

Supports: xenial
Add to new model

Description

OpenTSDB is a scalable time series database built on top of Hadoop and HBase.
It simplifies the process of storing and analyzing large amounts of
time-series data generated by endpoints like sensors or servers.

Overview

OpenTSDB is a scalable time series database built on top of Hadoop and HBase. It simplifies the process of storing and analyzing large amounts of time-series data generated by endpoints like sensors or servers.

http://opentsdb.net

Usage

Deployment

Deploy the OpenTSDB charm:

juju deploy cs:~tengu-team/opentsdb-0

For OpenTSDB to work you need to deploy HBase and Zookeeper:

juju deploy hbase
juju deploy zookeeper

Add the relations when you got HBase and Zookeeper running:

juju add-relation opentsdb hbase
juju add-relation opentsdb zookeeper

To make OpenTSDB's GUI public you have to expose OpenTSDB:

juju expose opentsdb

Browse to the public IP with port to see OpenTSDB's GUI. Here you can select
metrics and graph them.

How OpenTSDB can be used

HTTP API

You can communicate with OpenTSDB by using the HTTP API. For storing data the endpoint '/api/put' is used, for example:

35.194.184.37:4242/api/put

With as JSON body:

{
    "metric": "sys.cpu.nice",
    "timestamp": 1346846400,
    "value": 18,
    "tags": {
       "host": "web01",
       "dc": "lga"
    }
}

HTTP API documentation

TCollector

TCollector is a client-side process that gathers data from local collectors and pushes the data to OpenTSDB.

TCollector documentation

Graph

In order to inspect metrics you must go to OpenTSDB's GUI by browsing to the public
IP and port (after you exposed the charm). There you can select metrics for a certain
timeframe and plot them.

Telegraf

The Telegraf charm can be used with OpenTSDB. How to make it work:
1. Deploy Telegraf:
sh juju deploy cs:telegraf-6
2. Add relation between the service you want to collect data from and Telegraf.
For example if you want to collect metrics from your HAproxy instance and send
them to OpenTSDB then you first have to add a relation to Telegraf:

sh juju add-relation telegraf:juju-info haproxy:juju-info
3. Update the outputs_config of the Telegraf charm:
```
# Configuration for OpenTSDB server to send metrics to
[[outputs.opentsdb]]
## prefix for metrics keys
prefix = "my.specific.prefix."

  ## DNS name of the OpenTSDB server
  ## Using "opentsdb.example.com" or "tcp://opentsdb.example.com" will use the
  ## telnet API. "http://opentsdb.example.com" will use the Http API.
  host = "opentsdb.example.com"

  ## Port of the OpenTSDB server
  port = 4242

  ## Number of data points to send to OpenTSDB in Http requests.
  ## Not used with telnet API.
  httpBatchSize = 50

  ## Debug true - Prints OpenTSDB communication
  debug = false
```

Telegraf Charm

Important to know

Removing relation with HBase

When you remove the relation between OpenTSDB and HBase your data will not be removed from HBase. This prevents data loss from users that accidentally remove the relation between OpenTSDB and HBase.

To Do

  • Implement clustering

Contact Information

Authors

Configuration

async_io
(boolean) Whether or not to use NIO or tradditional blocking IO.
True
auto_create_metrics
(boolean) Whether or not to automatically create UIDs for new metric types.
True
bind
(string) The IPv4 network address to bind to.
0.0.0.0
enable_chunked
(boolean) Whether or not to enable incoming chunk support for the HTTP RPC.
True
enable_compaction
(boolean) Whether or not to enable data compaction in HBase.
True
flush_interval
(int) How often, in milliseconds, to flush the data point queue to storage.
1000
keep_alive
(boolean) Determines whether or not to send keepalive packets to peers.
True
mode
(string) Whether or not the TSD will allow writing data points. Must be either "rw" to allow writing data or "ro" to block data point writes.
rw
port
(int) The TCP port TSD should use for communications.
4242
reuse_address
(boolean) Determines if the same socket should be used for new connections.
True
tcp_no_delay
(boolean) Disable Nagel's algorithm.
True
worker_threads
(int) Number of worker threads dedicated to Netty.
8