opentsdb #0
Description
OpenTSDB is a scalable time series database built on top of Hadoop and HBase.
It simplifies the process of storing and analyzing large amounts of
time-series data generated by endpoints like sensors or servers.
- Tags:
- database ›
Overview
OpenTSDB is a scalable time series database built on top of Hadoop and HBase. It simplifies the process of storing and analyzing large amounts of time-series data generated by endpoints like sensors or servers.
Usage
Deployment
Deploy the OpenTSDB charm:
juju deploy cs:~tengu-team/opentsdb-0
For OpenTSDB to work you need to deploy HBase and Zookeeper:
juju deploy hbase
juju deploy zookeeper
Add the relations when you got HBase and Zookeeper running:
juju add-relation opentsdb hbase
juju add-relation opentsdb zookeeper
To make OpenTSDB's GUI public you have to expose OpenTSDB:
juju expose opentsdb
Browse to the public IP with port to see OpenTSDB's GUI. Here you can select
metrics and graph them.
How OpenTSDB can be used
HTTP API
You can communicate with OpenTSDB by using the HTTP API. For storing data the endpoint '/api/put' is used, for example:
35.194.184.37:4242/api/put
With as JSON body:
{
"metric": "sys.cpu.nice",
"timestamp": 1346846400,
"value": 18,
"tags": {
"host": "web01",
"dc": "lga"
}
}
TCollector
TCollector is a client-side process that gathers data from local collectors and pushes the data to OpenTSDB.
Graph
In order to inspect metrics you must go to OpenTSDB's GUI by browsing to the public
IP and port (after you exposed the charm). There you can select metrics for a certain
timeframe and plot them.
Telegraf
The Telegraf charm can be used with OpenTSDB. How to make it work:
1. Deploy Telegraf:
sh
juju deploy cs:telegraf-6
2. Add relation between the service you want to collect data from and Telegraf.
For example if you want to collect metrics from your HAproxy instance and send
them to OpenTSDB then you first have to add a relation to Telegraf:
sh
juju add-relation telegraf:juju-info haproxy:juju-info
3. Update the outputs_config of the Telegraf charm:
```
# Configuration for OpenTSDB server to send metrics to
[[outputs.opentsdb]]
## prefix for metrics keys
prefix = "my.specific.prefix."
## DNS name of the OpenTSDB server
## Using "opentsdb.example.com" or "tcp://opentsdb.example.com" will use the
## telnet API. "http://opentsdb.example.com" will use the Http API.
host = "opentsdb.example.com"
## Port of the OpenTSDB server
port = 4242
## Number of data points to send to OpenTSDB in Http requests.
## Not used with telnet API.
httpBatchSize = 50
## Debug true - Prints OpenTSDB communication
debug = false
```
Important to know
Removing relation with HBase
When you remove the relation between OpenTSDB and HBase your data will not be removed from HBase. This prevents data loss from users that accidentally remove the relation between OpenTSDB and HBase.
To Do
- Implement clustering
Contact Information
Authors
- Michiel Ghyselinck michiel.ghyselinck@tengu.io
Configuration
- async_io
- (boolean) Whether or not to use NIO or tradditional blocking IO.
- True
- auto_create_metrics
- (boolean) Whether or not to automatically create UIDs for new metric types.
- True
- bind
- (string) The IPv4 network address to bind to.
- 0.0.0.0
- enable_chunked
- (boolean) Whether or not to enable incoming chunk support for the HTTP RPC.
- True
- enable_compaction
- (boolean) Whether or not to enable data compaction in HBase.
- True
- flush_interval
- (int) How often, in milliseconds, to flush the data point queue to storage.
- 1000
- keep_alive
- (boolean) Determines whether or not to send keepalive packets to peers.
- True
- mode
- (string) Whether or not the TSD will allow writing data points. Must be either "rw" to allow writing data or "ro" to block data point writes.
- rw
- port
- (int) The TCP port TSD should use for communications.
- 4242
- reuse_address
- (boolean) Determines if the same socket should be used for new connections.
- True
- tcp_no_delay
- (boolean) Disable Nagel's algorithm.
- True
- worker_threads
- (int) Number of worker threads dedicated to Netty.
- 8