collectd
collectd is a daemon for collecting system performance. collectd provides various storage mechanisms to store different values. It periodically collects relevant statistical information about the system while it is running and storing information. Utilizing this information helps identify current system performance bottlenecks and predict future system loads.
Simply point the collectd configuration to the domain name (or IP address) and corresponding port of the server running taosAdapter to write the data collected by collectd into TDengine, fully leveraging TDengine's efficient storage and query performance and cluster processing capabilities for time-series data.
Prerequisites
Several preparations are needed to write collectd data into TDengine.
- TDengine cluster is deployed and running normally
- taosAdapter is installed and running normally, for details please refer to taosAdapter User Manual
- collectd is installed. For installation of collectd, please refer to official documentation
Configuration Steps
Configuring taosAdapter
Method to configure taosAdapter to receive collectd data:
- Enable the configuration item in the taosAdapter configuration file (default location is /etc/taos/taosadapter.toml)
...
[opentsdb_telnet]
enable = true
maxTCPConnections = 250
tcpKeepAlive = false
dbs = ["opentsdb_telnet", "collectd", "icinga2", "tcollector"]
ports = [6046, 6047, 6048, 6049]
user = "root"
password = "taosdata"
...
The default database name written by taosAdapter is collectd, but you can also modify the dbs item in the taosAdapter configuration file to specify a different name. Fill in user and password with the actual TDengine configuration values. After modifying the configuration file, taosAdapter needs to be restarted.
- You can also use taosAdapter command line parameters or set environment variables to start, to enable taosAdapter to receive collectd data, for more details please refer to the taosAdapter reference manual.
Configuring collectd
collectd uses a plugin mechanism that can write the collected monitoring data to different data storage software in various forms. TDengine supports direct collection plugins and write_tsdb plugins.
Configuring to receive direct collection plugin data
Modify the related configuration items in the collectd configuration file (default location /etc/collectd/collectd.conf).
LoadPlugin network
<Plugin network>
Server "<taosAdapter's host>" "<port for collectd direct>"
</Plugin>
Where <taosAdapter's host> should be filled with the domain name or IP address of the server running taosAdapter. <port for collectd direct> should be filled with the port used by taosAdapter to receive collectd data (default is 6045).
Example as follows:
LoadPlugin network
<Plugin network>
Server "127.0.0.1" "6045"
</Plugin>
Configuring write_tsdb plugin data
Modify the related configuration items in the collectd configuration file (default location /etc/collectd/collectd.conf).
LoadPlugin write_tsdb
<Plugin write_tsdb>
<Node>
Host "<taosAdapter's host>"
Port "<port for collectd write_tsdb plugin>"
...
</Node>
</Plugin>
Where <taosAdapter's host> should be filled with the domain name or IP address of the server running taosAdapter. <port for collectd write_tsdb plugin> should be filled with the port used by taosAdapter to receive collectd write_tsdb plugin data (default is 6047).
LoadPlugin write_tsdb
<Plugin write_tsdb>
<Node>
Host "127.0.0.1"
Port "6047"
HostTags "status=production"
StoreRates false
AlwaysAppendDS false
</Node>
</Plugin>
Then restart collectd:
systemctl restart collectd
Verification Method
Restart collectd
sudo systemctl restart collectd
Use TDengine CLI to verify that data is being written from collectd to TDengine and can be correctly read:
taos> show databases;
name |
=================================
information_schema |
performance_schema |
collectd |
Query OK, 3 row(s) in set (0.003266s)
taos> use collectd;
Database changed.
taos> show stables;
name |
=================================
load_1 |
memory_value |
df_value |
load_2 |
load_0 |
interface_1 |
irq_value |
interface_0 |
entropy_value |
swap_value |
Query OK, 10 row(s) in set (0.002236s)
taos> select * from collectd.memory_value limit 10;
ts | value | host | type_instance | type |
=========================================================================================================================================================
2022-04-20 09:27:45.459653462 | 54689792.000000000 | shuduo-1804 | buffered | memory |
2022-04-20 09:27:55.453168283 | 57212928.000000000 | shuduo-1804 | buffered | memory |
2022-04-20 09:28:05.453004291 | 57942016.000000000 | shuduo-1804 | buffered | memory |
2022-04-20 09:27:45.459653462 | 6381330432.000000000 | shuduo-1804 | free | memory |
2022-04-20 09:27:55.453168283 | 6357643264.000000000 | shuduo-1804 | free | memory |
2022-04-20 09:28:05.453004291 | 6349987840.000000000 | shuduo-1804 | free | memory |
2022-04-20 09:27:45.459653462 | 107040768.000000000 | shuduo-1804 | slab_recl | memory |
2022-04-20 09:27:55.453168283 | 107536384.000000000 | shuduo-1804 | slab_recl | memory |
2022-04-20 09:28:05.453004291 | 107634688.000000000 | shuduo-1804 | slab_recl | memory |
2022-04-20 09:27:45.459653462 | 309137408.000000000 | shuduo-1804 | used | memory |
Query OK, 10 row(s) in set (0.010348s)
- The default subtable names generated by TDengine are unique ID values generated according to rules.
