Integrate with Data Collection Tools
This section explains how to write data from common monitoring collectors, message queues, and stream-processing frameworks to TDengine, or synchronize data through connectors. For zero-code Kafka, MQTT, and other ingestion options in TDengine Enterprise, see Zero-Code Data Ingestion. For the Enterprise Flink Source/CDC and Sink connectors, see Flink under Data Distribution.
๐๏ธ Prometheus
Prometheus is a popular open-source monitoring and alerting system. In 2016, Prometheus joined the Cloud Native Computing Foundation (CNCF), becoming the second hosted project after Kubernetes. The project has a very active developer and user community.
๐๏ธ Telegraf
Telegraf is a very popular open-source metric collection software. In data collection and platform monitoring systems, Telegraf can collect operational information from various components without the need to manually write scripts for periodic collection, reducing the difficulty of data acquisition.
๐๏ธ collectd
collectd is a daemon for collecting system performance. collectd provides various storage mechanisms to store different values. It periodically collects relevant statistical information about the system while it is running and storing information. Utilizing this information helps identify current system performance bottlenecks and predict future system loads.
๐๏ธ StatsD
StatsD is a simple daemon for aggregating and summarizing application metrics that has rapidly evolved in recent years into a unified protocol for collecting application performance metrics.
๐๏ธ Icinga2
icinga2 is an open-source host and network monitoring software, originally developed from the Nagios network monitoring application. Currently, icinga2 is released under the GNU GPL v2 license.
๐๏ธ TCollector
TCollector is part of openTSDB, used for collecting client logs and sending them to the database.
๐๏ธ EMQX Platform
MQTT is a popular IoT data transmission protocol, and EMQX is an open-source MQTT Broker software. Without any coding, you can directly write MQTT data into TDengine by simply configuring "rules" in the EMQX Dashboard. EMQX supports saving data to TDengine by sending it to a web service and also provides a native TDengine driver in the enterprise version for direct saving.
๐๏ธ HiveMQ Broker
Write data from HiveMQ Broker to TDengine
๐๏ธ Kafka Connect
The TDengine Kafka Connector includes two plugins: TDengine Source Connector and TDengine Sink Connector. Users only need to provide a simple configuration file to synchronize data from a specified topic in Kafka to TDengine, or synchronize data from a specified database in TDengine to Kafka, either in batches or in real-time.
๐๏ธ Flink
Apache Flink is an open-source distributed stream batch integrated processing framework supported by the Apache Software Foundation, which can be used for many big data processing scenarios such as stream processing, batch processing, complex event processing, real-time data warehouse construction, and providing real-time data support for machine learning. At the same time, Flink has a wealth of connectors and various tools that can interface with numerous different types of data sources to achieve data reading and writing. In the process of data processing, Flink also provides a series of reliable fault-tolerant mechanisms, effectively ensuring that tasks can run stably and continuously even in the event of unexpected situations.