OpenTSDB
The features or components discussed in this document are available in TDengine TSDB-Enterprise only. TDengine TSDB-OSS does not include these features or components.
This section describes how to create a data migration task through the Explorer interface to migrate data from OpenTSDB to the current TDengine cluster.
Overview
OpenTSDB is a real-time monitoring information collection and display platform built on the HBase system. TDengine can efficiently read data from OpenTSDB through the OpenTSDB connector and write it into TDengine, achieving historical data migration or real-time data synchronization.
During the operation, the task will save progress information to the disk, so if the task is paused and restarted, or automatically recovers from an anomaly, it will not start over. For more options, it is recommended to read the explanations of each form field on the task creation page in detail.
Procedure
Add a Data Source
- In a web browser, open TDengine TSDB Explorer. This interface is located on port 6060 on the hostname or IP address running TDengine TSDB.
- In the main menu on the left, click Data In and then click Add Task.
- In the Name field, enter a unique name for the data ingestion task.
- From the Type drop-down list, select OpenTSDB.
- (Optional) If this data ingestion task requires an agent, select the agent from the Agent drop-down list. You can also click Create New Agent to create the agent now.
For more information, see Install taosX-Agent.
- From the Target DB drop-down list, select the database in which you want to store data ingested by this task. You can also click Create Database to create the database now.
Configure Connection Information
Fill in the connection information for the source OpenTSDB database in the Connection Configuration area, as shown below:
Below the Connection Configuration area, there is a Connectivity Check button. Users can click this button to check whether the information filled in above can normally access the data from the source OpenTSDB database. The check results are shown below:
Configure Task Information
Metrics are the physical quantities in which data is stored in the OpenTSDB database. Users can specify multiple metrics to synchronize, or synchronize all data in the database if none are specified. If users specify metrics, they need to first click the Get Metrics button on the right to obtain all the metric information from the current source OpenTSDB database, and then select from the dropdown menu, as shown below:
Start Time refers to the start time of the data in the source OpenTSDB database, using the timezone selected in explorer, and this field is required.
End Time refers to the end time of the data in the source OpenTSDB database. If no end time is specified, the synchronization of the latest data will continue; if an end time is specified, synchronization will only continue up to this end time, using the timezone selected in explorer, and this field is optional.
Time Range per Read (minutes) is the maximum time range for the connector to read data from the source OpenTSDB database in a single operation. This is a very important parameter, and users need to decide based on server performance and data storage density. If the range is too small, the execution speed of the synchronization task will be very slow; if the range is too large, it may cause the OpenTSDB database system to fail due to excessive memory usage.
Delay (seconds) is an integer ranging from 1 to 30. To eliminate the impact of out-of-order data, TDengine always waits for the duration specified here before reading the data.
Configure Advanced Options
The Advanced Options area is collapsed by default. Click the > on the right to expand it, as shown in the following images:
Completion of Creation
Click the Submit button to complete the creation of the OpenTSDB to TDengine data synchronization task. Return to the Data Source List page to view the status of the task.