Advanced Security Options

In addition to traditional user and permission management, TDengine also offers other security strategies such as IP whitelisting, audit logs, data encryption, etc., which are unique features of TDengine Enterprise. The whitelisting feature was first released in version 3.2.0.0, audit logs in version 3.1.1.0, and database encryption in version 3.3.0.0. It is recommended to use the latest version.

IP Whitelisting

IP whitelisting is a network security technology that allows IT administrators to control "who" can access the system and resources, enhancing the security of database access and preventing external malicious attacks. IP whitelisting works by creating a list of trusted IP addresses, assigning them as unique identifiers to users, and only allowing these IP addresses to access the target server. Please note that user permissions and IP whitelisting are managed separately. Below are the specific methods for configuring IP whitelisting.

The SQL to add an IP whitelist is as follows.

create user test pass password [sysinfo value] [host host_name1[,host_name2]] 
alter user test add host host_name1

The SQL to query the IP whitelist is as follows.

SELECT TEST, ALLOWED_HOST FROM INS_USERS;
SHOW USERS;

The command to delete from the IP whitelist is as follows.

ALTER USER TEST DROP HOST HOST_NAME1

Notes

• Both the open-source and enterprise versions can add successfully and can be queried, but the open-source version does not impose any restrictions on IPs.
• create user u_write pass 'taosdata1' host 'iprange1','iprange2', multiple ipranges can be added at once, the server will deduplicate, the logic for deduplication requires the ipranges to be exactly the same
• By default, 127.0.0.1 is added to the whitelist and can be queried in the whitelist
• The IP set of cluster nodes is automatically added to the whitelist, but cannot be queried.
• When taosadaper and taosd are not on the same machine, the taosadaper IP needs to be manually added to the taosd whitelist
• In a cluster situation, all nodes should have enableWhiteList set the same, either all false or all true, otherwise the cluster cannot start
• Whitelist changes take effect within 1s, no more than 2s, each change has a slight impact on transmission performance (one more judgment, which can be ignored), after the change, the impact is negligible, and there is no impact on the cluster during the change process, nor on the clients currently accessing (assuming their IPs are included in the whitelist)
• If adding two ip ranges, 192.168.1.1/16 (suppose A), 192.168.1.1/24 (suppose B), strictly speaking, A includes B, but considering the situation too complicated, A and B will not be merged
• When deleting, it must match exactly. That is, if added as 192.168.1.1/24, it must also be deleted as 192.168.1.1/24
• Only root has the permission to add or delete ip white list for other users
• Compatible with previous versions, but does not support rolling back from the current version to a previous version
• x.x.x.x/32 and x.x.x.x are considered the same iprange, displayed as x.x.x.x
• If the client gets 0.0.0.0/0, it means the whitelist is not enabled
• If the whitelist changes, the client will detect it in the heartbeat.
• For one user, the maximum number of IPs that can be added is 2048

Audit Logs

TDengine first records and manages user operations, then sends these as audit logs to taosKeeper, which then saves them to any TDengine cluster. Administrators can use audit logs for security monitoring and historical tracing. Enabling and disabling TDengine's audit log feature is very simple, just modify the TDengine configuration file and restart the service. The configuration details for audit logs are as follows.

taosd Configuration

Audit logs are generated by the database service taosd, and its relevant parameters need to be configured in the taos.cfg configuration file, detailed parameters are as follows.

Parameter Name	Parameter Meaning
audit	Whether to enable audit logs, default value is 0. 1 for enabled, 0 for disabled
monitorFqdn	The FQDN of the taosKeeper server that receives the audit logs
monitorPort	The port used by the taosKeeper service that receives the audit logs
monitorCompaction	Whether to compress data when reporting

taosKeeper Configuration

Configure the related parameters for audit logs in the taosKeeper configuration file keeper.toml, as shown in the table below

Parameter Name	Description
auditDB	The name of the database used to store audit logs, default is "audit". taosKeeper will check if this database exists upon receiving audit logs, and will create it automatically if it does not exist.

Data Format

The format of the reported audit logs is as follows

{
    "ts": timestamp,
    "cluster_id": string,
    "user": string,
    "operation": string,
    "db": string,
    "resource": string,
    "client_add": string,
    "details": string
}

Table Structure

taosKeeper will automatically create a supertable in the corresponding database based on the reported audit data to store the data. The definition of this supertable is as follows

CREATE STABLE operations(ts timestamp, details VARCHAR(64000), User VARCHAR(25), Operation VARCHAR(20), db VARCHAR(65), resource VARCHAR(193), client_add(25)) TAGS (clusterID VARCHAR(64));

Where:

1. db refers to the database involved in the operation, resource refers to the resource involved.
2. User and Operation are data columns, indicating which user performed what operation on the object.
3. timestamp is the timestamp column, indicating the time when the operation occurred.
4. details provide additional details of the operation, which in most cases is the SQL statement executed.
5. client_add is the client address, including IP and port.

Operation List

The current list of operations recorded in the audit logs and the meanings of each field in each operation are shown in the table below (Note: since the user field, timestamp, and client_add have the same meaning in all operations, they are not included in the table below)

Operation	Operation	DB	Resource	Details
create database	createDB	db name	NULL	SQL
alter database	alterDB	db name	NULL	SQL
drop database	dropDB	db name	NULL	SQL
create stable	createStb	db name	stable name	SQL
alter stable	alterStb	db name	stable name	SQL
drop stable	dropStb	db name	stable name	SQL
create user	createUser	NULL	username being created	User attribute parameters, (excluding password)
alter user	alterUser	NULL	username being modified	Password change logs the modified parameters and new values (excluding password); other operations log SQL
drop user	dropUser	NULL	username being deleted	SQL
create topic	createTopic	topic's DB	name of the created topic	SQL
drop topic	dropTopic	topic's DB	name of the deleted topic	SQL
create dnode	createDnode	NULL	IP:Port or FQDN:Port	SQL
drop dnode	dropDnode	NULL	dnodeId	SQL
alter dnode	alterDnode	NULL	dnodeId	SQL
create mnode	createMnode	NULL	dnodeId	SQL
drop mnode	dropMnode	NULL	dnodeId	SQL
create qnode	createQnode	NULL	dnodeId	SQL
drop qnode	dropQnode	NULL	dnodeId	SQL
login	login	NULL	NULL	appName
create stream	createStream	NULL	name of the created stream	SQL
drop stream	dropStream	NULL	name of the deleted stream	SQL
grant privileges	grantPrivileges	NULL	user granted	SQL
revoke privileges	revokePrivileges	NULL	user whose privileges were revoked	SQL
compact database	compact	database name	NULL	SQL
balance vgroup leader	balanceVgroupLead	NULL	NULL	SQL
restore dnode	restoreDnode	NULL	dnodeId	SQL
redistribute vgroup	redistributeVgroup	NULL	vgroupId	SQL
balance vgroup	balanceVgroup	NULL	vgroupId	SQL
create table	createTable	db name	NULL	table name
drop table	dropTable	db name	NULL	table name

View Audit Logs

After both taosd and taosKeeper are correctly configured and started, as the system continues to operate, various operations (as shown in the table above) will be recorded and reported in real-time. Users can log in to taosExplorer, click on "System Management" → "Audit" page to view the audit logs; they can also directly query the relevant databases and tables in the TDengine CLI.

Data Encryption

TDengine supports Transparent Data Encryption (TDE), which encrypts static data files to prevent potential attackers from bypassing the database and directly reading sensitive information from the file system. The database access program is completely unaware, and applications do not need to make any modifications or compilations to be directly applied to the encrypted database, supporting encryption algorithms such as the national standard SM4. In transparent encryption, database key management and database encryption scope are two of the most important topics. TDengine encrypts the database key with a machine code, which is stored locally rather than in a third-party manager. When data files are copied to another machine, the change in machine code prevents access to the database key, naturally preventing access to the data files. TDengine encrypts all data files, including write-ahead log files, metadata files, and time-series data files. After encryption, the data compression ratio remains unchanged, and both write and query performance only slightly decrease.

Configure Keys

Key configuration is divided into offline and online settings.

Method one, offline setting. The following command can configure keys separately for each node.

taosd -y {encryptKey}

Method two, online setting. When all nodes in the cluster are online, keys can be configured using SQL, as follows.

create encrypt_key {encryptKey};

The online setting method requires that all nodes that have joined the cluster have not used the offline setting method to generate keys, otherwise the online setting method will fail. Successfully setting the key online also automatically loads and uses the key.

Create Encrypted Database

TDengine supports creating encrypted databases through SQL, as follows.

create database [if not exists] db_name [database_options]
database_options:
 database_option ...
database_option: {
 encrypt_algorithm {'none' |'sm4'}
}

The main parameters are explained as follows. encrypt_algorithm: Specifies the encryption algorithm used for the data. The default is none, meaning no encryption is used. sm4 indicates the SM4 encryption algorithm.

View Encryption Configuration

Users can query the current encryption configuration of the database through the system database ins_databases, SQL as follows.

select name, `encrypt_algorithm` from ins_databases;
              name              | encrypt_algorithm |
=====================================================
 power1                         | none              |
 power                          | sm4               |

View Node Key Status

The following SQL command can be used to view the node key status:

show encryptions;

select * from information_schema.ins_encryptions;
  dnode_id   |           key_status           |
===============================================
           1 | loaded                         |
           2 | unset                          |
           3 | unknown                        |

key_status has three values:

• When the node has not set a key, the status column shows unset.
• When the key is verified successfully and loaded, the status column shows loaded.
• When the node is not started, and the status of the key cannot be detected, the status column shows unknown.

Update Key Configuration

When the hardware configuration of a node changes, the following command is used to update the key, the same as the command for offline key configuration:

taosd -y  {encryptKey}

Updating the key configuration requires stopping taosd first, and using the exact same key, meaning the key cannot be changed after the database is created.