Skip to main content

Data Compression

Starting from version 3.3.0.0, TDengine provides more advanced compression features, allowing users to configure whether to compress each column at the time of table creation, as well as the compression algorithm and compression level used.

Compression Terminology Definition

Compression Stages

  • First-level compression: Encoding the data, which is essentially a form of compression.
  • Second-level compression: Compressing data blocks on top of encoding.

Compression Levels

In this document, it specifically refers to the internal levels of the second-level compression algorithms, such as zstd, with at least 8 levels available, each level having different performances. Essentially, it's a tradeoff between compression ratio, compression speed, and decompression speed. To avoid difficulty in choosing, it is simplified into the following three levels:

  • high: Highest compression ratio, relatively worst compression and decompression speeds.
  • low: Best compression and decompression speeds, relatively lowest compression ratio.
  • medium: Balances compression ratio, compression speed, and decompression speed.

List of Compression Algorithms

  • Encoding algorithms list (First-level compression): simple8b, bit-packing, delta-i, delta-d, disabled

  • Compression algorithms list (Second-level compression): lz4, zlib, zstd, tsz, xz, disabled

  • Default compression algorithms list and applicable range for each data type

Data TypeAvailable Encoding AlgorithmsDefault Encoding AlgorithmAvailable Compression AlgorithmsDefault Compression AlgorithmDefault Compression Level
int/uintdisabled/simple8bsimple8blz4/zlib/zstd/xzlz4medium
tinyint/untinyint/smallint/usmallintdisabled/simple8bsimple8blz4/zlib/zstd/xzzlibmedium
bigint/ubigint/timestampdisabled/simple8b/delta-idelta-ilz4/zlib/zstd/xzlz4medium
float/doubledisabled/delta-ddelta-dlz4/zlib/zstd/xz/tszlz4medium
binary/nchardisableddisabledlz4/zlib/zstd/xzzstdmedium
booldisabled/bit-packingbit-packinglz4/zlib/zstd/xzzstdmedium

SQL Syntax

Specifying Compression When Creating Tables

CREATE [dbname.]tabname (colName colType [ENCODE 'encode_type'] [COMPRESS 'compress_type' [LEVEL 'level'], [, other create_definition]...])

Parameter Description

  • tabname: Name of the supertable or basic table
  • encode_type: First-level compression, see the list above
  • compress_type: Second-level compression, see the list above
  • level: Specifically refers to the level of second-level compression, default is medium, supports abbreviation as 'h'/'l'/'m'

Function Description

  • Specify the compression method for columns when creating a table

Changing the Compression Method of a Column

ALTER TABLE [db_name.]tabName MODIFY COLUMN colName [ENCODE 'ecode_type'] [COMPRESS 'compress_type'] [LEVEL "high"]

Parameter Description

  • tabName: Table name, can be a supertable or a basic table
  • colName: Column for which the compression algorithm is to be changed, can only be a normal column

Function Description

  • Change the compression method of a column

Viewing the Compression Method of a Column

DESCRIBE [dbname.]tabName

Function Description

  • Displays basic information of the column, including type and compression method

Compatibility

  • Fully compatible with existing data
  • Cannot revert to a lower version after upgrading to 3.3.0.0