Skip to main content

TDengine Go Client Library

driver-go is the official Go language client library for TDengine. It implements the database/sql package, the generic Go language interface to SQL databases. Go developers can use it to develop applications that access TDengine cluster data.

Connection types

driver-go provides 3 connection types.

  • Native connection, which connects to TDengine instances natively through the TDengine client driver (taosc), supporting data writing, querying, subscriptions, schemaless writing, and bind interface.
  • REST connection, which is implemented through taosAdapter. Some features like schemaless and subscriptions are not supported.
  • Websocket connection which is implemented through taosAdapter. The set of features implemented by the WebSocket connection differs slightly from those implemented by the native connection.

For a detailed introduction of the connection types, please refer to: Establish Connection

Compatibility

Supports minimum Go version 1.14, it is recommended to use the latest Go version

Supported platforms

Native connections are supported on the same platforms as the TDengine client driver. REST connections are supported on all platforms that can run Go.

Version support

Please refer to version support list

Handling exceptions

If it is a TDengine error, you can get the error code and error information in the following ways.

// import "github.com/taosdata/driver-go/v3/errors"
if err != nil {
tError, is := err.(*errors.TaosError)
if is {
fmt.Println("errorCode:", int(tError.Code))
fmt.Println("errorMessage:", tError.ErrStr)
} else {
fmt.Println(err.Error())
}
}

TDengine DataType vs. Go DataType

TDengine DataTypeGo Type
TIMESTAMPtime.Time
TINYINTint8
SMALLINTint16
INTint32
BIGINTint64
TINYINT UNSIGNEDuint8
SMALLINT UNSIGNEDuint16
INT UNSIGNEDuint32
BIGINT UNSIGNEDuint64
FLOATfloat32
DOUBLEfloat64
BOOLbool
BINARYstring
NCHARstring
JSON[]byte

Note: Only TAG supports JSON types

Installation Steps

Pre-installation preparation

  • Install Go development environment (Go 1.14 and above, GCC 4.8.5 and above)
  • If you use the native connector, please install the TDengine client driver. Please refer to Install Client Driver for specific steps

Configure the environment variables and check the command.

  • go env
  • gcc -v

Install the client library

  1. Initialize the project with the go mod command.

    go mod init taos-demo
  2. Introduce taosSql

    import (
    "database/sql"
    _ "github.com/taosdata/driver-go/v3/taosSql"
    )
  3. Update the dependency packages with go mod tidy.

    go mod tidy
  4. Run the program with go run taos-demo or compile the binary with the go build command.

go run taos-demo
go build

Establishing a connection

Data source names have a standard format, e.g. PEAR DB, but no type prefix (square brackets indicate optionally):

[username[:password]@][protocol[(address)]]/[dbname][?param1=value1&...&paramN=valueN]

DSN in full form.

username:password@protocol(address)/dbname?param=value

taosRestful implements Go's database/sql/driver interface via http client. You can use the database/sql interface by simply introducing the driver.

Use taosRestful as driverName and use a correct DSN as dataSourceName with the following parameters supported by the DSN.

  • disableCompression whether to accept compressed data, default is true do not accept compressed data, set to false if transferring data using gzip compression.
  • readBufferSize The default size of the buffer for reading data is 4K (4096), which can be adjusted upwards when the query result has a lot of data.

For example:

package main

import (
"database/sql"
"fmt"

_ "github.com/taosdata/driver-go/v3/taosRestful"
)

func main() {
var taosUri = "root:taosdata@http(localhost:6041)/"
taos, err := sql.Open("taosRestful", taosUri)
if err != nil {
fmt.Println("failed to connect TDengine, err:", err)
return
}
}

Specify the URL and Properties to get the connection

The Go client library does not support this feature

Priority of configuration parameters

The Go client library does not support this feature

Usage examples

Create database and tables

var taosDSN = "root:taosdata@tcp(localhost:6030)/"
taos, err := sql.Open("taosSql", taosDSN)
if err != nil {
log.Fatalln("failed to connect TDengine, err:", err)
}
defer taos.Close()
_, err := taos.Exec("CREATE DATABASE power")
if err != nil {
log.Fatalln("failed to create database, err:", err)
}
_, err = taos.Exec("CREATE STABLE power.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)")
if err != nil {
log.Fatalln("failed to create stable, err:", err)
}

Insert data

package main

import (
"database/sql"
"fmt"
"log"

_ "github.com/taosdata/driver-go/v3/taosRestful"
)

func createStable(taos *sql.DB) {
_, err := taos.Exec("CREATE DATABASE power")
if err != nil {
log.Fatalln("failed to create database, err:", err)
}
_, err = taos.Exec("CREATE STABLE power.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)")
if err != nil {
log.Fatalln("failed to create stable, err:", err)
}
}

func insertData(taos *sql.DB) {
sql := `INSERT INTO power.d1001 USING power.meters TAGS('California.SanFrancisco', 2) VALUES ('2018-10-03 14:38:05.000', 10.30000, 219, 0.31000) ('2018-10-03 14:38:15.000', 12.60000, 218, 0.33000) ('2018-10-03 14:38:16.800', 12.30000, 221, 0.31000)
power.d1002 USING power.meters TAGS('California.SanFrancisco', 3) VALUES ('2018-10-03 14:38:16.650', 10.30000, 218, 0.25000)
power.d1003 USING power.meters TAGS('California.LosAngeles', 2) VALUES ('2018-10-03 14:38:05.500', 11.80000, 221, 0.28000) ('2018-10-03 14:38:16.600', 13.40000, 223, 0.29000)
power.d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES ('2018-10-03 14:38:05.000', 10.80000, 223, 0.29000) ('2018-10-03 14:38:06.500', 11.50000, 221, 0.35000)`
result, err := taos.Exec(sql)
if err != nil {
log.Fatalln("failed to insert, err:", err)
}
rowsAffected, err := result.RowsAffected()
if err != nil {
log.Fatalln("failed to get affected rows, err:", err)
}
fmt.Println("RowsAffected", rowsAffected)
}

func main() {
var taosDSN = "root:taosdata@http(localhost:6041)/"
taos, err := sql.Open("taosRestful", taosDSN)
if err != nil {
log.Fatalln("failed to connect TDengine, err:", err)
}
defer taos.Close()
createStable(taos)
insertData(taos)
}

view source code

Querying data

package main

import (
"database/sql"
"log"
"time"

_ "github.com/taosdata/driver-go/v3/taosRestful"
)

func main() {
var taosDSN = "root:taosdata@http(localhost:6041)/power"
taos, err := sql.Open("taosRestful", taosDSN)
if err != nil {
log.Fatalln("failed to connect TDengine, err:", err)
}
defer taos.Close()
rows, err := taos.Query("SELECT ts, current FROM meters LIMIT 2")
if err != nil {
log.Fatalln("failed to select from table, err:", err)
}

defer rows.Close()
for rows.Next() {
var r struct {
ts time.Time
current float32
}
err := rows.Scan(&r.ts, &r.current)
if err != nil {
log.Fatalln("scan error:\n", err)
return
}
log.Println(r.ts, r.current)
}
}

view source code

execute SQL with reqId

The reqId is very similar to TraceID in distributed tracing systems. In a distributed system, a request may need to pass through multiple services or modules to be completed. The reqId is used to identify and associate all related operations of this request, allowing us to track and understand the complete execution path of the request. Here are some primary usage of reqId:

  • Request Tracing: By associating the same reqId with all related operations of a request, we can trace the complete path of the request within the system.
  • Performance Analysis: By analyzing a request's reqId, we can understand the processing time of the request across various services or modules, thereby identifying performance bottlenecks.
  • Fault Diagnosis: When a request fails, we can identify the location of the issue by examining the reqId associated with that request.

If the user does not set a reqId, the client library will generate one randomly internally, but it is still recommended for the user to set it, as it can better associate with the user's request.

db, err := sql.Open("taosSql", "root:taosdata@tcp(localhost:6030)/")
if err != nil {
panic(err)
}
defer db.Close()
ctx := context.WithValue(context.Background(), common.ReqIDKey, common.GetReqID())
_, err = db.ExecContext(ctx, "create database if not exists example_taos_sql")
if err != nil {
panic(err)
}

Writing data via parameter binding

package main

import (
"time"

"github.com/taosdata/driver-go/v3/af"
"github.com/taosdata/driver-go/v3/common"
"github.com/taosdata/driver-go/v3/common/param"
)

func main() {
db, err := af.Open("", "root", "taosdata", "", 0)
if err != nil {
panic(err)
}
defer db.Close()
_, err = db.Exec("create database if not exists example_stmt")
if err != nil {
panic(err)
}
_, err = db.Exec("create table if not exists example_stmt.tb1(ts timestamp," +
"c1 bool," +
"c2 tinyint," +
"c3 smallint," +
"c4 int," +
"c5 bigint," +
"c6 tinyint unsigned," +
"c7 smallint unsigned," +
"c8 int unsigned," +
"c9 bigint unsigned," +
"c10 float," +
"c11 double," +
"c12 binary(20)," +
"c13 nchar(20)" +
")")
if err != nil {
panic(err)
}
stmt := db.InsertStmt()
err = stmt.Prepare("insert into example_stmt.tb1 values(?,?,?,?,?,?,?,?,?,?,?,?,?,?)")
if err != nil {
panic(err)
}
now := time.Now()
params := make([]*param.Param, 14)
params[0] = param.NewParam(2).
AddTimestamp(now, common.PrecisionMilliSecond).
AddTimestamp(now.Add(time.Second), common.PrecisionMilliSecond)
params[1] = param.NewParam(2).AddBool(true).AddNull()
params[2] = param.NewParam(2).AddTinyint(2).AddNull()
params[3] = param.NewParam(2).AddSmallint(3).AddNull()
params[4] = param.NewParam(2).AddInt(4).AddNull()
params[5] = param.NewParam(2).AddBigint(5).AddNull()
params[6] = param.NewParam(2).AddUTinyint(6).AddNull()
params[7] = param.NewParam(2).AddUSmallint(7).AddNull()
params[8] = param.NewParam(2).AddUInt(8).AddNull()
params[9] = param.NewParam(2).AddUBigint(9).AddNull()
params[10] = param.NewParam(2).AddFloat(10).AddNull()
params[11] = param.NewParam(2).AddDouble(11).AddNull()
params[12] = param.NewParam(2).AddBinary([]byte("binary")).AddNull()
params[13] = param.NewParam(2).AddNchar("nchar").AddNull()

paramTypes := param.NewColumnType(14).
AddTimestamp().
AddBool().
AddTinyint().
AddSmallint().
AddInt().
AddBigint().
AddUTinyint().
AddUSmallint().
AddUInt().
AddUBigint().
AddFloat().
AddDouble().
AddBinary(6).
AddNchar(5)
err = stmt.BindParam(params, paramTypes)
if err != nil {
panic(err)
}
err = stmt.AddBatch()
if err != nil {
panic(err)
}
err = stmt.Execute()
if err != nil {
panic(err)
}
err = stmt.Close()
if err != nil {
panic(err)
}
// select * from example_stmt.tb1
}

Schemaless Writing

import (
"fmt"

"github.com/taosdata/driver-go/v3/af"
)

func main() {
conn, err := af.Open("localhost", "root", "taosdata", "", 6030)
if err != nil {
fmt.Println("fail to connect, err:", err)
}
defer conn.Close()
_, err = conn.Exec("create database if not exists example")
if err != nil {
panic(err)
}
_, err = conn.Exec("use example")
if err != nil {
panic(err)
}
influxdbData := "st,t1=3i64,t2=4f64,t3=\"t3\" c1=3i64,c3=L\"passit\",c2=false,c4=4f64 1626006833639000000"
err = conn.InfluxDBInsertLines([]string{influxdbData}, "ns")
if err != nil {
panic(err)
}
telnetData := "stb0_0 1626006833 4 host=host0 interface=eth0"
err = conn.OpenTSDBInsertTelnetLines([]string{telnetData})
if err != nil {
panic(err)
}
jsonData := "{\"metric\": \"meter_current\",\"timestamp\": 1626846400,\"value\": 10.3, \"tags\": {\"groupid\": 2, \"location\": \"California.SanFrancisco\", \"id\": \"d1001\"}}"
err = conn.OpenTSDBInsertJsonPayload(jsonData)
if err != nil {
panic(err)
}
}

Schemaless with reqId

func (s *Schemaless) Insert(lines string, protocol int, precision string, ttl int, reqID int64) error

You can get the unique id by common.GetReqID().

Data Subscription

The TDengine Go client library supports subscription functionality with the following application API.

Create a Topic

    db, err := af.Open("", "root", "taosdata", "", 0)
if err != nil {
panic(err)
}
defer db.Close()
_, err = db.Exec("create database if not exists example_tmq WAL_RETENTION_PERIOD 86400")
if err != nil {
panic(err)
}
_, err = db.Exec("create topic if not exists example_tmq_topic as DATABASE example_tmq")
if err != nil {
panic(err)
}

Create a Consumer

    consumer, err := tmq.NewConsumer(&tmqcommon.ConfigMap{
"group.id": "test",
"auto.offset.reset": "latest",
"td.connect.ip": "127.0.0.1",
"td.connect.user": "root",
"td.connect.pass": "taosdata",
"td.connect.port": "6030",
"client.id": "test_tmq_client",
"enable.auto.commit": "false",
"msg.with.table.name": "true",
})
if err != nil {
panic(err)
}

Subscribe to consume data

    err = consumer.Subscribe("example_tmq_topic", nil)
if err != nil {
panic(err)
}
for i := 0; i < 5; i++ {
ev := consumer.Poll(500)
if ev != nil {
switch e := ev.(type) {
case *tmqcommon.DataMessage:
fmt.Printf("get message:%v\n", e)
case tmqcommon.Error:
fmt.Fprintf(os.Stderr, "%% Error: %v: %v\n", e.Code(), e)
panic(e)
}
consumer.Commit()
}
}

Assignment subscription Offset

    partitions, err := consumer.Assignment()
if err != nil {
panic(err)
}
for i := 0; i < len(partitions); i++ {
fmt.Println(partitions[i])
err = consumer.Seek(tmqcommon.TopicPartition{
Topic: partitions[i].Topic,
Partition: partitions[i].Partition,
Offset: 0,
}, 0)
if err != nil {
panic(err)
}
}

Close subscriptions

    err = consumer.Close()
if err != nil {
panic(err)
}

Full Sample Code

package main

import (
"fmt"
"os"
"time"

"github.com/taosdata/driver-go/v3/af"
"github.com/taosdata/driver-go/v3/af/tmq"
tmqcommon "github.com/taosdata/driver-go/v3/common/tmq"
)

func main() {
db, err := af.Open("", "root", "taosdata", "", 0)
if err != nil {
panic(err)
}
defer db.Close()
_, err = db.Exec("create database if not exists example_tmq WAL_RETENTION_PERIOD 86400")
if err != nil {
panic(err)
}
_, err = db.Exec("create topic if not exists example_tmq_topic as DATABASE example_tmq")
if err != nil {
panic(err)
}
consumer, err := tmq.NewConsumer(&tmqcommon.ConfigMap{
"group.id": "test",
"auto.offset.reset": "latest",
"td.connect.ip": "127.0.0.1",
"td.connect.user": "root",
"td.connect.pass": "taosdata",
"td.connect.port": "6030",
"client.id": "test_tmq_client",
"enable.auto.commit": "false",
"msg.with.table.name": "true",
})
if err != nil {
panic(err)
}
err = consumer.Subscribe("example_tmq_topic", nil)
if err != nil {
panic(err)
}
_, err = db.Exec("create table example_tmq.t1 (ts timestamp,v int)")
if err != nil {
panic(err)
}
go func() {
for {
_, err = db.Exec("insert into example_tmq.t1 values(now,1)")
if err != nil {
panic(err)
}
time.Sleep(time.Millisecond * 100)
}
}()

for i := 0; i < 5; i++ {
ev := consumer.Poll(500)
if ev != nil {
switch e := ev.(type) {
case *tmqcommon.DataMessage:
fmt.Printf("get message:%v\n", e)
case tmqcommon.Error:
fmt.Fprintf(os.Stderr, "%% Error: %v: %v\n", e.Code(), e)
panic(e)
}
consumer.Commit()
}
}
partitions, err := consumer.Assignment()
if err != nil {
panic(err)
}
for i := 0; i < len(partitions); i++ {
fmt.Println(partitions[i])
err = consumer.Seek(tmqcommon.TopicPartition{
Topic: partitions[i].Topic,
Partition: partitions[i].Partition,
Offset: 0,
}, 0)
if err != nil {
panic(err)
}
}

partitions, err = consumer.Assignment()
if err != nil {
panic(err)
}
for i := 0; i < len(partitions); i++ {
fmt.Println(partitions[i])
}

err = consumer.Close()
if err != nil {
panic(err)
}
}

More sample programs

Frequently Asked Questions

  1. bind interface in database/sql crashes

    REST does not support parameter binding related interface. It is recommended to use db.Exec and db.Query.

  2. error [0x217] Database not specified or available after executing other statements with use db statement

    The execution of SQL command in the REST interface is not contextual, so using use db statement will not work, see the usage restrictions section above.

  3. use taosSql without error but use taosRestful with error [0x217] Database not specified or available

    Because the REST interface is stateless, using the use db statement will not take effect. See the usage restrictions section above.

  4. readBufferSize parameter has no significant effect after being increased

    Increasing readBufferSize will reduce the number of syscall calls when fetching results. If the query result is smaller, modifying this parameter will not improve performance significantly. If you increase the parameter value too much, the bottleneck will be parsing JSON data. If you need to optimize the query speed, you must adjust the value based on the actual situation to achieve the best query performance.

  5. disableCompression parameter is set to false when the query efficiency is reduced

    When set disableCompression parameter to false, the query result will be compressed by gzip and then transmitted, so you have to decompress the data by gzip after getting it.

  6. go get command can't get the package, or timeout to get the package

    Set Go proxy go env -w GOPROXY=https://goproxy.cn,direct.

API Reference

Full API see driver-go documentation