TDengine Python Client Library

taospy is the official Python client library for TDengine. taospy wraps the REST interface of TDengine. Additionally taospy provides a set of programming interfaces that conforms to the Python Data Access Specification (PEP 249). It is easy to integrate taospy with many third-party tools, such as SQLAlchemy and pandas.

The source code for the Python client library is hosted on GitHub.

Connection types

taospy mainly provides 3 connection types. TDengine Cloud can be accessed by both REST and WebSocket connections.

• Native connection, which correspond to the taos modules of the taospy package, connects to TDengine instances natively through the TDengine client driver (taosc), supporting data writing, querying, subscriptions, schemaless writing, and bind interface.
• REST connection, which correspond to the taosrest modules of the taospy package, which is implemented through taosAdapter. Some features like schemaless and subscriptions are not supported.
• Websocket connection taos-ws-py is an optional package to enable using WebSocket to connect TDengine, which is implemented through taosAdapter. The set of features implemented by the WebSocket connection differs slightly from those implemented by the native connection.

IMPORTANT

1. The direct connection to the server using the native interface provided by the client driver is referred to hereinafter as a "native connection".
2. The connection to the server using the REST or WebSocket interface provided by taosAdapter is referred to hereinafter as a "REST connection" or "WebSocket connection".

For detailed information on how to establish a connection, please refer to: Programming - Connect - Python.

Installation

Preparation

1. Install Python. Python >= 3.6 is recommended. If Python is not available on your system, refer to the Python BeginnersGuide to install it.
2. Install pip. In most cases, the Python installer comes with the pip utility. If not, please refer to pip documentation to install it.

Install via pip

pip3 install -U taospy[ws]

Install vial conda

conda install -c conda-forge taospy taospyws

Installation verification

Verifying that the taosrest module can be imported successfully. This can be done in the Python Interactive Shell by typing.

import taosrest

Establish connection

import os

url = os.environ["TDENGINE_CLOUD_URL"]
token = os.environ["TDENGINE_CLOUD_TOKEN"]

conn = taosrest.connect(url=url, token=token)
# test the connection by getting version info
print("server version:", conn.server_info)

view source code

All arguments to the connect() function are optional keyword arguments. The following are the connection parameters specified.

• url： The cloud URL.
• token: The cloud token.
• timeout: HTTP request timeout in seconds. The default is socket._GLOBAL_DEFAULT_TIMEOUT. Usually, no configuration is needed.

Sample program

Use of TaosRestConnection Class

import taosrest
import os

url = os.environ["TDENGINE_CLOUD_URL"]
token = os.environ["TDENGINE_CLOUD_TOKEN"]

conn = taosrest.connect(url=url, token=token)
# test the connection by getting version info
print("server version:", conn.server_info)
affected_row = conn.execute("CREATE STABLE IF NOT EXISTS power.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)")
print("affected_row", affected_row)  # 0
affected_row = conn.execute("""INSERT INTO 
    power.d1001 USING power.meters TAGS('California.SanFrancisco', 2) VALUES ('2018-10-03 14:38:05.000', 10.30000, 219, 0.31000) ('2018-10-03 14:38:15.000', 12.60000, 218, 0.33000) ('2018-10-03 14:38:20.800', 12.30000, 221, 0.31000)
    power.d1002 USING power.meters TAGS('California.SanFrancisco', 3) VALUES ('2018-10-03 14:39:19.650', 10.30000, 218, 0.25000)
    power.d1003 USING power.meters TAGS('California.LosAngeles', 2) VALUES ('2018-10-03 14:41:05.500', 11.80000, 221, 0.28000) ('2018-10-03 14:42:16.600', 13.40000, 223, 0.29000)
    power.d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES ('2018-10-03 14:48:05.000', 10.80000, 223, 0.29000) ('2018-10-03 14:55:06.500', 11.50000, 221, 0.35000)
               """)
print("affected_row", affected_row)  # 8

result = conn.query("SELECT ts, current FROM power.meters LIMIT 2")

print("metadata of each column:\n", result.fields)  
# [{'name': 'ts', 'type': 'TIMESTAMP', 'bytes': 8}, {'name': 'current', 'type': 'FLOAT', 'bytes': 4}]

print("total rows:", result.rows)  # 2

# Iterate over result.
for row in result:
    print(row)
# output:
# [datetime.datetime(2018, 10, 3, 14, 38, 5), 10.3]
# [datetime.datetime(2018, 10, 3, 14, 38, 15), 12.6]

# Or get all rows as a list
print(result.data)  
# [[datetime.datetime(2018, 10, 3, 14, 38, 5), 10.3], [datetime.datetime(2018, 10, 3, 14, 38, 15), 12.6]]

view source code

Use of TaosRestCursor Class

The TaosRestCursor class is an implementation of the PEP249 Cursor interface.

import taosrest
import os

url = os.environ["TDENGINE_CLOUD_URL"]
token = os.environ["TDENGINE_CLOUD_TOKEN"]

conn = taosrest.connect(url=url, token=token)
# test the connection by getting version info
print("server version:", conn.server_info)
from taosrest import TaosRestCursor
# create STable
cursor: TaosRestCursor = conn.cursor()
cursor.execute("CREATE STABLE IF NOT EXISTS power.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)")

# insert data
cursor.execute("""INSERT INTO 
    power.d1001 USING power.meters TAGS('California.SanFrancisco', 2) VALUES ('2018-10-03 14:38:05.000', 10.30000, 219, 0.31000) ('2018-10-03 14:38:15.000', 12.60000, 218, 0.33000) ('2018-10-03 14:38:20.800', 12.30000, 221, 0.31000)
    power.d1002 USING power.meters TAGS('California.SanFrancisco', 3) VALUES ('2018-10-03 14:39:19.650', 10.30000, 218, 0.25000)
    power.d1003 USING power.meters TAGS('California.LosAngeles', 2) VALUES ('2018-10-03 14:41:05.500', 11.80000, 221, 0.28000) ('2018-10-03 14:42:16.600', 13.40000, 223, 0.29000)
    power.d1004 USING power.meters TAGS('California.LosAngeles', 3) VALUES ('2018-10-03 14:48:05.000', 10.80000, 223, 0.29000) ('2018-10-03 14:55:06.500', 11.50000, 221, 0.35000)
               """)
print("inserted row count:", cursor.rowcount)

# query data
cursor.execute("SELECT * FROM power.meters LIMIT 3")
# get total rows
print("queried row count:", cursor.rowcount)
# get column names from cursor
column_names = [meta[0] for meta in cursor.description]
# get rows
data: list[tuple] = cursor.fetchall()
print(column_names)
for row in data:
    print(row)

# output:
# inserted row count: 8
# queried row count: 3
# ['ts', 'current', 'voltage', 'phase', 'location', 'groupid']
# [datetime.datetime(2018, 10, 3, 14, 38, 5), 10.3, 219, 0.31, 'California.SanFrancisco', 2]
# [datetime.datetime(2018, 10, 3, 14, 38, 15), 12.6, 218, 0.33, 'California.SanFrancisco', 2]
# [datetime.datetime(2018, 10, 3, 14, 38, 20, 800000), 12.3, 221, 0.31, 'California.SanFrancisco', 2]

view source code

• cursor.execute : Used to execute arbitrary SQL statements.
• cursor.rowcount : For write operations, returns the number of successful rows written. For query operations, returns the number of rows in the result set.
• cursor.description : Returns the description of the field. Please refer to TaosRestCursor for the specific format of the description information.

Use of the RestClient class

The RestClient class is a direct wrapper for the REST API. It contains only a sql() method for executing arbitrary SQL statements and returning the result.

import os
from taosrest import RestClient

url = os.environ["TDENGINE_CLOUD_URL"]
token = os.environ["TDENGINE_CLOUD_TOKEN"]

client = RestClient(url, token)
res: dict = client.sql("SELECT ts, current FROM power.meters LIMIT 1")
print(res)

# output:
# {'status': 'succ', 'head': ['ts', 'current'], 'column_meta': [['ts', 9, 8], ['current', 6, 4]], 'data': [[datetime.datetime(2018, 10, 3, 14, 38, 5, tzinfo=datetime.timezone(datetime.timedelta(seconds=28800), '+08:00')), 10.3]], 'rows': 1}

view source code

For a more detailed description of the sql() method, please refer to RestClient.

Other notes

Exception handling

All errors from database operations are thrown directly as exceptions and the error message from the database is passed up the exception stack. The application is responsible for exception handling. For example:

import taos

try:
    conn = taos.connect()
    conn.execute("CREATE TABLE 123")  # wrong sql
except taos.Error as e:
    print(e)
    print("exception class: ", e.__class__.__name__)
    print("error number:", e.errno)
    print("error message:", e.msg)
except BaseException as other:
    print("exception occur")
    print(other)

# output:
# [0x0216]: syntax error near 'Incomplete SQL statement'
# exception class:  ProgrammingError
# error number: -2147483114
# error message: syntax error near 'Incomplete SQL statement'

view source code

About nanoseconds

Due to the current imperfection of Python's nanosecond support (see link below), the current implementation returns integers at nanosecond precision instead of the datetime type produced by ms and us, which application developers will need to handle on their own. And it is recommended to use pandas' to_datetime(). The Python client library may modify the interface in the future if Python officially supports nanoseconds in full.

1. https://stackoverflow.com/questions/10611328/parsing-datetime-strings-containing-nanoseconds
2. https://www.python.org/dev/peps/pep-0564/

Important Update

Release Notes

API Reference

• taos
• taosrest