Integration With Perspective

Perspective is an open-source and powerful data visualization library developed by Prospective.co. Leveraging the technologies of WebAssembly and Web Workers, it enables interactive real-time data analysis in web applications and provides high-performance visualization capabilities on the browser side. With its help, developers can build dashboards, charts, etc. that update in real time, and users can easily interact with the data, filtering, sorting, and exploring it as needed. It boasts high flexibility, adapting to various data formats and business scenarios. It is also fast, ensuring smooth interaction even when dealing with large-scale data. Moreover, it has excellent usability, allowing both beginners and professional developers to quickly build visualization interfaces.

In terms of data connection, Perspective, through the Python connector of TDengine, perfectly supports TDengine data sources. It can efficiently retrieve various types of data, such as massive time-series data, from TDengine. Additionally, it offers real-time functions including the display of complex charts, in-depth statistical analysis, and trend prediction, helping users gain insights into the value of the data and providing strong support for decision-making. It is an ideal choice for building applications with high requirements for real-time data visualization and analysis.

Prerequisites

Perform the following installation operations in the Linux system:

• TDengine is installed and running normally (both Enterprise and Community versions are available).
• taosAdapter is running normally, refer to taosAdapter Reference.
• Python version 3.10 or higher has been installed (if not installed, please refer to Python Installation).
• Download or clone the perspective-connect-demo project. After entering the root directory of the project, run the "install.sh" script to download and install the TDengine client library and related dependencies locally.

Visualize data

Step 1, Run the "run.sh" script in the root directory of the perspective-connect-demo project to start the Perspective service. This service will retrieve data from the TDengine database every 300 milliseconds and transmit the data in a streaming form to the web-based Perspective Viewer.

sh run.sh

Step 2, Start a static web service. Then, access the prsp-viewer.html resource in the browser, and the visualized data can be displayed.

python -m http.server 8081

The effect presented after accessing the web page through the browser is shown in the following figure:

Instructions for use

Write Data to TDengine

The producer.py script in the root directory of the perspective-connect-demo project can periodically insert data into the TDengine database with the help of the TDengine Python connector. This script will generate random data and insert it into the database, thus simulating the process of writing real-time data. The specific execution steps are as follows:

1. Establish a connection to TDengine.
2. Create the power database and the meters table.
3. Generate random data every 300 milliseconds and write it into the TDengine database.

For detailed instructions on writing using the Python connector, please refer to Python Parameter Binding.

Load Data from TDengine

The perspective_server.py script in the root directory of the perspective-connect-demo project will start a Perspective server. This server will read data from TDengine and stream the data to a Perspective table via the Tornado WebSocket.

1. Start a Perspective server.
2. Establish a connection to TDengine.
3. Create a Perspective table (the table structure needs to match the type of the table in the TDengine database).
4. Call the Tornado.PeriodicCallback function to start a scheduled task, thereby achieving the update of the data in the Perspective table. The sample code is as follows:

def perspective_thread(perspective_server: perspective.Server, tdengine_conn: taosws.Connection):
    """
    Create a new Perspective table and update it with new data every 50ms
    """
    # create a new Perspective table
    client = perspective_server.new_local_client()
    schema = {
        "timestamp": datetime,
        "location": str,
        "groupid": int,
        "current": float,
        "voltage": int,
        "phase": float,
    }
    # define the table schema
    table = client.table(
        schema,
        limit=1000,                     # maximum number of rows in the table
        name=PERSPECTIVE_TABLE_NAME,    # table name. Use this with perspective-viewer on the client side
    )
    logger.info("Created new Perspective table")

    # update with new data
    def updater():
        data = read_tdengine(tdengine_conn)
        table.update(data)
        logger.debug(f"Updated Perspective table: {len(data)} rows")

    logger.info(f"Starting tornado ioloop update loop every {PERSPECTIVE_REFRESH_RATE} milliseconds")
    # start the periodic callback to update the table data
    callback = tornado.ioloop.PeriodicCallback(callback=updater, callback_time=PERSPECTIVE_REFRESH_RATE)
    callback.start()

view source code

HTML Page Configuration

The prsp-viewer.html file in the root directory of the perspective-connect-demo project embeds the Perspective Viewer into the HTML page. It connects to the Perspective server via a WebSocket and displays real-time data according to the chart configuration.

• Configure the displayed charts and the rules for data analysis.
• Establish a Websocket connection with the Perspective server.
• Import the Perspective library, connect to the Perspective server via a WebSocket, and load the meters_values table to display dynamic data.

<script type="module">
    // import the Perspective library
    import perspective from "https://unpkg.com/@finos/perspective@3.1.3/dist/cdn/perspective.js";

    document.addEventListener("DOMContentLoaded", async function () {
        // an asynchronous function for loading the view
        async function load_viewer(viewerId, config) {
            try {
                const table_name = "meters_values";
                const viewer = document.getElementById(viewerId);
                // connect WebSocket server
                const websocket = await perspective.websocket("ws://localhost:8085/websocket");
                // open server table
                const server_table = await websocket.open_table(table_name);
                // load the table into the view
                await viewer.load(server_table);
                // use view configuration
                await viewer.restore(config);
            } catch (error) {
                console.error('发生错误:', error);
            }
        }

        // configuration of the view
        const config1 = {
            "version": "3.3.1",          // Perspective library version (compatibility identifier)
            "plugin": "Datagrid",        // View mode: Datagrid (table) or D3FC (chart)
            "plugin_config": {           // Plugin-specific configuration
                "columns": {
                    "current": {
                        "width": 150       // Column width in pixels
                    }
                },
                "edit_mode": "READ_ONLY",  // Edit mode: READ_ONLY (immutable) or EDIT (editable)
                "scroll_lock": false       // Whether to lock scroll position
            },
            "columns_config": {},        // Custom column configurations (colors, formatting, etc.)
            "settings": true,            // Whether to show settings panel (true/false)
            "theme": "Power Meters",     // Custom theme name (must be pre-defined)
            "title": "Meters list data", // View title
            "group_by": ["location", "groupid"], // Row grouping fields (equivalent to `row_pivots`)
            "split_by": [],              // Column grouping fields (equivalent to `column_pivots`)
            "columns": [                 // Columns to display (in order)
                "timestamp",
                "location",
                "current",
                "voltage",
                "phase"
            ],
            "filter": [],                // Filter conditions (triplet format array)
            "sort": [],                  // Sorting rules (format: [field, direction])
            "expressions": {},           // Custom expressions (e.g., calculated columns)
            "aggregates": {              // Aggregation function configuration
                "timestamp": "last",       // Aggregation: last (takes the latest value)
                "voltage": "last",         // Aggregation: last
                "phase": "last",           // Aggregation: last
                "current": "last"          // Aggregation: last
            }
        };

        // load the first view
        await load_viewer("prsp-viewer-1", config1);
    });
</script>

<!-- define the HTML Structure of the Dashboard -->
<div id="dashboard">
    <div class="viewer-container">
        <perspective-viewer id="prsp-viewer-1" theme="Pro Dark"></perspective-viewer>
    </div>
</div>

view source code

Reference Materials

• Perspective Docs
• TDengine Python Connector
• TDengine Stream Processing