# TCMS Automation System Design Document

## 1. Background
The primary goal of this system is to enable Product Managers (PMs) to quickly initiate and manage test tasks for new TiDB SPUs (Service Processing Units).
*   **Immediate Objective**: Support the testing of new AWS machine types scheduled for December.
*   **Strategy**: Leverage the existing TCMS (Test Case Management System) infrastructure to avoid reinventing the wheel. Therefore, the initial scope will prioritize Cloud Providers and Regions currently supported by TCMS.

## 2. Overview
The goal is to build a tool for Product Managers (PMs) to automate testing of different TiDB machine types (SPUs). Users can configure test parameters (Cloud, Region, SPU, Component Counts), submit tasks to TCMS, and view results/comparisons via a web interface.

## 2. Architecture
The system adopts a decoupled architecture where the Frontend and Backend operate independently, communicating solely through the Database.

1.  **Frontend (Streamlit)**:
    *   Responsible for User Input and Result Visualization.
    *   Connects directly to TiDB Serverless using credentials stored in `st.secrets`.
    *   **No background processing** happens here; it only reads/writes DB records.
2.  **Backend (Scheduler)**:
    *   A standalone Python process (e.g., running on a server or local machine).
    *   Polls the database for pending tasks.
    *   Submits tasks to TCMS and updates status/results in the DB.
    *   Totally unaware of the Streamlit frontend.
3.  **Database**: TiDB Serverless acting as the shared state and message broker.

## 3. Database Schema

### 3.1 Table: `test_tasks`
Stores the configuration and status of each test task.

| Column Name | Type | Description |
| :--- | :--- | :--- |
| `id` | INTEGER | Primary Key, Auto-increment. |
| `user_email` | TEXT | User identifier (Email or Github ID). |
| `cloud_provider` | TEXT | AWS, GCP, Azure. |
| `region` | TEXT | Cloud region (e.g., us-west-2). |
| `tidb_spu` | TEXT | Selected TiDB Machine Type. |
| `tidb_count` | INTEGER | Number of TiDB nodes. |
| `tidb_config` | JSON | TiDB configuration overrides. |
| `tikv_spu` | TEXT | Selected TiKV Machine Type. |
| `tikv_count` | INTEGER | Number of TiKV nodes. |
| `tikv_config` | JSON | TiKV configuration overrides. |
| `tiflash_spu` | TEXT | Selected TiFlash Machine Type. |
| `tiflash_count` | INTEGER | Number of TiFlash nodes. |
| `tiflash_config` | JSON | TiFlash configuration overrides. |
| `pd_spu` | TEXT | Selected PD Machine Type. |
| `pd_count` | INTEGER | Number of PD nodes. |
| `pd_config` | JSON | PD configuration overrides. |
| `tiproxy_spu` | TEXT | Selected TiProxy Machine Type. |
| `tiproxy_count` | INTEGER | Number of TiProxy nodes. |
| `tiproxy_config` | JSON | TiProxy configuration overrides. |
| `workload_config` | JSON | Advanced workload configuration (default folded). |
| `workflow_execution_id` | TEXT | ID returned by TCMS upon submission. |
| `status` | TEXT | `PENDING`, `RUNNING`, `SUCCESS`, `FAILURE`. |
| `created_at` | DATETIME | Timestamp of creation. |
| `updated_at` | DATETIME | Timestamp of last status update. |

### 3.2 Table: `test_results`
Stores the outcomes of completed tasks.

| Column Name | Type | Description |
| :--- | :--- | :--- |
| `id` | INTEGER | Primary Key. |
| `task_id` | INTEGER | Foreign Key to `test_tasks.id`. |
| `result_type` | TEXT | `SUCCESS` or `FAILURE`. |
| `result_data` | JSON | For Success: Parsed JSON content from result URLs. For Fail: Empty or partial data. |
| `error_msg` | TEXT | Error message if failed. |
| `raw_execution_response` | JSON | Result URLs {"block1": "url1", "block2": "url2"} (Phase 1 output). |

## 4. Modules Detailed Design

### 4.1 Frontend (Streamlit)
The frontend is divided into three main sections:

#### A. User Input (Task Submission)
*   **Identity**: Simple text input for Email (Phase 1). Future: Github OAuth.
*   **SPU Selection (Cascading Dropdowns)**:
    1.  **Cloud Provider**: Enum selection.
    2.  **Region**: Populated based on Cloud Provider (via `tidbcloud_client`).
    3.  **TiDB SPU**: Populated based on Region (via `tidbcloud_client`).
    4.  **TiKV/TiFlash SPU**: Filtered based on TiDB SPU selection (matching vCPU/Architecture rules).
    5.  **Component Counts**: Number inputs for each component.
*   **Advanced Options**:
    *   `st.expander` containing Workload Configuration (JSON/YAML editor or form fields).
*   **Action**: "Submit Test" button.
    *   Inserts a record into `test_tasks` with status `PENDING`.
    *   Displays the generated Task ID to the user.

#### B. Dashboard (Query & Monitor)
*   **Filter/Group**: Options to filter by User, Cloud, SPU, Status.
*   **Task List**: A table view of `test_tasks` joined with `test_results`.
*   **Detail View**: Clicking a task shows detailed configuration and results.
    *   If `SUCCESS`: Show parsed JSON metrics/results.
    *   If `FAILURE`: Show error message.

### 4.2 Backend (Polling Service)
A separate Python script (`scheduler.py`) running continuously.

**Logic Loop**:
1.  **Fetch Pending**: `SELECT * FROM test_tasks WHERE status IN ('PENDING', 'RUNNING')`.
2.  **Process Pending**:
    *   If `PENDING`:
        *   Construct workflow spec using `gen_workflow.py`.
        *   Call `tcms_client.create_workflow_execution`.
        *   Update `workflow_execution_id` and set status to `RUNNING`.
    *   If `RUNNING`:
        *   Call `tcms_client.get_workflow_execution` (need to implement/verify).
        *   Check status.
        *   If `COMPLETED/SUCCESS`:
            *   Fetch result artifacts (URLs).
            *   (Phase 2) Download and parse content from URLs.
            *   Insert into `test_results`.
            *   Update `test_tasks.status` to `SUCCESS`.
        *   If `FAILURE`:
            *   Capture error message.
            *   Insert into `test_results`.
            *   Update `test_tasks.status` to `FAILURE`.
3.  **Sleep**: Wait for N seconds before next poll.

## 5. Integration Points
*   **TiDB Cloud API**: Used in Frontend for dynamic SPU fetching (`tidbcloud_client.py`).
*   **TCMS API**: Used in Backend for workflow submission and status checking (`tcms_client.py`).
*   **Database**: TiDB Serverless using `mysql-connector-python` or `SQLAlchemy`.
    *   Connection parameters (Host, Port, User, Password) will be stored in `.env`.

## 6. Implementation Plan (Phased)
1.  **Phase 1**:
    *   Setup TiDB Serverless Cluster and create tables.
    *   Implement Streamlit Input Form with mocked SPU data or real `tidbcloud_client`.
    *   Implement basic Polling Service (just state transitions, no real TCMS call initially, then integrate).
    *   Basic Dashboard.
2.  **Phase 2**:
    *   Full TCMS integration.
    *   Result parsing (fetching JSON from URLs).
    *   Advanced comparisons in Dashboard.