# System Architecture

## System Overview

tcidm is a regional-level TiDB Cloud service that provides infrastructure management capabilities, focusing on data import, data migration, and network connectivity. It is built with Gin framework for HTTP handling and exposes REST APIs for interacting with regional resources, managing import operations, and handling data migration jobs.

## Architecture Diagram

```mermaid
graph TD
    A[HTTP Client] --> B[Gin HTTP Server]
    B --> C[API Layer]
    C --> D[Controller Layer]
    
    subgraph "Controllers"
        E[Regional Import Service]
        F[Regional Util Service]
        G[DM Job Management]
        H[PrivateLink Management]
        I[Proxy Service]
    end
    
    D --> J[Service Layer]
    J --> K[Data Access Layer]
    K --> L[Kubernetes API]
    L --> M[Kubernetes Cluster]
    
    J --> N[External Services]
    N --> O[TiDB Management Service]
    N --> P[Dataflow Service]
    N --> Q[Cloud Providers]
    
    subgraph "Monitoring & Observability"
        R[Prometheus Metrics]
        S[OpenTelemetry Tracing]
    end
```

## Component Descriptions

### tcidm

- **Purpose**: Provide regional-level infrastructure services for TiDB Cloud
- **Responsibilities**:
  - Handle HTTP requests for regional operations
  - Process data import operations
  - Manage data migration jobs
  - Handle PrivateLink endpoint management
  - Provide utility operations for TiDB clusters
  - Proxy requests to other TiDB Cloud services

- **Dependencies**:
  - Kubernetes API for resource management
  - TiDB Management Service for cluster operations
  - Dataflow Service for data operations
  - Cloud providers (AWS, GCP, Azure) for resource access

- **Type**: Application/API Service

### Gin HTTP Server

- **Purpose**: HTTP request handling
- **Responsibilities**:
  - Serve HTTP requests
  - Handle routing and middleware
  - Manage request/response lifecycle

- **Type**: Web Framework

### API Layer

- **Purpose**: API definition and documentation
- **Responsibilities**:
  - Define API routes
  - Handle CORS
  - Serve Swagger documentation

- **Type**: API Definition

### Controller Layer

- **Purpose**: Request handling and business logic entry points
- **Responsibilities**:
  - Process HTTP requests
  - Validate input
  - Call service layer methods
  - Handle error responses

- **Subcomponents**:
  - Regional Import Service: Import operation management
  - Regional Util Service: Utility operations
  - DM Job Management: Data migration job handling
  - PrivateLink Management: Network connectivity
  - Proxy Service: Request proxying

### Service Layer

- **Purpose**: Business logic implementation
- **Responsibilities**:
  - Implement core business logic
  - Interact with data access layer
  - Handle external service calls

- **Type**: Business Logic

### Data Access Layer

- **Purpose**: Data and resource access
- **Responsibilities**:
  - Interact with Kubernetes API
  - Manage CRD (Custom Resource Definition) operations
  - Handle Kubernetes resource management

- **Type**: Data Access

### External Services Integration

- **TiDB Management Service**: For cluster management operations
- **Dataflow Service**: For data operations like import and migration
- **Cloud Providers**: For access to storage and network resources

## Data Flow

```mermaid
sequenceDiagram
    participant Client as HTTP Client
    participant Server as Gin Server
    participant Ctrl as Controller
    participant Svc as Service
    participant DA as Data Access
    participant K8s as Kubernetes API
    participant Ext as External Services
    
    Client->>Server: HTTP Request
    Server->>Ctrl: Route to Controller
    Ctrl->>Svc: Business Logic Call
    Svc->>DA: Resource Access
    DA->>K8s: Kubernetes API Call
    K8s-->>DA: API Response
    DA-->>Svc: Resource Data
    
    Svc->>Ext: External Service Call
    Ext-->>Svc: Service Response
    
    Svc-->>Ctrl: Processed Result
    Ctrl-->>Server: Response Data
    Server-->>Client: HTTP Response
```

## Integration Points

### External APIs
- **Kubernetes API Server**: For managing custom resources
- **TiDB Management Service API**: For cluster operations
- **Dataflow Service API**: For data operations
- **Cloud Provider APIs**: For access to storage and network resources

### Databases
- **etcd**: For Kubernetes resource storage

### Third-party Services
- **Prometheus**: Metrics collection and storage
- **OpenTelemetry**: Distributed tracing
- **Jaeger**: Tracing visualization

## Infrastructure Components

- **Kubernetes Cluster**: Primary deployment platform
- **Cloud Providers**: AWS, GCP, Azure for infrastructure resources
- **Networking**: VPC, subnets, security groups, PrivateLink
- **Storage**: Cloud storage for data and backups

## Deployment Model

- **Containerized Application**: Docker container deployment
- **Kubernetes Deployment**: Managed Kubernetes cluster
- **CI/CD Pipeline**: GitHub Actions for deployment
- **Regional Deployment**: Per-region service instances

## Key Technologies

- **Language**: Go (1.23.2)
- **Web Framework**: Gin Gonic (v1.8.2)
- **Kubernetes Client**: Client-go (v0.28.3)
- **API Documentation**: Swagger (go-swagger)
- **Metrics**: Prometheus (client_golang v1.17.0)
- **Tracing**: OpenTelemetry (v1.11.0)
- **Configuration**: Viper (v1.14.0)

## Monitoring & Observability

- **Prometheus Metrics**: Request duration, error rates, operation counts
- **OpenTelemetry Tracing**: Distributed tracing for debugging
- **Jaeger Integration**: Tracing visualization
- **Health Check**: `/healthz` endpoint for liveness checks