# TiDB Cloud Deployment Structure Analysis

**Version:** 1.1.0  
**Last Updated:** 2026-03-24  
**Changes:** Fixed stack count, added multi-cloud support (AWS/GCP/Alicloud/Azure), enhanced descriptions

## Summary

TiDB Cloud deployment follows a three-layer architecture: **Infrastructure → Application → Configuration**. Five core repositories manage everything from EKS/ACK clusters to application services and runtime configurations, serving both **Serverless** and **Dedicated** product lines across multi-cloud environments (AWS, GCP, Alicloud, Azure).

---

## Core Repositories

| Repo | Technology | Purpose | Local Path |
|------|------------|---------|------------|
| aws-shared-cd | Pulumi/TypeScript | Dev/Staging EKS/ACK cluster infrastructure | ~/coding/aws-shared-cd |
| infra-cd | Pulumi/TypeScript | Infra API + Multi-cloud account management (AWS+GCP) | ~/coding/infra-cd |
| app-delivery | ArgoCD/Kustomize | Application service manifests (68 services) | ~/coding/app-delivery |
| auto-deploy | Go | Multi-cloud auto-deployment tool (AWS/GCP/Alicloud/Azure) | ~/coding/auto-deploy |
| oneconfig | Git/Jenkins | Service configuration management | ~/coding/oneconfig |

---

## Deployment Architecture

```
┌─────────────────────────────────────────────────────────────────────┐
│                        Configuration Layer                          │
│                         oneconfig (Git)                             │
│              account | billing | dataflow | ecosystem | ...         │
└─────────────────────────────────────────────────────────────────────┘
                                 ↓
┌─────────────────────────────────────────────────────────────────────┐
│                        Application Layer                            │
│                     app-delivery (ArgoCD)                           │
│     Serverless Services | Dedicated Services | O11y | Infra        │
│                     68 registered applications                      │
└─────────────────────────────────────────────────────────────────────┘
                                 ↓
┌─────────────────────────────────────────────────────────────────────┐
│                      Infrastructure Layer                           │
│       aws-shared-cd (Clusters) + infra-cd (API/Accounts)            │
│   EKS Free Tier | ACK Free Tier | Infra API | Multi-Account        │
└─────────────────────────────────────────────────────────────────────┘
```

---

## Infrastructure Layer Components

### 1. aws-shared-cd - Cluster Infrastructure

**Related Repositories:**
- `aws-shared-cd` - Dev/Staging cluster deployment (this repo)
- `aws-shared-cd-prod` - Production version lock (not a deployment definition repo)
- `aws-shared-provider` - Infrastructure provider components

**Directory Structure:**
```
pool/
├── eks-freetier/      # EKS clusters (44 stacks)
│   ├── index.ts       # Main entry
│   ├── serverless.ts  # Serverless components
│   ├── o11y.ts        # Observability
│   └── ...
├── ack-freetier/      # ACK clusters
├── lib/               # Shared libraries
│   ├── serverless/    # Serverless libs
│   ├── k8s/           # K8s utilities
│   └── ...
└── manifests/         # K8s manifests
monitoring/            # Monitoring stacks
```

**Deployed Regions (Pulumi Stacks):**

 | Environment | Regions | Stack Count |
|-------------|---------|-------------|
| dev | us-east-1, ap-southeast-1 | 2 (EKS: + 1 (ACK) |
| staging | us-east-1 | 1 (EKS) |
| prod | ap-east-1, ap-northeast-1/2, ap-south-1, ap-southeast-1/5, cn-hongkong, eu-central-1, na-south-1, us-east-1/4, us-west-2 | 3 (EKS) + 4 (ACK) |
| **Total** | **22 stacks** (15 EKS + 7 ACK) | |

**EKS Cluster Components:**
- `serverless` - TiDB Serverless cluster setup
- `asg` - Auto Scaling Groups
- `autoscaler` - Cluster autoscaler
- `calico` - Network policy
- `cert-manager` - Certificate management
- `cni` - VPC CNI
- `csi` - Storage classes
- `etcd` - etcd clusters
- `external-secret` - External Secrets Operator
- `ingress` - NGINX ingress controller
- `istio` - Service mesh
- `kms` - KMS key management
- `loadbalancer` - AWS Load Balancer Controller
- `local-provisioner` - Local volume provisioner
- `o11y` - Observability (VM, Grafana, etc.)
- `regional-infra` - Regional infrastructure (RDS, Redis)
- `metering` - Serverless metering resources
- `ai-gateway` - AI gateway
- `tiflash-autoscale` - TiFlash autoscaling
- `tiflash-manager` - TiFlash management
- `cassandra-proxy` - Cassandra proxy

**Manifests Components:**
- ai-gateway
- aws-cluster-auto-scaler
- cluster-certs
- fts-server (Full Text Search)
- lambda
- node-problem-detector
- regional-redis
- serverless
- victoria-metrics-operator
- vector

### 2. infra-cd - Infrastructure API & Multi-Cloud Management

**Projects:**
```
projects/
├── apiserver/      # Infra API Server deployment
├── aws-accounts/   # AWS multi-account management
├── gcp-projects/   # GCP project management
└── infra-api/      # Infra API deployment
```

**Managed Components:**
- Infra API Server
- Infra etcd
- Crossplane provider-aws
- Crossplane provider-gcp
- AWS multi-account management
- GCP project management

**Multi-Cloud Architecture:** Provides unified deployment and management across **AWS and GCP** through consistent Pulumi programs (6500+ lines of TypeScript). Enables:
- Hybrid deployments where applications run on both cloud providers
- Unified control plane through Infra API Server
- Centralized multi-account/project governance
- Consistent security and compliance policies across clouds

**Branch Strategy:** Only `main` branch, merge → dev + staging auto-apply

**Related Repositories:**
- `aws-shared-provider` - AWS-specific infrastructure provider components

---

## Application Layer Components

### app-delivery - Service Manifests (68 Applications)

**Product Lines:**

#### Serverless Services
- serverless-svc
- serverless-global-service
- serverless-regional-service
- serverless-meta-service
- serverless-meta-store
- serverless-metering
- serverless-ops-service
- serverless-credits-campaign-svc
- serverless-global-regional-proxy

#### Dedicated Services
- dedicated-cluster-service
- dedicated-cluster-ng-service
- dedicated-global-service
- dedicated-regional-server
- dedicated-region-base-services
- dedicated-infra-apiserver
- dedicated-metering-server
- dedicated-ticdc-service
- dedicated-3-layer-migrate

#### Core Business Services
- account-svc
- authserver
- billing-svc
- auditservice
- core-svc
- devops-service
- tidb-mgmt-portal
- tidb-mgmt-service
- tidb-worker
- rbac-management-service
- quota-exporter

#### Data Services
- dataflow-svc
- dataflow-service-ng
- dataapps
- dataservice-central-svc
- dataservice-regional-svc
- tiflow-crd

#### Configuration Services
- config-center
- config-center-regional
- config-manager
- config-service

#### Observability Services
- o11y-svc
- o11y-agent-control-server
- o11y-cluster-agent-crd
- o11y-regional-server
- o11y-regional-server2
- o11y-secret-store
- loki
- tempo-migrate
- tempo-proxy
- tempo-wrapper
- metrics-integration

#### Infrastructure Providers
- eks-provider
- gke-provider
- gardener-provider
- chaos-provider

#### Infrastructure Webhooks/CRDs
- infra-api-crd
- infra-api-webhook
- infra-api-providerconfig
- infra-webhook
- infra-etcd-backup
- infra-exporter

#### Other Services
- clinic
- novu-svc
- nextgen-global-service
- nextgen-regional-control-plane
- ipip-svc
- cost-metering-service-svc
- vpa-agent
- tcidm
- secretstore

**Directory Pattern:**
```
{service}/
├── base/               # Common manifests
│   └── kustomization.yaml
└── envs/
    ├── dev-us-west-2/  # Dev overlays
    ├── stg-us-central1/
    └── prod-us-east-1/
```

**ArgoCD Endpoints:**
- prod: https://tidbcloud-cd.pingcap.net
- dev: https://dev-tidbcloud-cd.pingcap.net
- staging: https://staging-tidbcloud-cd.pingcap.net

---

## Configuration Layer

### oneconfig - Service Configurations

**Directory Structure (per service):**
```
account/           # Account service config
billing/           # Billing service config
dataflow/          # Dataflow service config
dataservices/      # Dataservices config
ecosystem/         # Ecosystem config
eventbus/          # EventBus config
cost/              # Cost service config
aipower/           # AI Power config
...
```

**Branch → Environment Mapping:**
| Branch | Environment | Namespace |
|--------|-------------|-----------|
| dev | dev | nightly |
| staging | staging | staging |
| prod | prod | prod |
| personal/${name} | dev | ${name} |

**Workflow:** PR → Review → Jenkins → Apply to environment

**Runtime:** config-center API service + Go viper SDK

---

## Deployment Workflows

### Infrastructure Deployment (aws-shared-cd)
```
PR to main → GitHub Actions → Dev EKS cluster
PR to release-* → GitHub Actions → Staging EKS cluster
Manual → Prod EKS cluster
```

### Application Deployment (app-delivery)
```
PR to main → ArgoCD sync → K8s cluster
(ArgoCD continuously syncs from git)
```

### Configuration Deployment (oneconfig)
```
PR to branch → Jenkins → Config-center → Services
```

---

## Observability Stack

**Monitoring (aws-shared-cd/monitoring):**
- Victoria Metrics Operator
- Grafana
- Prometheus PodScrape
- Multitenant PodScrape

**Logging:**
- Loki
- Vector

**Tracing:**
- Tempo (migrate/proxy/wrapper)

**Other:**
- Node Problem Detector
- Falco (security)
- Prober

---

## Key Design Patterns

### 1. Kustomize Overlays
- `base/` - Common configuration
- `envs/{env}-{region}/` - Environment-specific overrides
- Gradual rollout by updating overlays one by one

### 2. Secret Management
- No secrets in git
- ExternalSecret CR → AWS/GCP Secret Manager
- ESO (External Secrets Operator) reconciles

### 3. Multi-Region Architecture
- Regional clusters (dev/staging/prod)
- Regional infrastructure (RDS, Redis)
- Global services vs Regional services

### 4. Infrastructure as Code
- Pulumi for infrastructure (TypeScript)
- Declarative cluster configuration
- Stack-based environment management

---

## Relationships

```
                    ┌─────────────┐
                    │  oneconfig  │
                    │  (configs)  │
                    └──────┬──────┘
                           │ SDK
                           ↓
┌─────────────────────────────────────────────┐
│              app-delivery                   │
│     (68 services: Serverless + Dedicated)   │
└─────────────────────────────────────────────┘
          │                       │
          ↓                       ↓
┌─────────────────┐     ┌─────────────────┐
│  aws-shared-cd  │     │    infra-cd     │
│ (EKS/ACK Clusters)│    │ (Multi-Cloud:  │
│                 │     │  AWS + GCP)     │
└─────────────────┘     └─────────────────┘
          │                       │
          └───────────┬───────────┘
                      ↓
              ┌─────────────┐
              │  AWS / GCP  │
              │ (Hybrid)    │
              └─────────────┘
```

**Key Relationships:**
- **oneconfig** → Application Layer: Services read configs via Go viper SDK at runtime
- **app-delivery** → Infrastructure Layer: Applications deployed on EKS/ACK clusters managed by aws-shared-cd
- **infra-cd** → Cloud Providers: Manages AWS accounts and GCP projects through Crossplane providers
- **Multi-cloud**: Same application can run on AWS EKS or GCP GKE, managed through unified infra-cd

---

## Sources
- aws-shared-cd readme.md - reliability: high
- aws-shared-cd/pool/eks-freetier/index.ts - reliability: high
- aws-shared-cd/pool/eks-freetier/serverless.ts - reliability: high
- infra-cd/README.md - reliability: high
- app-delivery/README.md - reliability: high
- auto-deploy/README.md - reliability: high
- Local repository exploration - reliability: high