# E2E Tests for Dataplane Manager

This directory contains end-to-end tests for the dataplane-manager service. These tests create **real cloud infrastructure** (EKS/GKE/AKS clusters, VPCs, buckets, etc.) to validate the complete functionality.

## Directory Structure

```
test/e2e/
├── driver/
│   ├── driver.go          # Main test driver managing DataPlane lifecycle
│   └── config.go          # Configuration structures
├── bucket_test.go         # Bucket resource tests
├── load_balancer_test.go  # LoadBalancer and PrivateLink tests
├── node_pool_test.go      # Node pool enumeration tests
├── node_pool_config_test.go # Node pool configuration tests
├── dataplane_test.go      # Main test entry point with TestMain
├── upbound_cr_checker.go  # Crossplane resource status verification
├── utils.go               # Retry utilities
└── config.yaml            # Runtime configuration
```

## Prerequisites

1. **Running Dataplane Manager**: Tests require an active dataplane-manager API endpoint
2. **Cloud Account Credentials**: AWS/GCP/Azure/AliCloud account with proper permissions
3. **Crossplane Providers**: Must be installed and configured in the target cluster

## Configuration

### config.yaml

The `config.yaml` file defines multiple logical regions:

```yaml
regions:
  - cloud-provider: aws          # Cloud provider: aws, gcp, azure, alicloud
    region: us-east-1            # Cloud region
    logical-region: us-east-1-ng # Unique identifier for this config
    endpoint-url: "http://127.0.0.1:10000"  # Dataplane manager API endpoint
    cloud-account-id: "123456789"
    data-plane-id: ""            # Optional: existing DataPlane ID (see below)
```

### Available Regions

| logical-region | cloud-provider | region |
|----------------|----------------|--------|
| `us-west-2-ng-0711` | aws | us-west-2 |
| `us-west-2-byoc-ng` | aws | us-west-2 |
| `staging-ap-southeast-1-ng` | alicloud | ap-southeast-1 |
| `westus3-ng` | azure | westus3 |

### Environment Variables

| Variable | Description | Default |
|----------|-------------|---------|
| `LOGICAL_REGION` | Selects which region config to use | First region in config.yaml |

## Running Tests

### Basic Commands

```bash
# Run all tests (default: first region in config.yaml)
go test -v -timeout 90m ./...

# Run specific test
go test -v -timeout 90m -run TestBucket
go test -v -timeout 90m -run TestNodePools
go test -v -timeout 90m -run TestLoadBalancerPrivateLink
go test -v -timeout 90m -run TestNodePoolConfigs

# Run multiple tests (regex match)
go test -v -timeout 90m -run "TestBucket|TestNodePools"
```

### Specify Cloud Provider

```bash
# Test AWS staging (us-west-2)
LOGICAL_REGION=us-west-2-ng-0711 go test -v -timeout 150m -run TestBucket

# Test AWS (us-west-2 BYOC)
LOGICAL_REGION=us-west-2-byoc-ng go test -v -timeout 90m -run TestBucket

# Test Azure
LOGICAL_REGION=westus3-ng go test -v -timeout 90m -run TestBucket

# Test AliCloud
LOGICAL_REGION=staging-ap-southeast-1-ng go test -v -timeout 150m -run TestBucket
```

### Using Make Commands

```bash
# Run all e2e tests (90m timeout)
make e2e

# Run in CI (generates JUnit XML report)
make ci-e2e
```

## Reusing Existing DataPlane (Recommended for Development)

Creating a new DataPlane takes **30+ minutes**. To speed up local development, you can reuse an existing DataPlane:

### Step 1: Get DataPlane ID

Run the test once and note the DataPlane ID from the logs, or use an existing one.

### Step 2: Update config.yaml

```yaml
regions:
  - cloud-provider: azure
    region: westus3
    logical-region: westus3-ng
    endpoint-url: "http://127.0.0.1:10000"
    cloud-account-id: "377b2461-86c6-4849-a689-8ecf954d3183"
    data-plane-id: "dp-xxxxxxxx"  # <-- Add existing DataPlane ID here
```

### Benefits

When `data-plane-id` is set:
- **Setup**: Skips DataPlane creation, only validates existing DataPlane is ready
- **TearDown**: Skips DataPlane deletion, preserves it for next run
- **Time saved**: ~60 minutes (30min create + 30min delete)

## Local Development Workflow

```bash
# 1. Use the controlplane kubeconfig

# 2. Port-forward dataplane-manager service
kubectl port-forward svc/dataplane-manager -n dataplane-manager 8080:80

# 3. Update config.yaml endpoint-url
#    endpoint-url: "http://127.0.0.1:8080"

# 4. Run tests
cd test/e2e
LOGICAL_REGION=westus3-ng go test -v -timeout 90m -run TestBucket
```

## Test Lifecycle

```
┌─────────────────────────────────────────────────────────────────┐
│                        Test Lifecycle                           │
├──────────┬─────────────────┬────────────────────────────────────┤
│  Phase   │  Duration       │  Action                            │
├──────────┼─────────────────┼────────────────────────────────────┤
│ Setup    │ ~30+ min        │ Create DataPlane (or reuse)        │
│ Wait     │ retry 120x/30s  │ Wait for DataPlane to be Active    │
│ Test     │ varies          │ Run test suites in parallel        │
│ Cleanup  │ ~30+ min        │ Delete DataPlane (skip if reused)  │
│ Verify   │ retry 60x/30s   │ Confirm 404 response               │
└──────────┴─────────────────┴────────────────────────────────────┘
```

## Test Suites

| Test | Description |
|------|-------------|
| `TestBucket` | Create, list, delete object storage buckets |
| `TestLoadBalancerPrivateLink` | Load balancer CRUD, tag updates, PrivateLink operations |
| `TestNodePools` | List and validate node pools (K8s version, instance type, CPU, memory) |
| `TestNodePoolConfigs` | List and validate node pool configurations |

## Troubleshooting

### Test Timeout

The default Go test timeout is 10 minutes, which is insufficient. Always use `-timeout 90m`:

```bash
go test -v -timeout 90m -run TestBucket
```

### DataPlane Not Ready

If you see "not all upbound cr is ready", the Crossplane resources haven't finished reconciling. This typically means:
- Cloud resources are still being created
- There's an error in the Crossplane provider

Check the Crossplane resources:
```bash
kubectl get managed -l tidbcloud.com/dataplane-id=<dataplane-id>
```

### Connection Refused

Ensure the endpoint URL is correct and accessible:
- For port-forward: verify `kubectl port-forward` is running
- For Kong: verify the gateway URL is reachable

### Save Test Logs

```bash
LOGICAL_REGION=westus3-ng go test -v -timeout 90m -run TestBucket 2>&1 | tee test.log
```

## Quick Reference

| Task | Command |
|------|---------|
| Run all tests | `go test -v -timeout 90m ./...` |
| Run Bucket test | `go test -v -timeout 90m -run TestBucket` |
| Run NodePool test | `go test -v -timeout 90m -run TestNodePools` |
| Test AWS staging | `LOGICAL_REGION=us-west-2-ng-0711 go test -v -timeout 150m` |
| Test Azure | `LOGICAL_REGION=westus3-ng go test -v -timeout 90m` |
| Test AliCloud staging | `LOGICAL_REGION=staging-ap-southeast-1-ng go test -v -timeout 150m` |