# Agent Control Server API Documentation

## Overview

Agent Control Server provides RESTful APIs for managing observability agents in Kubernetes clusters.

### API Endpoints Summary

| Method | Endpoint | Description                                |
|--------|----------|--------------------------------------------|
| GET | `/agents` | List agents with optional filtering by type and name |
| GET | `/agents/spec` | Get resource specification settings        |
| POST | `/agents/spec` | Update resource specification              |
| POST | `/agents/pause` | Pause agent reconciliation                 |
| POST | `/agents/resume` | Resume agent reconciliation                |
| POST | `/agents/restart` | Restart a component (vmagent/vector)       |
| POST | `/agents/drainnode` | Drain a node for maintenance               |
| POST | `/agents/storage` | Set storage size for a component (vmagent) |
| GET | `/ops/nodegroups` | List o11y node groups with version info    |
| POST | `/ops/nodegroups/upgrade` | Upgrade node group version              |
| GET | `/metrics` | Prometheus metrics endpoint                |
| GET | `/debug/pprof/*` | Runtime profiling endpoints                |

## Reference Data

### Agent Types

| Type | Description |
|------|-------------|
| o11ytidb | Observability TiDB Agent |
| o11yk8s | Observability Kubernetes Agent |
| kubernetes | Kubernetes Agent |
| sharedpool | Shared Pool Agent |
| tidbgroup | TiDB Group Agent |

### Component Types

| Component | Description |
|-----------|-------------|
| vmagent | VMAgent metrics collector |
| vector | Vector log collector |


## /agents Endpoints

### GET /agents

Lists agents with optional filtering.

**Query Parameters**:

| Parameter              | Type   | Description                    |
|------------------------|--------|--------------------------------|
| agentType              | string | Filter by agent type           |
| agentName              | string | Filter by agent name           |
| kubernetesClusterName  | string | Filter by cluster name         |
| tenantId               | string | Filter by tenant ID            |
| o11yId                 | string | Filter by observability ID     |

All parameters are optional and can be freely combined.

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": [
    {
      "name": "my-k8s-agent",
      "agentType": "kubernetes",
      "kubernetesClusterName": "my-cluster",
      "tenantId": "12345",
      "applicationId": "app-123",
      "o11yId": "o11y-789",
      "phase": "Running"
    },
    {
      "name": "my-o11yk8s-agent",
      "agentType": "o11yk8s",
      "kubernetesClusterName": "my-cluster",
      "tenantId": "12345",
      "applicationId": "app-123",
      "projectId": "proj-456",
      "o11yId": "o11y-789",
      "phase": "Running"
    }
  ]
}
```

| Field                  | Type   | Description                                |
|------------------------|--------|--------------------------------------------|
| name                   | string | Name of the agent                          |
| agentType              | string | Type of the agent                          |
| kubernetesClusterName  | string | Kubernetes cluster name                    |
| tenantId               | string | Tenant ID                                  |
| applicationId          | string | Application ID                             |
| projectId              | string | Project ID (optional, not all agents)      |
| o11yId                 | string | Observability ID                           |
| sharedPoolId           | string | SharedPool ID (optional, not all agents)   |
| phase                  | string | Current phase of the agent                 |

### GET /agents/spec

Retrieves the current resource specification settings for an agent.

**Query Parameters**:

| Parameter  | Type   | Required | Description              |
|------------|--------|----------|--------------------------|
| agentType  | string | Yes      | Type of the agent        |
| agentName  | string | Yes      | Name of the agent        |

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": {
    "agentType": "o11yk8s",
    "agentName": "my-agent",
    "desiredSpec": "default",
    "minSpec": "small",
    "maxSpec": "large"
  }
}
```

| Field       | Type   | Description                                    |
|-------------|--------|------------------------------------------------|
| agentType   | string | Type of the agent                              |
| agentName   | string | Name of the agent                              |
| desiredSpec | string | Desired specification (may be empty)           |
| minSpec     | string | Minimum specification (may be empty)           |
| maxSpec     | string | Maximum specification (may be empty)           |

### POST /agents/spec

Updates the resource specification for an agent.

**Request Body**:

```json
{
  "agentType": "o11yk8s",
  "agentName": "my-agent",
  "desiredSpec": "default",
  "minSpec": "small",
  "maxSpec": "large"
}
```

| Field       | Type   | Required | Description              |
|-------------|--------|----------|--------------------------|
| agentType   | string | Yes      | Type of the agent        |
| agentName   | string | Yes      | Name of the agent        |
| desiredSpec | string | No       | Desired specification    |
| minSpec     | string | No       | Minimum specification    |
| maxSpec     | string | No       | Maximum specification    |

**Note**: `desiredSpec`, `minSpec` or `maxSpec` is required. Specs: [default, medium, large].

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": {
    "agentType": "o11yk8s",
    "agentName": "my-agent",
    "desiredSpec": "default",
    "minSpec": "small",
    "maxSpec": "large"
  }
}
```

### POST /agents/pause

Pauses the reconciliation loop for an agent. The agent will stop processing changes until resumed.

**Request Body**:

```json
{
  "agentType": "o11yk8s",
  "agentName": "my-agent"
}
```

| Field     | Type   | Required | Description              |
|-----------|--------|----------|--------------------------|
| agentType | string | Yes      | Type of the agent        |
| agentName | string | Yes      | Name of the agent        |

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": {
    "agentType": "o11yk8s",
    "agentName": "my-agent",
    "isPaused": true
  }
}
```

| Field     | Type    | Description              |
|-----------|---------|--------------------------|
| agentType | string  | Type of the agent        |
| agentName | string  | Name of the agent        |
| isPaused  | boolean | Pause status (true)      |

### POST /agents/resume

Resumes the reconciliation loop for a previously paused agent.

**Request Body**:

```json
{
  "agentType": "o11yk8s",
  "agentName": "my-agent"
}
```

| Field     | Type   | Required | Description              |
|-----------|--------|----------|--------------------------|
| agentType | string | Yes      | Type of the agent        |
| agentName | string | Yes      | Name of the agent        |

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": {
    "agentType": "o11yk8s",
    "agentName": "my-agent",
    "isPaused": false
  }
}
```

| Field     | Type    | Description               |
|-----------|---------|---------------------------|
| agentType | string  | Type of the agent         |
| agentName | string  | Name of the agent         |
| isPaused  | boolean | Pause status (false)      |

### POST /agents/restart

Requests a restart of a specific component (vmagent or vector) within an agent.

**Request Body**:

```json
{
  "agentType": "o11yk8s",
  "agentName": "my-agent",
  "component": "vmagent"
}
```

| Field     | Type   | Required | Description                                         |
|-----------|--------|----------|-----------------------------------------------------|
| agentType | string | Yes      | Type of the agent                                   |
| agentName | string | Yes      | Name of the agent                                   |
| component | string | Yes      | Component to restart: "vmagent" or "vector"        |

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": {
    "agentType": "o11yk8s",
    "agentName": "my-agent",
    "component": "vmagent"
  }
}
```

### POST /agents/drainnode

Initiates a node drain operation for a specific pod. This is typically used for maintenance operations.

**Request Body**:

```json
{
  "agentType": "o11yk8s",
  "agentName": "my-agent",
  "namespace": "default",
  "podName": "my-pod"
}
```

| Field     | Type   | Required | Description              |
|-----------|--------|----------|--------------------------|
| agentType | string | Yes      | Type of the agent        |
| agentName | string | Yes      | Name of the agent        |
| namespace | string | Yes      | Kubernetes namespace     |
| podName   | string | Yes      | Name of the pod          |

**Response**:

```json
{
  "code": 200,
  "message": "drain node task initiated",
  "data": {
    "agentType": "o11yk8s",
    "agentName": "my-agent",
    "podName": "my-pod",
    "namespace": "default"
  }
}
```

### POST /agents/storage

Sets the storage size for a specific component (vmagent) within an agent.

**Request Body**:

```json
{
  "agentType": "o11yk8s",
  "agentName": "my-agent",
  "component": "vmagent",
  "size": "50Gi"
}
```

| Field     | Type   | Required | Description                            |
|-----------|--------|----------|----------------------------------------|
| agentType | string | Yes      | Type of the agent                      |
| agentName | string | Yes      | Name of the agent                      |
| component | string | Yes      | Component type: "vmagent"              |
| size      | string | Yes      | Storage size (e.g., "50Gi", "100Gi")   |

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": {
    "agentType": "o11yk8s",
    "agentName": "my-agent",
    "component": "vmagent",
    "size": "50Gi"
  }
}
```

| Field     | Type   | Description               |
|-----------|--------|---------------------------|
| agentType | string | Type of the agent         |
| agentName | string | Name of the agent         |
| component | string | Component type            |
| size      | string | Storage size              |

**Supported Agent Types**: kubernetes, sharedpool, o11ytidb, o11yk8s

## /ops/nodegroups Endpoints

### GET /ops/nodegroups

Lists o11y agent node groups with version and count info read from Crossplane
NodeGroup (AWS), NodePool (GCP), or KubernetesClusterNodePool (Azure) CRs.

For GCP, `version` and `count` come from the NodePool spec because the
Crossplane NodePool status does not report an observed node version or current
instance count. The GCP count is the configured initial count, not the live
autoscaler replica count.

For Azure, `version` prefers the NodePool observed status and falls back to
spec; `count` prefers the spec desired size and falls back to status (so the
autoscaler live count does not flip the reported value). The `version` is
pinned on the agent via the `o11y-node-version` annotation on first reconcile
and is the only source the NodePool task honors afterwards. If the AKS cluster
is upgraded out-of-band (Terraform, portal), existing o11y NodePools stay on
the pinned version until `/ops/nodegroups/upgrade` is called.

**Query Parameters**:

| Parameter | Type   | Required | Description                    |
|-----------|--------|----------|--------------------------------|
| tenantId  | string | No       | Filter by tenant ID            |
| projectId | string | No       | Filter by project ID           |

**Response**:

```json
{
  "code": 200,
  "message": "success",
  "data": {
    "nodeGroups": [
      {
        "kind": "o11y-k8s-nodes",
        "count": 3,
        "version": "1.29",
        "agentType": "o11yk8s",
        "agentName": "my-o11yk8s-agent",
        "kubernetesClusterName": "my-cluster",
        "tenantId": "12345",
        "projectId": "proj-456",
        "applicationId": "app-123",
        "nodeGroupName": "tidbcloud-o11y-k8s-proj-456",
        "phase": "Normal"
      },
      {
        "kind": "o11y-tidb-nodes",
        "count": 2,
        "version": "1.28",
        "agentType": "o11ytidb",
        "agentName": "my-o11ytidb-agent",
        "kubernetesClusterName": "my-cluster",
        "tenantId": "12345",
        "projectId": "proj-456",
        "applicationId": "app-789",
        "nodeGroupName": "tidbcloud-o11y-tidb-app-789",
        "phase": "Normal"
      }
    ],
    "total": 2
  }
}
```

| Field                  | Type   | Description                                          |
|------------------------|--------|------------------------------------------------------|
| kind                   | string | Node group kind: "o11y-k8s-nodes" or "o11y-tidb-nodes" |
| count                  | int    | Configured node count from the provider Crossplane CR |
| version                | string | Kubernetes node version from the provider Crossplane CR |
| agentType              | string | Agent type                                           |
| agentName              | string | Agent CR name                                        |
| kubernetesClusterName  | string | Kubernetes cluster name                              |
| tenantId               | string | Tenant ID                                            |
| projectId              | string | Project ID (optional, o11y agents only)              |
| applicationId          | string | Application ID                                       |
| nodeGroupName          | string | Stable logical node group name; may differ from the physical Crossplane CR name |
| phase                  | string | Current phase of the agent                           |

### POST /ops/nodegroups/upgrade

Upgrades the node group version for an o11y agent. Sets the version annotation on the agent CR; the controller applies the change during reconciliation.

The agent must be in `Normal` or `Hibernated` phase. If the agent is already at
the requested version, the request succeeds with no change. A successful
response records the requested version on the agent CR; the controller applies
it asynchronously during reconciliation.

**Request Body**:

```json
{
  "kind": "o11y-k8s-nodes",
  "agentName": "my-o11yk8s-agent",
  "version": "1.30"
}
```

| Field     | Type   | Required | Description                                         |
|-----------|--------|----------|-----------------------------------------------------|
| kind      | string | Yes      | Node group kind: "o11y-k8s-nodes" or "o11y-tidb-nodes" |
| agentName | string | Yes      | Agent CR name                                       |
| version   | string | Yes      | Target Kubernetes node version                      |

**Response**:

```json
{
  "code": 200,
  "message": "nodegroup upgrade: agent o11y-k8s-nodes/my-o11yk8s-agent version 1.29 -> 1.30",
  "data": {
    "kind": "o11y-k8s-nodes",
    "agentType": "o11yk8s",
    "agentName": "my-o11yk8s-agent",
    "currentVersion": "1.29",
    "targetVersion": "1.30"
  }
}
```

| Field          | Type   | Description                     |
|----------------|--------|---------------------------------|
| kind           | string | Node group kind                 |
| agentType      | string | Agent type                      |
| agentName      | string | Agent CR name                   |
| currentVersion | string | Version before upgrade          |
| targetVersion  | string | Requested target version        |

**Error Responses**:

| Status | Condition                                         |
|--------|---------------------------------------------------|
| 400    | Missing required fields or invalid kind           |
| 404    | Agent not found                                   |
| 409    | Agent is not in Normal phase                      |
| 500    | Failed to update agent annotation                 |

## /debug Endpoints

Profiling endpoints for development and debugging.

### GET /debug/pprof/*

Runtime profiling data for performance analysis.

**Available Endpoints**:
- `/debug/pprof/` - Profile index
- `/debug/pprof/cmdline` - Command line arguments
- `/debug/pprof/profile` - CPU profile
- `/debug/pprof/symbol` - Symbol table
- `/debug/pprof/trace` - Execution trace

**Note**: These endpoints should only be enabled in development environments.

## /metrics Endpoint

### GET /metrics

Prometheus metrics endpoint. Exposes metrics in Prometheus format for monitoring and alerting.