# vm-migration

Single-binary tool for migrating VictoriaMetrics tenant history data across
vmstorage topologies. Local commands author the inputs — tenant discovery,
plan init / compile. Execution (manifest
generation, vmctl shard migration, integrity verification, loop
orchestration across tenant batches) runs in the long-running HTTP service
(`serve`).

## Build

```bash
cd vm-migration
make build                # injects version/commit/date via -ldflags
./vm-migration --help
./vm-migration version
```

Or install directly (no version metadata):

```bash
go install ./cmd/vm-migration
```

Requires Go 1.26 (see `go.mod`).

## Test

```bash
make test                 # go test ./... with an 80% coverage gate
make lint                 # gofmt/goimports/gofumpt/golangci-lint
```

Tests use dependency injection (`vmquery.Client`, `VmctlRunner`, typed
action executors). No external services are required.

## Subcommands

```
vm-migration
├── plan        {init, compile}                # plan.json lifecycle
├── tenants     {dedicated, essential, merge}  # build tenant list inputs
├── serve                                      # long-running HTTP service
├── completion  {bash, zsh, fish}              # emit shell completion script
└── version                                    # print build version
```

| Subcommand | Purpose |
|---|---|
| `plan init` | Generate a fresh plan.json from embedded template |
| `plan compile` | Validate plan.json + merged tenants and write a compact reviewable compiled plan (e.g. `plan-compile.json`) |
| `tenants dedicated {k8s, s3}` | Discover dedicated tenants from live Kubernetes clusters and/or the S3 archive via `kubectl` + `aws s3` |
| `tenants essential {mgmt, rds, s3}` | Collect Essential-tier tenants in stages: mgmt listing, regional-server join, S3 archive resolution |
| `tenants merge` | Merge multiple tenant lists into a flat tenant array |
| `serve` | Load prepared migrations and run the long-running HTTP service |
| `completion bash\|zsh\|fish` | Emit shell completion script |
| `version` | Print vm-migration version |

All inputs are explicit flags (no positional args): `plan compile --plan-path`.

## Quick start

Compile and review a plan, then start the service and create the migration
over HTTP:

```bash
vm-migration plan init
# edit plan.json, fill in the null config fields
vm-migration tenants merge --input tenants-k8s.json --input tenants-s3.json --output tenants.json
vm-migration plan compile --tenants tenants.json --output plan-compile.json
# review plan-compile.json
vm-migration serve --migrations-dir ./migrations
curl -X POST localhost:8080/migrations/prod-2026-08 \
  -H 'Content-Type: application/json' \
  --data-binary @plan-compile.json
```

`plan compile --tenants` embeds the flat array written by `tenants merge` and
resolves only static configuration. Tenant-specific placeholders remain compact
in the compiled JSON. `POST /migrations` reads the embedded list, specializes
the plan per tenant, and writes the runtime loop and tenant files.

Each runtime step carries a typed `action` that the service executes
directly in-process; there is no shell command to replay. `GET
/migrations/{name}/steps` previews the pending vmctl commands before
anything runs. To recover from a failed tenant or step, POST the run
endpoint again — the loop resumes, skipping completed steps and retrying
from the first incomplete one. Serve also auto-resumes interrupted jobs at
startup:

```bash
curl localhost:8080/migrations/prod-2026-08/steps
curl -X POST localhost:8080/migrations/prod-2026-08/run
```

## Service mode

`vm-migration serve` scans `--migrations-dir` at startup and loads every
prepared migration containing `migration-loop.json`. Jobs are single-flight
per migration; different migrations may run concurrently. Creating migrations
goes through `POST /migrations` (body = compiled plan).

**Full HTTP API reference: [docs/api.md](./docs/api.md).**

```bash
vm-migration serve --addr :8080 --migrations-dir ./migrations
curl -X POST localhost:8080/migrations/prod-2026-08/run
curl localhost:8080/migrations/prod-2026-08/summary?format=text
```

## Logging

Local command results write to **stdout**. Concise operator progress,
warnings, and diagnostic `log/slog` records write to **stderr**, keeping
stdout safe for `jq`/`awk`/redirect pipelines.

In service mode, progress and observed action output flow through the
current job's `/logs` stream; detailed action output is also retained in
each tenant's on-disk `*.log` file. Internal diagnostics write to process
stderr.

The root-level `-v / --v` flag controls verbosity:

- `-v=0` error
- `-v=1` warn
- `-v=2` info (default)
- `-v=3` debug

Logs are plain text lines on stderr; JSON log format is not currently
exposed as a flag — if you need structured JSON for log shipping,
post-process the text lines or wrap the binary.

## Output format

Local command results are JSON documents or human-readable tables. In
service mode, `GET /migrations/{name}/summary` serves both forms via
`?format=text|json`.

## Wire-format compatibility

The wire schema is pre-release; prepared runtime artifacts must be
regenerated after typed action schema changes. Concretely:

- **Unknown fields** are ignored by normal Go JSON decoding. Required typed
  action fields are validated before execution.
- **Phase literals**: `"pending" / "running" / "imported" / "failed" / "unknown"`.
- **CLI surface**: the CLI is beta until the first tagged release. Subcommand
  names are stable; flag names and defaults may evolve during beta. Pin to a
  specific binary version in operator scripts.
- **vmctl invocation**: `vmctl vm-native --vm-native-src-addr … --vm-native-dst-addr …
  --vm-native-filter-match '{__name__!=""}' --vm-native-filter-time-start/end …
  --vm-native-step-interval … --vm-rate-limit … --vm-concurrency …
  --disable-progress-bar -s`.
- **vmselect HTTP**: `/api/v1/query` (POST form-encoded), `/api/v1/export/native`
  (streaming). Tenant IDs in URL paths preserve `:` (e.g. `1001:0`) without
  percent-encoding.
- **JSON HTML escaping**: disabled. `>`, `<`, `&` survive literally in status
  JSON so error messages and PromQL selectors remain human-readable.

## Architecture notes

- **Single owner for wire types**: `internal/schema/` owns every type that
  crosses the process boundary via JSON. Other packages alias or import from
  there; no duplicate definitions across reader/writer packages.
- **Atomic file writes**: every status / state / plan write goes through
  `tmp-file + os.Rename`. Readers see either the old or the new file, never
  a partial write.
- **Dependency injection for external systems**: `vmquery.Client`,
  `VmctlRunner` are interfaces; production wires HTTP /
  `exec.Cmd` implementations, tests inject fakes.
- **Bounded memory on long subprocesses**: external-tool output handling is
  line-based or byte-capped (no whole-stream accumulation).
- **Typed errors + retry policy**: `internal/retry/` provides `Do(ctx, policy, fn)`
  with `ErrTransient` / `ErrPermanent` sentinels. `vmquery.HTTPError` and
  `ConnError` classify via `Is(target)` so callers can ask "is this
  retryable?" without string matching.
- **Sequential tenant processing**: backend vmstorage cannot tolerate
  concurrent tenant migrations. The pipeline driver enforces this
  structurally — no goroutines span tenants. (Shard-level parallelism within
  a tenant would be OK if added later.)
- **Cobra subcommand routing**: each subcommand file owns an unexported
  cobra constructor (`new<Name>Cmd`); `root.go` adds them directly (no
  `init()` registration dance).
- **Typed in-process loop actions**: `migration-loop.json` carries a compact
  shared `runtime` block and a per-step `action.kind`. The loop and service
  dispatch Go package APIs directly.

## Known limitations

- `tenants dedicated` and `tenants essential`
  shell out to the `kubectl` / `aws` CLIs and add **no SDK dependency** to
  this binary.
- Staging / production validation is the operator's responsibility — run an
  end-to-end smoke on staging before cutover.
- **SIGINT / SIGTERM**: the first signal triggers graceful shutdown — the
  in-flight job (vmctl subprocess, status write) cancels and interrupted
  jobs resume at next startup; a second signal forces immediate exit
  (status 130). Job outcomes are recorded per job in the service, not via
  process exit codes.