# Remote Dev Full On AWS

Read [`safety-and-ownership.md`](safety-and-ownership.md) before mutation. If product/account/session scope is unclear, return to [`../SKILL.md`](../SKILL.md).

Remote Dev full is the shared CP9 runtime for Premium and Dedicated control planes plus optional Starter, Essential V2, UI integrations, and BYOC. It runs the controller, builds, image import, k3d, Tilt, and control-plane workloads on one session-owned EC2 host. Product dataplanes and the Starter regional substrate remain on their AWS runtimes.

## Configure

Copy [`../../../cp9.dev-full.remote-ec2-k3d.example.yaml`](../../../cp9.dev-full.remote-ec2-k3d.example.yaml) to a gitignored Session file. Its comments are the complete config source of truth; do not copy field descriptions into this reference.

Edit only task-owned values. The Session Tiltfile owns image-source selection and the service graph. Copy `personas/dev-full/runtime/example.Tiltfile` to a gitignored `Tiltfile.<user>` when selecting source-built services, then point `tilt.file` at that copy.

Before `up`, resolve:

- exact account, region, owner, unique Session name, TTL, and source refs;
- runtime-account `cp9-developer` role and account-specific wildcard certificate;
- EC2 key/subnet/client CIDR and architecture compatible with selected images;
- optional product blocks required by the task.

The authenticated AWS identity determines the control-plane account and owner. The Session name determines the runtime profile, state path, tags, and resource prefixes. CP9 derives the AL2023 AMI. Staging is an immutable source input, not a runtime to patch.

## Preflight

Complete the shared `setup -> doctor -> up dry-run` gate in [`safety-and-ownership.md`](safety-and-ownership.md). Stop on a required red check. For handoff-quality evidence, rerun doctor as JSON and report `checks[].failure_class`, `field`, and `remediation`.

`setup` may update the local release checkout and binary but creates no cloud resource. `doctor` and `up dry-run=true` are the no-mutation gates. They do not authorize a different account, Session, or source boundary.

## Run And Observe

Run `bin/cp9 up config=cp9.yaml`. It emits a baseline before lifecycle mutation and keeps the same actionable watch through runtime/product convergence. Continue only when it exits `ready` with code 0.

Ctrl-C during lifecycle preserves resumable state; rerun the same `up`. After lifecycle completion, Ctrl-C only detaches observation; reattach with `bin/cp9 status --watch config=cp9.yaml`.

Use:

- JSON `cp9 status` for authoritative lifecycle, host, Tilt, image-source, UI, O11Y, product-capability, artifact, and cleanup layers;
- `cp9 logs ... limit=<n>` for bounded session-scoped logs;
- `cp9 deploy ... dry-run=true` before source-service deployment;
- `cp9 deploy ...` only for a source-owned service, and require all selected Tilt resources to converge;
- `cp9 artifacts resolve/refresh` for immutable owner-artifact selection and explicit refresh.

The local Tilt URL and kube API are SSH forwards. One systemd Tilt controller on EC2 owns the runtime. Repair it through the same `cp9 up`; do not start a second local controller or edit internal state.

## Product Capabilities

| Capability | Owning surface | Readiness boundary |
| --- | --- | --- |
| Premium | [`../../premium/SKILL.md`](../../premium/SKILL.md) | Top-level Dev full ready plus the exact product observation requested. |
| Dedicated | [`../../dedicated/SKILL.md`](../../dedicated/SKILL.md) | Top-level Dev full ready plus matched Dedicated route/product observation. |
| Starter / Essential v1 | [`../../starter/SKILL.md`](../../starter/SKILL.md) | `cp9 starter status=ready`; eager and day-2 activation use one engine. |
| Essential V2 | [`../../essential-v2/SKILL.md`](../../essential-v2/SKILL.md) | `cp9 essential-v2 status=ready`. |
| AWS BYOC Region | [`../../byoc/SKILL.md`](../../byoc/SKILL.md) | `cp9 byoc status=ready` with every activation layer ready. |
| Browser UI | `cp9 status` and `cp9 browser-acceptance` | Auth0/account prerequisites plus a separately recorded redacted browser result when requested. |
| Shared O11Y | `layers.o11y` in JSON status | Persisted/live dimensions reported independently; absent logs, alerts, or DrHouse remain explicit. |

Infrastructure readiness is not product acceptance. Keep Product, Harness, Security, O11Y/Agent, and Cleanup results separate.

## Source Development

Fast-start services use checked-in staging-aligned image pins. Services listed in the copied Tiltfile's `source_build_services` build from the pinned source checkout on EC2; local Billing/O11Y stubs remain session-owned. Do not create a second config-level service graph.

CP9 atomically syncs selected repositories before the EC2 controller builds. Private Go modules use the approved GitHub App Device Flow and a short-lived user token; credential material must not be written into images, logs, artifacts, or remote persistent files.

Use `services.source_build_parallelism` only through the root config reference. If SSH/controller responsiveness degrades, reduce parallelism instead of adding another scheduler.

For one service:

1. `cp9 deploy ... dry-run=true` identifies the source-owned service and Tilt resources.
2. `cp9 deploy ...` overlays the current checkout/source snapshot and waits for compilation/shipping convergence.
3. Verify the live Pod/ReplicaSet image digest and runtime behavior. A successful build does not prove rollout consumption.

## UI

Remote Dev full always exposes one same-origin UI/API endpoint using the shared DNS zone and the runtime account's certificate. The default seed identity is resolved by CP9; override it only for an Auth0 identity already prepared by an administrator.

Record interactive browser acceptance only with `cp9 browser-acceptance`. Store outcome and bounded stage/path, never passwords, tokens, cookies, headers, response bodies, or traces. Auth0 client/user repair belongs to an Auth0 administrator.

## Diagnose

Start from CP9-owned evidence before live AWS/Kubernetes inspection:

1. JSON `cp9 status` and the named failing layer/action.
2. Bounded `cp9 logs` for the cited service.
3. `.cp9/<session>/` resolved config, lifecycle/Tilt/context/artifact/cleanup reports.
4. `CP9_VERBOSE=1` only for the failing command when protocol/subprocess detail is required.

For remote execution claims, cite `layers.host.execution`: controller, source sync, private-module download, build/import hosts, k3d runtime, kube tunnel/config, SSH transport, and remote state directory.

Use AWS or dataplane `kubectl` only after resolving the exact Session-owned account, region, cluster, owner, and Session from current evidence. A live patch may prove causality but does not replace a source-owned fix.

Known infrastructure failures and routing rules live in [`troubleshooting-remote-cp9-up.md`](troubleshooting-remote-cp9-up.md).

## Metadb And Kernel Images

`premium-metadb` in namespace `cp9-system` is local control-plane state, not a customer TiDB endpoint. Inspect only the owning database/tables and record narrow statements without passwords or broad dumps.

Kernel image selection has product-specific catalog/live-swap boundaries. Use [`change-kernel-images.md`](change-kernel-images.md). Control-plane services instead use the source-owned Tilt path above.

## Closeout

Use the shared `cp9 destroy` contract in [`safety-and-ownership.md`](safety-and-ownership.md). Remote closeout must additionally preserve the final host-controller/tunnel state and distinguish Session-owned AWS resources from retained customer/shared prerequisites. A concurrent task uses a new Session name and reports the old/new tracks separately.

## Completion

Report exact source SHAs, account/region/owner/Session, host-execution facts, requested product result, unverified dimensions, and cleanup receipt. Preserve the first failing artifact rather than regenerating a cleaner story.

## References

- [`../../../cp9.dev-full.remote-ec2-k3d.example.yaml`](../../../cp9.dev-full.remote-ec2-k3d.example.yaml)
- [`safety-and-ownership.md`](safety-and-ownership.md)
- [`troubleshooting-remote-cp9-up.md`](troubleshooting-remote-cp9-up.md)
- [`change-kernel-images.md`](change-kernel-images.md)
- [`cli-output.md`](cli-output.md)