# Web E2E Testing

本文是 AgentTeam Web E2E harness 的 canonical 文档。

适用范围：

- `make e2e-web`
- `web/package.json` 中的 `e2e*` 脚本
- `web/e2e/*` Playwright 用例与 helper
- `cmd/agentteam-e2e-server`

## Goals

- 以真实前后端跑用户旅程，而不是测试实现细节
- 让 suite 保持 deterministic、可复跑、环境独立
- 把重型启动成本收敛到可复用 harness，而不是每次单测都全量重启

## Regression Policy

- Documentation-only changes do not require regression.
- Any change to code, runtime behavior, schema, build outputs, or test fixtures must run the appropriate regressions before completion.
- Features defined in `SPECS.md` and canonical product specs should be covered by E2E journeys where user-visible behavior matters.

## Commands

Full one-shot run:

```bash
make e2e-web
```

`make e2e-web` 会先自动执行 `make ensure-web-deps`，再确保 `web/dist` 已构建，然后进入 full CI suite。Web preview 产物由 `scripts/ensure-web-dist.sh` 维护：它用 E2E 专用的 Vite-only build 生成 `web/dist`，并在源码指纹未变化时跳过重复构建。`make e2e-web` 现在只是 `scripts/ci-web-e2e-full.sh` 的 thin wrapper，这样本地和 Jenkins 复用同一条 repo-owned 路径。这个 wrapper 会为 `web/node_modules`、`web/dist` 和 runner execution 打印粗粒度 `[e2e:timing]` 日志。

Required browser smoke:

```bash
cd web && npm run e2e:ci:smoke
```

`npm run e2e:ci:smoke` 会走 `scripts/ci-web-smoke.sh`：准备 `web/node_modules`、通过 `scripts/ensure-web-dist.sh` 确保 `web/dist` 可用，然后以 preview frontend 跑 `ci:smoke` mode。这个 mode 固定只跑 `e2e/smoke.spec.ts`，并强制 `workers=1`、`retries=2`。这个 wrapper 会为 `web/node_modules`、`web/dist` 和 runner execution 打印粗粒度 `[e2e:timing]` 日志。

or:

```bash
cd web && npm run e2e
```

First-time browser setup:

```bash
cd web && npm run e2e:install
```

Run only the default CI suite or a selected lane:

```bash
cd web
npm run e2e:ci:plan
npm run e2e:ci:default -- --lane=ui-auth
npm run e2e:ci:default -- --lane=ui-auth --list
```

CI job entry for one lane:

```bash
make e2e-web-lane lane=ui-auth
cd web
npm run e2e:ci:lane -- ui-auth
npm run e2e:ci:lane -- ui-auth --list
```

`e2e:ci:plan` prints the current wave/lane plan as JSON without starting a harness. CI systems that split the full suite should use that output, or the table below, as the source of truth instead of duplicating spec lists. `e2e:ci:lane` is a thin CI wrapper around the existing lane scheduler. It validates the lane against the runner-owned plan before dependency or dist preflight, prepares `web/node_modules` and `web/dist`, then runs exactly one existing lane through `ci:default --lane <name>`. The wrapper does not define lane membership, workers, retries, or isolation; those remain owned by `web/e2e/run-web-e2e.mjs`. With `--list`, the wrapper only prepares `web/node_modules` and skips dist preflight because Playwright list-only does not start a harness.

Local fast rerun flow:

```bash
cd web
npm run e2e:up
npm run e2e:reuse -- e2e/auth.spec.ts -g "register, log out, and log back in with a 6-character password"
npm run e2e:down
```

### Retained Service Account Acceptance

服务号的最终 Human 验收使用独立、长期保留的 AgentTeam server 和 TiDB database，
而不是普通 Web E2E harness。它覆盖三个独立浏览器身份、两个 Workspace、
移动端 publisher，以及以下完整链路：

- Workspace 服务号 direct-create、申请 approve/reject、publisher 发消息、全员未读与只读收件
- server owner 创建平台官方号并向多个 Workspace 广播
- 私有自定义 App 跨 Workspace 发起接入申请，经 Approval approve/reject，并向申请人返回结果通知

先构建当前 Web 产物，并确保指定服务由这个 `web/dist` 提供静态资源：

```bash
cd web
npm run build
```

然后在独立环境已经 ready 的前提下运行：

```bash
cd web
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE=1
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_URL=http://127.0.0.1:33095
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_RUN_ID="service-accounts-$(date +%Y%m%d-%H%M%S)"
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_FIXTURE_FILE="$PWD/../.data/service-account-acceptance/fixture-map.json"
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_ARTIFACT_DIR="$PWD/../.data/service-account-acceptance/artifacts"
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_OUTPUT_DIR="$PWD/../.data/service-account-acceptance/playwright-results"
npm run e2e:service-accounts-acceptance
```

首次运行后的数据库已经有 server owner。复跑同一保留数据库时，额外提供该
owner 的本地验收凭据，让 suite 继续通过 UI 登录并执行平台官方号步骤：

```bash
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_SERVER_OWNER_EMAIL='owner@example.test'
export AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_SERVER_OWNER_PASSWORD='local-acceptance-password'
```

环境契约：

- `AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_URL` 必须指向专用 AgentTeam Web/API；不要指向共享开发栈。
- 专用服务必须使用独立 TiDB database 和端口；suite 不创建、清空或绕过产品数据。
- 每轮 suite 都通过 Web UI 注册三个 Human、创建 Workspace、邀请成员、创建/申请/审批/发布/收件；不得用 `page.request`、内部 API、mock 或直接插库替代用户动作。
- suite 使用 realtime 有界等待和语义 selector，不使用固定 sleep。
- `AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_RUN_ID` 必须唯一，以免与保留 fixture 冲突。
- 同一保留数据库复跑时，必须成对设置 `AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_SERVER_OWNER_EMAIL` 和 `AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE_SERVER_OWNER_PASSWORD`；suite 仍会为该轮 Workspace 链路注册三个新 Human。
- 成功后保留 server、TiDB、fixture map、日志和截图供复核。fixture map 含本地验收账号密码，只能放在被忽略的 `.data/` 目录，不得提交。
- 只有显式设置 `AGENTTEAM_SERVICE_ACCOUNT_ACCEPTANCE=1` 时该 spec 才会执行；它不属于普通 CI lane。

Notes:

- `e2e:reuse` skips backend and frontend restart when the harness is still healthy.
- If backend code changed, run `npm run e2e:restart` before reusing the harness.
- `make e2e-web` currently delegates to the CI-oriented full-suite script `scripts/ci-web-e2e-full.sh`.
- `npm run e2e:ci:smoke` is the required browser-smoke path and allows raw Playwright args such as `--list` because it only scopes a single smoke spec, not the lane scheduler.
- `npm run e2e:ci` runs all CI waves. `npm run e2e:ci:default` runs the same lane scheduler entry point for default CI usage.
- `npm run e2e:ci:plan` prints the canonical wave/lane plan for external CI wiring.
- `npm run e2e:ci:lane -- <lane>` is the recommended entry when CI splits the full suite into lane jobs. Preserve the wave order: run `ui-auth` and `ui-collab` in parallel first, then `runtime-daemon`, and finally the isolated `performance` lane. This keeps the existing lane topology while keeping timing budgets out of contended waves.
- The lane-scheduled `ci` and `ci:default` commands only accept runner-level `--lane <name>` / `--lane=<name>` and `--list`. They intentionally reject raw Playwright args such as `--shard` so CI cannot bypass the repo-managed lane topology.
- `AGENTTEAM_E2E_WORKERS` still overrides Playwright worker count for ad-hoc local commands such as `npm run e2e -- ...` and `npm run e2e:reuse -- ...`, but CI lanes pin their own worker counts. `AGENTTEAM_E2E_RETRIES` still overrides retry count; otherwise retries keep Playwright's previous behavior (`2` only when `CI` is set, `0` locally).
- Timing logs use a stable key-value format on stderr, for example `[e2e:timing] key=wave plan=ci wave=wave-1 status=0 duration_ms=12345`. The runner logs `full_runner`, `wave`, `suite`, `backend_ready`, `frontend_ready`, `playwright`, `cleanup`, and `backend_binary_cache`; shell wrappers log `ensure_web_node_modules`, `web_dist`, and `runner_execution`. Backend and frontend startup are intentionally parallel inside a fresh harness, so `suite` startup time is bounded by the slower of `backend_ready` and `frontend_ready`, not their sum. The lane wrapper includes `lane=<name>` on its wrapper timing lines.

## Harness Topology

Single source of truth:

- Backend is started by `cmd/agentteam-e2e-server`.
- Frontend is started by the Vite dev server for `up` / `reuse` and local ad-hoc runs, or by `vite preview` for `ci*` runs.
- Playwright attaches to an already-running harness in `reuse` mode.

Current backend behavior:

- uses a fresh TiDB database per fresh harness
- starts its own local TiDB runtime in `unistore` mode from the TiUP-managed `tidb` component unless `AGENTTEAM_E2E_TIDB_DSN`, `AGENTTEAM_TIDB_TEST_DSN`, `AGENTTEAM_E2E_TIDB_ADMIN_DSN`, or `AGENTTEAM_TIDB_TEST_ADMIN_DSN` is set
- when an admin DSN is provided, creates and later drops a unique harness database on that existing TiDB instance
- one-shot, CI, and persistent `npm run e2e:up` fresh starts cache the `cmd/agentteam-e2e-server` binary under
  `${XDG_CACHE_HOME:-~/.cache}/agentteam/web-e2e-backend-binaries`; set
  `AGENTTEAM_E2E_BACKEND_BINARY_CACHE_DIR` to override the cache location. The cache key includes platform, architecture, relevant Go build environment, and backend source fingerprint, so unchanged local reruns reuse the binary while source or keyed Go environment changes produce a new cache entry.
- exposes `/api/*`, `/socket.io/*`, and `/internal/*`

Shared TiDB fast-start example:

```bash
export AGENTTEAM_E2E_TIDB_ADMIN_DSN='root@tcp(127.0.0.1:4000)/?multiStatements=true&parseTime=true&loc=UTC'
make e2e-web
```

Use `AGENTTEAM_E2E_TIDB_DSN` only when a single preselected database is acceptable. Prefer the admin DSN for full or sharded runs because each harness gets an isolated database. If no DSN is configured, the harness still creates an isolated database on its own temporary local `unistore` runtime.

Current frontend behavior:

- both `vite dev` and `vite preview` use `VITE_DEV_PROXY_TARGET` to point the web app at the E2E backend
- `ci*` commands require an existing `web/dist/index.html`
- `ci*` commands use `scripts/ensure-web-dist.sh` before launching the runner; this path intentionally builds preview assets with `npm run build:e2e` instead of the full `npm run build` typecheck gate
- `ci*` commands isolate Playwright `outputDir` / HTML report per lane under `test-results/<lane>` and `playwright-report/<lane>`

The backend URL is exported to Playwright as `AGENTTEAM_E2E_BACKEND_URL`. Helper utilities such as [`web/e2e/e2eMailbox.ts`](../../web/e2e/e2eMailbox.ts) depend on this.

### CI Lane Topology

Default CI waves:

- `wave-1`
  - `ui-auth`: `smoke`, `auth`
  - `ui-collab`: `inbox`, `channel`, `tasks`, `taskNotifications`, `drafts`, `attachments`, `knowledgeBases`, `scheduledMessages`, `workflows`, `mobileComposerFocus`, `humanDm`, `replyThread`
- `wave-2`
  - `runtime-daemon`: `runtime`, `daemonTasks`, `messageAgent`, `opencode`
- `wave-3`
  - `performance`: `perf` (`workers=1`, `retries=0`, `AGENTTEAM_E2E_PERF=1`, `AGENTTEAM_E2E_PERF_RETAINED_SURFACE=1`, `AGENTTEAM_E2E_PERF_TIMING_MODE=baseline`)

Lane contract:

- each lane gets an isolated backend, database, frontend preview server, and Playwright artifact directories
- at most two lanes run concurrently, matching the fixed wave layout above
- external CI lane jobs must preserve wave ordering; do not flatten `runtime-daemon` into the same parallel stage as `wave-1`
- `smoke.spec.ts` is the required browser-smoke suite and owns the login page open + register/login happy path + Inbox landing checks
- `auth.spec.ts` is reserved for non-required auth edge coverage such as duplicate registration and wrong-password flows
- `runtime-daemon` builds and runs the pure Go daemon binary without a frontend-asset prerequisite
- `performance` runs alone after the functional waves with one worker and no retries; it controls API latency deterministically, asserts request counts/blank frames/CLS/long tasks, and attaches `performance-metrics.json` while Playwright retains failure trace, screenshot, and video

## Email And Mobile Verification Flows

Do not depend on real email or SMS delivery in E2E.

Use the test-only endpoints exposed by `cmd/agentteam-e2e-server`:

- `GET /internal/e2e/mailbox/latest?kind=invite|reset|verify&email=<email>`
- `GET /internal/e2e/mobile-verification/latest?mobile=<mobile>`

These endpoints return the latest verification token or code for the requested target.

When email verification is enabled in Web E2E, tests should complete the verification flow via `/?verify=<token>` or shared helpers such as [`web/e2e/emailVerification.ts`](../../web/e2e/emailVerification.ts).

## Writing Standards

Prefer semantic selectors:

- `getByRole`
- `getByLabel`
- `getByPlaceholder`
- `getByText` when scoped and stable

Avoid CSS or XPath selectors tied to implementation details.

If stable semantics are missing, fix the UI instead of making the test brittle:

- add `label` / `htmlFor` / `id` for form controls
- add minimal `data-testid` only where semantic queries are still insufficient

Keep tests deterministic:

- use unique emails and names per run
- use shared session helpers for non-auth journeys; they seed accounts through the API
  and still verify email through the E2E mailbox instead of replaying the full sign-up UI
- keep tests serial unless safety is already proven
- avoid parsing logs as a product behavior mechanism

## Performance Gate

The isolated `performance` lane owns route and first-frame budgets. Run it locally with the same lane contract:

```bash
cd web
npm run e2e:ci:lane -- performance
```

For fast harness reuse while editing the spec:

```bash
cd web
npm run e2e:up
AGENTTEAM_E2E_PERF=1 AGENTTEAM_E2E_PERF_TIMING_MODE=baseline npm run e2e:reuse -- e2e/perf.spec.ts
npm run e2e:down
```

The initial budgets are hot navigation p95 `<= 120ms`, cold client work p95 `<= 180ms` after subtracting the fixture delay, committed-Surface blank frames `= 0`, unexpected duplicate bootstrap requests `= 0`, CLS `<= 0.05`, navigation long-task p95 `< 50ms`, and no single long task `> 90ms`. Channel-to-channel sampling starts when the destination pathname commits and accepts only the exact destination conversation scope as a committed Surface; the previous scope cannot satisfy the nonblank invariant under the new URL. Correctness metrics (request invariants, semantic blank-frame sampling, CLS, and long tasks) are always strict. The isolated CI lane explicitly uses `AGENTTEAM_E2E_PERF_RETAINED_SURFACE=1` for the landed retained-Surface contract and `AGENTTEAM_E2E_PERF_TIMING_MODE=baseline`, which records but temporarily does not gate only duration/client timing while a stable preview baseline is collected; omit the timing-mode variable or set it to `strict` to enforce timing too. Any other timing-mode value fails closed. Override timing limits with the `AGENTTEAM_E2E_PERF_*` variables defined in `e2e/perfMetrics.ts`.

Desktop builds emit `agentteam:desktop-static-underlay-dom-ready` after the parser-time inline bootstrap mounts its underlay and `agentteam:desktop-react-shell-commit-reveal` immediately before the committed shell removes it. These marks intentionally describe DOM/commit semantics rather than browser paint. A real Desktop harness can combine them with the native process/app-start timestamp; Browser Playwright must not infer native process startup from Navigation Timing.

## Machines And Agents

Do not depend on real Codex or other LLM access in E2E.

Current daemon helpers:

- [`web/e2e/agentteamDaemon.ts`](../../web/e2e/agentteamDaemon.ts): default helper for first-party `agentteam daemon`

The daemon helper caches built `agentteam` binaries by daemon source fingerprint under
`${XDG_CACHE_HOME:-~/.cache}/agentteam/web-e2e-daemon-binaries` so parallel Playwright
workers and later reruns do not rebuild the daemon binary for unchanged sources.
Set `AGENTTEAM_E2E_DAEMON_BINARY_CACHE_DIR` to keep this cache in a custom location.

Rules:

- the helper must inject the same fake codex binary from `e2e/testdata/fake-codex.mjs`
- default Web E2E coverage should use `agentteamDaemon.ts`
- daemon-driven runs build only the pure Go daemon binary; the browser frontend continues to use `web/dist`

## Where To Update What

- Update this doc when the harness contract, commands, test-only endpoints, or daemon-helper policy changes.
- Update `AGENTS.md` only for short testing policy and index pointers.
- Update `README.md` only for quick-start commands and links.

## Electron desktop smoke

The native Desktop lane lives under `desktop/electron/e2e`. It launches a real
Electron process through Playwright and builds the real Go sidecar from
production packages. The lane always uses an isolated `AGENTTEAM_CONFIG_DIR`;
credentials are never written to Playwright logs or argv.

```bash
cd desktop/electron
AGENTTEAM_ELECTRON_SMOKE=1 npm run test:e2e
```

The command builds the renderer plus Electron main/preload before launching
the lane. Individual smoke cases may build focused Go probe binaries as part
of their own setup.