# Code diagnosis E2E record — 2026-09-02

## Scope

This record covers an authenticated, isolated dev-workspace test after the
structured `submit_code_artifact` change (#1150).  The temporary workspace was
created only for this test; no credentials are recorded here.

## Failed case

**Input:** an explicit code-diagnosis request for `pingcap/tidb` at v8.5.7,
asking about `"MemTableReaderExec"` across a newline and including the literal
`C:\\tmp`.

**Expected:** the code investigator completes and submits an artifact.

**Observed:**

- The explicit diagnosis boundary worked: exactly one `code-investigator` task
  was created.
- The task prepared and inspected SHA
  `202b7f47286a1109b5c957401d34c9358d130ae0` successfully.
- `submit_code_artifact` was invoked, so quotes, newlines, and backslashes did
  not cause the former JSON parsing failure.
- Submission was rejected with `invalid_envelope`: each supplied
  `hypothesis_assessment` requires non-empty, source-cited observations and
  evidence.
- The parent task therefore ended as `artifact_delivery_failed`, and the turn
  was marked failed because no `artifact_ready` result was delivered.

## Analysis

This is distinct from the pre-#1150 invalid-JSON defect.  The structured tool
input reaches the backend and is parsed correctly.  The remaining failure is
semantic: the model sometimes emits an incomplete optional
`hypothesis_assessment` object.  Backend validation should remain strict so
uncited diagnostic claims are not persisted.

Recommended follow-up: change the code-investigator instruction to omit
`hypothesis_assessment` unless every required field has concrete source
evidence, and add a regression test for an artifact that includes special
characters but no incident hypothesis.  A structured schema for each
assessment could provide an additional guardrail.

## Baseline passed cases

- Ordinary chat, source-investigation wording in ordinary chat, and a spoofed
  internal header: no code investigator task was created.
- Explicit v8.5.7 diagnosis: succeeded and used the fixed SHA above.
- Explicit v7.5.1 diagnosis: succeeded and used
  `7d16cc79e81bbf573124df3fd9351c26963f3e70`.

## Second round: six additional explicit-diagnosis cases

All six cases below completed with exactly one `code-investigator` task, an
`artifact_ready` task result, and an `artifact_submitted` tool result.

| Case | Selected version / fixed SHA | Focus |
| --- | --- | --- |
| `v856_executor_builder` | v8.5.6 / `ae18096e023780bb56bfce33698abec0d4640d0a` | `executorBuilder.buildMemTable` retriever selection |
| `v856_unicode_input` | v8.5.6 / same SHA | Chinese quotes, newline, Unicode symbols and full-width punctuation |
| `v856_path_focused` | v8.5.6 / same SHA | `pkg/executor/slow_query_retriever.go` and `slowQueryRetriever` |
| `v856_direct_sha` | direct fixed SHA / same SHA | `SlowQueryExtractor` output |
| `v751_slow_query_retriever` | v7.5.1 / `7d16cc79e81bbf573124df3fd9351c26963f3e70` | time-range filtering |
| `v751_predicate_extractor` | v7.5.1 / same SHA | reversed time-range behavior |

The direct-SHA case confirms the original SHA path still works.  The Unicode
case succeeded, providing additional evidence that #1150 removed the JSON
escaping failure rather than merely handling one particular ASCII input.

## Follow-up observation: inactive selected context

After the scripted set had completed, another request using the already
selected v8.5.6 SHA received HTTP 409:

`Selected code context is no longer active; load it again before analysis`.

Reloading the same version through `code-context/prepare` allowed submission.
This may be an intentional active-context/lease rule, but it conflicts with
the user expectation that a workspace keeps one selected repository version.
Confirm the intended lifetime and, if persistence is intended, investigate why
the context becomes inactive after a completed diagnosis.

## Third-round test design: state and contract coverage

The next six scenarios deliberately exclude input-escaping variations:

1. A newly selected version becomes usable for code diagnosis.
2. Selecting a new version replaces the previous version for the same repo.
3. A request carrying the replaced (stale) SHA is rejected rather than silently
   analysing the wrong revision.
4. An invalid release tag is rejected without replacing the current version.
5. An unknown 40-character SHA is rejected without replacing the current
   version.
6. A successful result is persisted with one investigator task, one submitted
   artifact, and provenance matching the active SHA.

## Third-round results

All six state/contract scenarios passed in an isolated workspace.

1. Selecting v8.5.6 activated SHA
   `ae18096e023780bb56bfce33698abec0d4640d0a`; the diagnosis completed with an
   investigator artifact.
2. Selecting v7.5.1 then replaced the workspace's TiDB source rather than
   adding a second one.
3. A code-diagnosis request carrying the replaced v8.5.6 SHA was rejected with
   HTTP 409, so analysis cannot silently run against an old revision.
4. The v7.5.1 request completed and its investigator task submitted an
   artifact, using `7d16cc79e81bbf573124df3fd9351c26963f3e70`.
5. Preparing non-existent tag `v99.999.999` returned HTTP 404.
6. Preparing all-zero SHA returned HTTP 404.  After both failures, persisted
   workspace sources still contained exactly one `git_repo` source, `TiDB
   v7.5.1`, with the v7.5.1 SHA above.

The persisted artifact table is not the storage location for code-investigator
handoffs in this flow; the success criterion was the completed investigator
task plus `artifact_ready` and `artifact_submitted` tool events.

## Cross-module diagnosis suite (in progress before transport fix)

The planned suite has ten source-only questions, all selected through explicit
code diagnosis at TiDB v8.5.6:

1. server panic/recovery;
2. SQL error logging and wrapping;
3. Go GC metrics;
4. pessimistic transaction lock waiting;
5. optimizer join-plan selection;
6. DDL job retry and state transitions;
7. Coprocessor error propagation;
8. Raftstore diagnostic logging;
9. resource-control / memory quota enforcement; and
10. slow-log observability.

The normal per-user concurrency limit is two.  A third simultaneous request is
correctly rejected with HTTP 429 and
`code_diagnosis_concurrency_limit (active_count=2, limit=2)`.  Future batches
must use concurrency two, not treat that response as a product failure.

Successful completed cases before the fix include panic/recovery, SQL error
logging, Go GC metrics, pessimistic lock waiting, and optimizer join-plan
selection.  The DDL case reached source inspection and invoked the submission
tool but did not become `artifact_ready`; several other queued cases showed
the same symptom.  The original shared test workspace also reached its $1.00
spending cap; that is a test-workspace safety limit, not a diagnosis defect.

## Artifact delivery root cause and local fix

The DDL tool event provided the decisive evidence:

- `submit_code_artifact` was completed with an `artifact` field containing a
  JSON **string**;
- its metadata correctly included `opencode/subagent.parentTaskCallId` and
  `subagentName=code-investigator`;
- the backend accepted only an object-valued `artifact` (or legacy
  `artifact_json`), returned no submission acknowledgement, and the parent
  Task consequently failed with `missing_code_artifact_submission`.

Local changes now address both observed failure modes:

- `backend/investibook/core/agent_runtime.py` accepts a string-valued
  `artifact` as the canonical envelope. The existing persistence layer still
  performs strict JSON/schema/fixed-SHA/evidence validation.
- `agentkit/config/investibook/agents/code-investigator.md` makes
  `hypothesis_assessment` genuinely optional. The agent must omit it unless it
  is assessing a concrete incident hypothesis; if included, every required
  field must be source-cited.
- `backend/tests/test_acp_workspace_evidence.py` adds a regression test for a
  serialized structured artifact containing quotes, newlines, and backslashes.

Validation after the local fix: `32 passed` for
`backend/tests/test_acp_workspace_evidence.py`; Ruff passed for the changed
Python files.

## Remaining post-deployment verification

After this fix is deployed to dev, resume the cross-module suite with fresh,
isolated workspaces and at most two concurrent turns.  A case passes only when
it has one code-investigator Task, `artifact_ready`,
`artifact_submitted`, and a completed parent turn. Re-run the previously
affected DDL case first, then Coprocessor, Raftstore, resource-control/memory
quota, and slow-log cases; record any source-question limitation separately
from a delivery failure.

## Clarification on the earlier HTTP 409 observation

The observed 409 was reproduced when a workspace had already switched from
v8.5.6 to v7.5.1 and a subsequent request still carried the old v8 SHA. This
is expected stale-version protection, not an inactive-context lifetime bug.