# Comment Draft Review Doc Plan

Date: 2026-07-02

## Current Design

The dry-run pipeline now replays an L1/oncall reply point and returns:

- `replayContextMarkdown`: internal replay context.
- `simulatedL1ReplyMarkdown`: simulated L1 customer-facing reply draft.
- `decision`: `internal_only` or `human_required`.
- `requestedContext` / `blockedReason`: safety metadata.

For the June L1 team dry run, a temporary renderer generated one Feishu document
from `/tmp/auto-oncall-june-consulting/dryrun-results.jsonl`:

- Document: `https://pingcap.feishu.cn/docx/Wyptd856Xo3UCJxVIjrcPwqunXe`
- Current checked revision: 15
- Layout: one result table with columns for ticket id, link, assignee, user
  question summary, and simulated answer.
- Human-required rows show only `Human required`.

## Problems Found

1. The "simulated answer" column is overloaded.
   It currently mixes final review status, customer-facing draft text, code
   examples, reference links, and missing-context requests in one table cell.
   This makes it unclear whether the cell is a Jira comment draft, a model
   explanation, or review metadata.

2. Feishu table cells are the wrong container for long comment drafts.
   Markdown code fences are escaped and flattened with `<br/>`, so SQL/config
   examples show up as inline text instead of real code blocks. Long replies
   such as APID-29015 and APID-29101 become hard to review.

3. Human-required rows lost the reason.
   The local dry-run output still has useful `blockedReason` and
   `requestedContext`, but the Feishu doc only displays `Human required`.
   Reviewers cannot tell whether the blocker is destructive action, production
   risk, missing cluster information, or an escalation requirement.

4. The doc does not show replay boundaries.
   A reviewer cannot see which historical comment was being simulated, which
   comments were visible, or how many comments were excluded. That makes it hard
   to verify that the generated reply did not read later L1/customer comments.

5. Source artifacts are not stable enough for repeatable review.
   The temporary JSONL/table renderers are outside the repo, and the table
   generation is not a first-class output contract. The doc therefore depends
   on ad-hoc scripts and formatting choices.

6. Some generated drafts still need post-processing before being shown as
   "comment" examples.
   Example: a short ambiguous ticket such as APID-29123 produces a reply mostly
   asking for missing context. That can be useful, but should be clearly labeled
   as "needs context" rather than presented the same way as a direct answer.

7. Jira comment rendering is not the same as GitHub/Feishu Markdown.
   Jira Cloud REST API v3 stores issue comments as Atlassian Document Format
   (ADF). The model may draft in a Markdown-like source format for readability,
   but the service must not assume raw Markdown can be posted to Jira and render
   correctly.

8. The replay currently models only one L1 reply point.
   For completely ambiguous tickets, the desired behavior is closer to a short
   interaction simulation: L1 asks a focused follow-up, waits 5 minutes for
   customer/user supplementation, then records whether there was enough new
   context to answer or whether another clarification is needed.

## Target Contract

- A Jira comment draft is a separate field named `commentDraftMarkdown`.
- `commentDraftMarkdown` is the human-readable source draft. It must use a
  Jira-safe Markdown subset that can be deterministically converted to ADF
  before posting to Jira.
- The Jira write payload, when writes are implemented, is `commentBodyAdf`.
  It is derived from `commentDraftMarkdown`; it is not generated free-form by
  the model.
- Review metadata is separate from the customer-facing draft:
  - `decision`
  - `decisionReason`
  - `requestedContext`
  - `evidenceLinks`
  - `replayTarget`
  - `visibleCommentIndexes`
  - `excludedCommentSummary`
- For `human_required`, the public doc table can show `Human required` with a
  short reason. This is internal-review metadata only and should not be part of
  the reply-format skill.
- For ambiguous tickets, the output should include a `clarificationSimulation`:
  - L1 follow-up question.
  - Whether a user/customer supplement arrived within the 5-minute window.
  - The supplement content if present.
  - The resulting next draft or the next clarification needed.
- The Feishu doc should use:
  - A compact summary table for scanning.
  - One detail section per ticket for the full comment draft.
  - Real `<pre><code>` blocks for SQL/config snippets.
  - Explicit labels: `拟 L1 外部回复草稿`, `Human required reason`,
    `Evidence / links`, and `Replay boundary`.
- The customer-facing comment draft must not contain internal-only reasoning,
  final ticket-state analysis, or statements that imply Jira/customer actions
  were already taken.
- The reply-format skill owns only how to write Jira-ready customer-facing
  comments. It should not own human-required classification policy or internal
  triage reasons.

## Jira-Safe Markdown Subset

The model should write `commentDraftMarkdown` using only:

- Plain paragraphs.
- Short bullet or numbered lists.
- Inline code using backticks.
- Fenced code blocks for SQL, shell, TOML, YAML, JSON, and logs.
- Plain URLs on their own line or in simple bullets.

Avoid in Jira comment drafts:

- Tables.
- HTML tags.
- Markdown footnotes.
- Nested lists deeper than one level.
- Strikethrough, task checkboxes, callouts, or frontmatter.
- Markdown headings unless the comment is unusually long; prefer short lead-in
  sentences instead.

## Implementation Steps

1. Update result schema naming.
   Keep backward compatibility in the HTTP response if needed, but internally
   treat the simulated customer-facing text as `commentDraftMarkdown` instead
   of a generic "answer".

2. Add a Jira comment format layer.
   Introduce a deterministic Markdown-subset-to-ADF converter for the supported
   subset above. The model writes the source draft; service code owns the Jira
   REST payload shape.

3. Add structured review fields.
   Extend the runner output with `decisionReason`, `evidenceLinks`, and a
   compact `replayAudit` object derived from `replay-target.json` and
   `excluded-comments.json`.

4. Tighten prompt instructions.
   Require Codex to separate:
   - customer-facing comment draft,
   - internal replay context,
   - missing context,
   - evidence links.

5. Add an answer-style skill asset.
   Keep the skill focused on Jira comment writing style and Jira-safe Markdown.
   Do not put human-required handling policy in this skill; that remains in the
   safety/replay prompt and schema.

6. Add ambiguous-ticket clarification simulation.
   When the ticket is too underspecified to answer, produce an L1 follow-up
   question and simulate a 5-minute wait for user/customer supplementation. If
   a reply-safe customer supplement exists in the historical window, include it
   in `clarificationSimulation`; otherwise mark `supplementReceived=false` and
   produce the next clarification ask.

7. Add a repo-owned batch report renderer.
   Move the temporary June table/doc rendering logic into a script or service
   utility with tests. Its output should be deterministic and not depend on
   positional assumptions in ad-hoc files.

8. Regenerate the Feishu document layout.
   Prefer replacing the current single-table doc with:
   - Summary section
   - Compact ticket table
   - Ticket detail sections
   - Filter and local artifact appendix

9. Add contract tests.
   Cover answerable rows, human-required rows, long code blocks, missing
   context, link extraction, Jira-safe Markdown conversion, ADF output, the
   5-minute supplement simulation, and replay-boundary metadata.

## Verification Checklist

- `npm run typecheck`
- `npm test`
- Run one dry-run for `TCID-12743` and verify:
  - L1/support/future comments are absent from `input/comments.md`.
  - `commentDraftMarkdown` is customer-facing only and uses the supported
    Jira-safe Markdown subset.
  - The generated ADF payload preserves paragraphs, bullets, inline code,
    code blocks, and links.
  - `replayContextMarkdown` is internal-only.
- Regenerate the 7-ticket June report and verify:
  - Human-required rows show `Human required: <short reason>` in the summary
    table.
  - Code examples render as code blocks, not escaped inline markdown.
  - Each ticket has evidence links and replay-boundary metadata.
  - Ambiguous tickets show the L1 follow-up and the 5-minute supplement result.
- Fetch the Feishu doc after update and inspect the rendered Markdown/XML
  structure, not just the create/update command response.

## Non-Goals

- Do not post anything back to Jira.
- Do not execute production operations or destructive actions.
- Do not change the L1 assignee JQL or the June ticket selection criteria in
  this change.
- Do not use later Jira comments as fallback context for a replay draft.