# Watchdog Downstream Environment And Contract Closure Plan

## Goal

This document defines how to close the downstream test-environment and
assembly-contract gaps fast enough that the watchdog system stops primarily
detecting setup noise and starts primarily detecting real business defects.

Target state:

1. readiness failures are rare and explicit
2. skipped scenes mean intentional defer, not missing basic assembly
3. failed business cases are mostly real product or contract bugs

## Current Problem Statement

The current watchdog stack is still spending too much signal on downstream env
assembly gaps.

Primary evidence:

1. current API watchdog run: `30 passed / 14 failed / 9 skipped`
2. `#1111` says most staging failures are configuration / metadata absence
3. `#1112` proves the new env lacks much of the old overlay shape
4. `#1124` shows quote / approval / order blockers still come from env shape

The result is a false testing economy:

1. a failed or skipped case often does not mean business logic is wrong
2. it often means the downstream env is not yet the intended test object

## Closure Principles

The tested object in this repo is the downstream assembled pack plus downstream
bootstrap / deploy contract, not a generic upstream runtime.

Therefore closure must happen in downstream-owned surfaces:

1. final pack assembly
2. env bootstrap
3. remote deploy
4. shared seed import
5. readiness verification

Do not try to paper over missing downstream assembly only in case logic.

## Contract Boundary Rule

The downstream team owns the boundary between:

1. global environment contract
2. scene-specific readiness contract

That boundary must stay explicit and stable.

### Global environment contract

Put a dependency in the global pre-run contract only when it is:

1. shared by most watchdog cases
2. stable across runs
3. cheap enough to maintain as common baseline
4. required to prove the environment is the intended downstream test object

Current examples:

1. overlay / install-shape minimum
2. shared baseline seed
3. shared quote/order baseline graph
4. security bootstrap required by broad actor-visibility coverage
5. pack-installed metadata such as connected app / installed object presence

### Scene-specific readiness contract

Keep a dependency in scene readiness when it is:

1. needed only by one scene family or a small subset of cases
2. mutation-heavy during execution
3. expensive or noisy to maintain globally
4. closer to scene setup than to environment identity

Current examples:

1. approval definitions and approver routing
2. approval probe quote state graph
3. reporting fixtures
4. webhook rows and integration-flow config

### Decision rule for future changes

When adding a new preflight check:

1. do not add it to the global pre-run gate by default
2. first ask whether unrelated scenes really depend on it
3. if the dependency is narrow or mutation-heavy, add or refine a scene readiness bucket instead
4. only promote it to global contract after the repo docs, seed ownership, and workflow gate all agree

This rule exists to prevent two failure modes:

1. every watchdog run being blocked by heavy scene-local config
2. docs, seeds, and workflow checks drifting into different definitions of "environment ready"

## Closure Scope

This plan focuses on four downstream closure layers:

1. overlay / metadata presence
2. shared business-fixture presence
3. actor / approval / security env bindings
4. readiness gate shape

## Layer 1: Overlay And Metadata Closure

## Problem

Current envs can still come up without the intended PingCAP business shape.

Typical symptoms:

1. missing `__c` fields
2. missing labels / record types
3. missing business-specific actions / tabs / metadata
4. describe surfaces drift from old staging expectations

## Closure Actions

### P0

1. freeze a downstream overlay contract
   - define the minimum business metadata expected in watchdog env
   - include:
     - key fields
     - labels
     - record types
     - layouts or actions where business flow depends on them

2. make deploy/bootstrap fail closed if overlay contract is not present
   - do not let watchdog run against a greenfield org shape accidentally

3. add a single post-deploy overlay verify step
   - query describe / metadata probes for key objects:
     - `Campaign`
     - `Lead`
     - `Quote`
     - `Order`
     - approval/security admin objects if used by scenes

### Deliverables

1. a machine-readable overlay-minimum checklist
2. one deploy-time verify script or script block
3. one readiness artifact that records pass/fail before watchdog starts

## Layer 2: Shared Business Fixture Closure

## Problem

Many current failures are not logic defects. They are missing shared graph.

Typical missing graph:

1. standard pricebook
2. products
3. pricebook entries
4. source opportunity with OCR and OLI
5. quote / order / contract probe graph
6. reporting seed
7. integration config / webhook fixtures

## Closure Actions

### P0

1. promote all stable reusable graph into shared baseline seed
   - examples:
     - `product-catalog`
     - `opportunity-core`
     - `account-contact-core`
     - stable quote/order source fixtures where mutation is not required

2. keep mutation-heavy records scenario-local
   - approval probe quote
   - upload targets
   - actor-owned visibility contrast rows if mutation is expected

3. version the seed baseline by contract, not by ad-hoc manual data
   - every required record needs:
     - stable logical key
     - owning bucket
     - unlocking case list

4. run shared-seed verify immediately after deploy and before case execution
   - this verify should prove:
     - shared graph exists
     - graph relationship chain is intact
     - actor visibility for those rows is sane

### Deliverables

1. updated `watchdog/seeds/import-buckets.yaml` ownership clarity
2. expanded shared-seed verification checklist
3. one baseline-seed health artifact per run

## Layer 3: Actor / Approval / Security Binding Closure

## Problem

Several scene families are not blocked by CRUD. They are blocked by env-owned
bindings that cases cannot self-heal.

Examples:

1. approval definitions and approver routing
2. order record-type availability
3. contrasted security actors
4. permission-set / role / field-permission bindings

## Closure Actions

### P0

1. treat approval/security actors as env contract, not optional test garnish
2. move their presence check into readiness
3. fail the watchdog run before business cases if these bindings are absent

### Approval contract

Minimum required:

1. one active Quote-targeted approval definition
2. one submit-capable sales actor
3. one approver actor
4. one observable pending-state path

### Order contract

Minimum required:

1. required Order record type exists
2. watchdog actor can see / use it
3. create-from-quote path is not blocked by metadata absence

### Security contract

Minimum required:

1. restricted actor can authenticate
2. extended actor can authenticate
3. contrasted role / permset bindings exist
4. one field-level contrast exists
5. one row-visibility contrast exists

### Deliverables

1. readiness checks for approval, order metadata, and security actors
2. explicit case-independent env validation output

## Layer 4: Readiness Gate Reform

## Problem

Current readiness is still too coarse in some scenes.

Bad current outcome:

1. a scene is marked skipped
2. but the operator still cannot tell whether the real gap is:
   - overlay absence
   - seed absence
   - actor absence
   - approval config absence
   - reporting config absence

## Closure Actions

### P0

1. split readiness into sub-contract buckets
   - `overlay`
   - `baseline-seed`
   - `approval-env`
   - `order-metadata`
   - `security-bootstrap`
   - `reporting-fixture`
   - `integration-config`

2. keep the global pre-run gate limited to shared stable buckets
   - `overlay`
   - `baseline-seed`
   - `order-metadata`
   - `security-bootstrap`
   - install-owned integration metadata
3. use scene readiness buckets for heavier narrow dependencies
   - `approval-env`
   - `reporting-fixture`
   - webhook/config-heavy integration checks
4. make scene gating depend on those readiness buckets explicitly
5. when a scene is skipped, emit the exact failed readiness bucket

### Deliverables

1. one structured readiness summary artifact
2. one explicit mapping:
   - scene -> required readiness buckets

## Execution Sequence

The fastest closure path is not "fix all failed cases one by one".

It is:

### Wave 1: Make env contract explicit

1. define overlay minimum
2. define shared-seed minimum
3. define approval/order/security minimum
4. define scene-to-readiness-bucket mapping

Expected result:

1. fewer ambiguous skips
2. fewer hidden greenfield-env runs

### Wave 2: Make deploy/bootstrap fail closed

1. deploy must verify overlay-minimum presence
2. deploy must apply or verify baseline shared seed
3. deploy must verify approval/order/security bindings

Expected result:

1. readiness scenes no longer fail because the target env is the wrong system

### Wave 3: Tighten watchdog run semantics

1. do readiness verify first
2. stop the run early when env contract is broken
3. only then execute business scenes

Expected result:

1. business-case failures become much more trustworthy

### Wave 4: Reclassify residual failures

Once waves 1-3 land:

1. remaining skipped cases become true deferred scope
2. remaining failed cases are much more likely to be:
   - API contract bugs
   - logic bugs
   - business behavior defects

## Closure Backlog Matrix

This is the implementation backlog view of the downstream env closure work.

| Closure Item | Surface | Priority | Current Symptom | Owner Slice | Output |
| --- | --- | --- | --- | --- | --- |
| overlay-minimum contract | pack/deploy | P0 | greenfield env shape slips through | deploy/bootstrap | machine-readable overlay checklist |
| post-deploy overlay verify | deploy/readiness | P0 | missing fields/labels detected too late by cases | deploy/bootstrap | readiness artifact with key-object parity result |
| shared baseline seed contract | seed/import | P0 | case failures caused by missing reusable graph | seed/baseline | stable bucket ownership + required rows |
| baseline-seed verify | seed/readiness | P0 | missing product/quote/order graph only discovered inside case execution | seed/baseline | baseline health artifact |
| approval-env contract | env metadata | P0 | approval cases fail or skip ambiguously | approval/order | explicit approval readiness bucket |
| order-metadata contract | env metadata | P0 | Order path blocked by record-type ambiguity | approval/order | explicit order readiness bucket |
| security-actor contract | env bootstrap | P0 | security cases skip because contrasted actors are absent or unusable | security/bootstrap | explicit security readiness bucket |
| scene-to-readiness-bucket mapping | workflow/reporting | P0 | scene skip is too coarse | readiness/reporting | structured scene dependency map |
| reporting fixture contract | env bootstrap | P1 | reports scene skip hides exact missing setup | reporting | explicit reporting readiness bucket |
| integration config contract | env bootstrap | P1 | integration scene skip hides exact missing config | integration | explicit integration readiness bucket |

## Suggested Execution Slices

These are the smallest slices likely to move the system quickly.

### Slice A: Readiness contract skeleton

1. scene-to-readiness-bucket mapping
2. readiness summary artifact
3. fail-closed skip reporting

Expected outcome:

1. every skip points to a named missing contract bucket

### Slice B: Overlay and baseline prove-before-run

1. overlay-minimum contract
2. post-deploy overlay verify
3. baseline-seed verify

Expected outcome:

1. watchdog stops running on the wrong downstream shape

### Slice C: Approval / Order env closure

1. approval-env contract
2. order-metadata contract
3. quote/order shared graph verify

Expected outcome:

1. quote / approval / order failures shift from env ambiguity to real behavior

### Slice D: Security actor closure

1. contrasted actor bootstrap
2. role / permset / assignment verify
3. security readiness bucket

Expected outcome:

1. security scene stops being skip-heavy and starts exposing true permission defects

### Slice E: Reporting and integration closure

1. reporting fixture contract
2. integration config contract

Expected outcome:

1. remaining deferred scenes become explicitly scope-deferred, not assembly-noisy

## Sequence Dependency Rules

Do not execute this in arbitrary order.

1. readiness bucket mapping must land before adding more coarse skip cases
2. overlay verify should land before describe parity cases are treated as business blockers
3. baseline-seed verify should land before quote/order/browser upload flows are treated as runtime blockers
4. actor/approval/order closure should land before claiming business maturity in those scenes

## Fastest Path To Trustworthy Signal

If only two slices can be funded first, do these:

1. Slice A: readiness contract skeleton
2. Slice B: overlay and baseline prove-before-run

Reason:

1. these two slices reduce the largest amount of false business signal
2. they improve every scene, not just one feature family

## Cross-Link To Case Closure

This env-contract plan feeds:

1. [watchdog-issue-1111-1112-1124-case-closure-plan.md](/Users/bingwang/github/nexuscrm-pingcap-pack/.worktrees/exec-watchdog-issue-1111-1112-1124-plan/docs/watchdog-issue-1111-1112-1124-case-closure-plan.md)

That companion document defines which new API/browser cases should be added once
the downstream env signal is trustworthy enough.

## Ownership Split

To move quickly, split by downstream surface instead of by issue.

### Owner A: Deploy / Bootstrap Contract

1. remote deploy env shape
2. fail-closed verify
3. post-deploy readiness artifact

### Owner B: Shared Seed Contract

1. shared baseline fixture graph
2. bucket ownership
3. baseline-seed health verification

### Owner C: Approval / Order Metadata Contract

1. approval definition readiness
2. order record-type readiness
3. quote/order/contract env graph

### Owner D: Security / Actor Contract

1. contrasted actors
2. roles / permsets / assignments
3. security readiness proof

## Done Definition

This closure plan is complete when all of the following are true:

1. a watchdog env cannot silently come up without required downstream overlay
2. shared seed graph is verified before business scenes run
3. approval / order / security env contracts are verified before dependent scenes run
4. scene skips identify exact missing readiness buckets
5. business-scene failures mostly indicate real business or API defects

## Success Metric

Use these metrics after the closure waves land:

1. readiness skips drop sharply
2. skipped cases are mostly intentional defers, not missing assembly
3. business-contract failure rate becomes more stable run-to-run
4. issue triage shifts from "is env missing shape?" to "is behavior wrong?"

## Final Intended State

The watchdog system should stop answering:

1. "did we forget to assemble the downstream test object?"

And start answering:

1. "given the correct downstream test object, what real business defects remain?"

That is the bar for saying the downstream env and contract assembly line has
been closed enough for trustworthy regression signal.