# Issue #149 — Ride pre-run rerun recovery matrix

## Scope and boundary

This document describes only the bounded cleanup that runs before an Internet
selected rerun. It is enabled only by `DomainOneBatchRequest.RerunCleanup`.
Normal Run retains its ordinary queue/retry behavior and rejects a retained
non-restartable execution before it can invoke the rerun-only path.

The cleanup never writes business tables directly. It discovers a durable
source through the Ride state registry, then uses an existing legal Ride
transition, its normal transaction, event/fact tail, and queue completion.
After every Ride node drains, the aggregate verifier is read-only.

## Exhaustive durable-source map

`rideRehydrateStates` is the executable source-of-truth list. It contains all
13 non-terminal Ride order states; `TestRideRerunCleanupStateCoverage` asserts
that it is complete and every state has exactly one cleanup treatment.

| Durable source state | Pre-run treatment | Legal next step | Resource result required before next Run |
| --- | --- | --- | --- |
| `ride_scheduled` | cancellation | `R-CANCEL-S` | hold closed; no offer/capacity allocation |
| `ride_requested` | cancellation | `R-CANCEL-S` | hold closed; no offer/capacity allocation |
| `driver_offer_pending` | cancellation | `R-CANCEL-S` | offered capacity released; hold closed |
| `ride_reassign_pending` | cancellation | `R-CANCEL-S` | hold closed |
| `ride_assigned` | cancellation | `R-CANCEL-S` | assignment/trip cancelled; committed capacity released; hold closed |
| `driver_enroute_pickup` | cancellation | `R-CANCEL-S` | assignment/trip cancelled; committed capacity released; hold closed |
| `driver_arrived_pickup` | cancellation | `R-CANCEL-S` | assignment/trip cancelled; committed capacity released; hold closed |
| `rider_onboard` | natural continuation | `R-START` | occupied capacity is retained until the normal downstream edge releases it |
| `trip_in_progress` | natural continuation | `R-DROPOFF` | dropoff releases occupied capacity |
| `dropoff_completed` | exact continuation replay | committed `R-PAYABLE` | no capacity remains allocated |
| `ride_payable` | natural finance decision | existing payment outcome | capture settles, or failure closes the open hold |
| `ride_payment_failed_retryable` | natural finance decision | existing payment outcome | capture settles, or failure closes the open hold |
| `ride_completed` | natural continuation | `R-SETTLE` | settlement batch and payment reach their normal terminal state |

The first seven states are exactly the registry states that accept
`R-CANCEL-S`. After pickup, cleanup may not invent cancellation or rewrite a
business row: it resumes the sealed normal state-machine route. A persisted
contradiction (missing event/fact/continuation, unknown class, or an
unauthenticated stale winner) is fail-closed with evidence preserved.

## Registry and entry coverage

`ride.CanonicalRegistry()` continues to define all 44 legal business tuples:
three roots and 41 queued transitions. Roots have no predecessor queue, so
ready/point/expired recovery is N/A for them. The 13-state table above is the
rerun closure mapping; it deliberately does not duplicate each retained error
class into separate state-machine chains.

| Dispatcher ingress | Shared cleanup behavior | Guard |
| --- | --- | --- |
| ready | authority-scoped candidate, exact claim/start fence, then the source treatment above | ordinary Run cannot use rerun-fatal SQL |
| point | same queue identity supplied by committed progression | queue ID and authority both rechecked |
| expired | expires only the lease, preserves the exact fatal class, then uses the same cleanup treatment | normal expiry cannot reopen a fatal execution as retryable work |

These are two different layers. A durable source is discovered by a full
source scan or a progression point hint; it is not its own queue claim. The
expired path applies only to an already published continuation. The
`expired-sweep` cases in `TestRideDurableRegistryRerunIngress` prove that the
authority-scoped locator runs before source discovery; they are deliberately
not labelled as an expired continuation test. For the seven cancellable
sources, cleanup proves that this ordinary continuation has not executed and
decides `R-CANCEL-S` first. For the six post-pickup sources, an exact sealed
continuation may be claimed through ready, point, or expired recovery after
its zero-effect proof succeeds.

`dropoff_completed` is the one source whose current event is not itself
point-addressable. Its actual point identity is the `R-PAYABLE` continuation
published by the committed `R-DROPOFF` transaction; the registry point case
therefore invokes that continuation rather than synthesising a point hint for
the source event.

## Outcome and caller map

The following table is executable mapping, not a substitution for the tests:
the success column identifies the production `DomainEngine` entry which
creates each residual through normal transactions and drains it to a terminal
aggregate; the other columns identify the production caller that owns the
distinct replay/stale/failure mechanics. `ErrEvidence` is the representative
retained class for the natural-route caller, while
`TestRideDurableRetainedClassesUseOnlyRerunContinuationCAS` enumerates all 14
non-restartable classes at the exact execution fence.

| Source state | Disposition | Success entry | Commit-loss/replay | Stale winner | Failure / exact class |
| --- | --- | --- | --- | --- | --- |
| `ride_scheduled` | `R-CANCEL-S` | `TestRideDurableRegistryRerunIngress/ride_scheduled/{scan,point}` | `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources/ride_scheduled` | `TestTerminalizeRideRerunFatalClaimUsesOnlyProvenOutcomes` | `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun/ride_scheduled` |
| `ride_requested` | `R-CANCEL-S` | `TestRideDurableRegistryRerunIngress/ride_requested/{scan,point}` | `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources/ride_requested` | `TestTerminalizeRideRerunFatalClaimUsesOnlyProvenOutcomes` | `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun/ride_requested` |
| `driver_offer_pending` | `R-CANCEL-S` | `TestRideDurableRegistryRerunIngress/driver_offer_pending/{scan,point}` | `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources/driver_offer_pending` | `TestTerminalizeRideRerunFatalClaimUsesOnlyProvenOutcomes` | `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun/driver_offer_pending` |
| `ride_reassign_pending` | `R-CANCEL-S` | `TestRideDurableRegistryRerunIngress/ride_reassign_pending/{scan,point}` | `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources/ride_reassign_pending` | `TestTerminalizeRideRerunFatalClaimUsesOnlyProvenOutcomes` | `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun/ride_reassign_pending` |
| `ride_assigned` | `R-CANCEL-S` | `TestRideDurableRegistryRerunIngress/ride_assigned/{scan,point}` | `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources/ride_assigned` | `TestTerminalizeRideRerunFatalClaimUsesOnlyProvenOutcomes` | `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun/ride_assigned` |
| `driver_enroute_pickup` | `R-CANCEL-S` | `TestRideDurableRegistryRerunIngress/driver_enroute_pickup/{scan,point}` | `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources/driver_enroute_pickup` | `TestTerminalizeRideRerunFatalClaimUsesOnlyProvenOutcomes` | `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun/driver_enroute_pickup` |
| `driver_arrived_pickup` | `R-CANCEL-S` | `TestRideDurableRegistryRerunIngress/driver_arrived_pickup/{scan,point}` | `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources/driver_arrived_pickup` | `TestTerminalizeRideRerunFatalClaimUsesOnlyProvenOutcomes` | `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun/driver_arrived_pickup` |
| `rider_onboard` | `R-START` | `TestRideDurableRegistryRerunIngress/rider_onboard/{scan,point}` | `TestRideRerunFatalNaturalResumeCommitResponseLostReplaysRetainedClass` | `TestRideRerunFatalStaleCommitResponseLostReauthenticatesWinner` | `TestRideRerunFatalNaturalResumeRejectsIncompleteProofAndNormalRun` |
| `trip_in_progress` | `R-DROPOFF` | `TestRideDurableRegistryRerunIngress/trip_in_progress/{scan,point}` | `TestRideRerunFatalNaturalResumeCommitResponseLostReplaysRetainedClass` | `TestRideRerunFatalStaleCommitResponseLostReauthenticatesWinner` | `TestRideRerunFatalNaturalResumeRejectsIncompleteProofAndNormalRun` |
| `dropoff_completed` | committed `R-PAYABLE` | `TestRideDurableRegistryRerunIngress/dropoff_completed/{scan,point continuation}` | `TestRideRerunFatalNaturalResumeCommitResponseLostReplaysRetainedClass` | `TestRideRerunFatalStaleCommitResponseLostReauthenticatesWinner` | `TestRideRerunFatalNaturalResumeRejectsIncompleteProofAndNormalRun` |
| `ride_payable` | payment outcome | `TestRideDurableRegistryRerunIngress/ride_payable/{scan,point}` | `TestRideRerunFatalNaturalResumeCommitResponseLostReplaysRetainedClass` | `TestRideRerunFatalStaleCommitResponseLostReauthenticatesWinner` | `TestRideRerunFatalNaturalResumeRejectsIncompleteProofAndNormalRun` |
| `ride_payment_failed_retryable` | payment outcome | `TestRideDurableRegistryRerunIngress/ride_payment_failed_retryable/{scan,point}` | `TestRideRerunFatalNaturalResumeCommitResponseLostReplaysRetainedClass` | `TestRideRerunFatalStaleCommitResponseLostReauthenticatesWinner` | `TestRideRerunFatalNaturalResumeRejectsIncompleteProofAndNormalRun` |
| `ride_completed` | `R-SETTLE` | `TestRideDurableRegistryRerunIngress/ride_completed/{scan,point}` | `TestRideRerunFatalNaturalResumeCommitResponseLostReplaysRetainedClass` | `TestRideRerunFatalStaleCommitResponseLostReauthenticatesWinner` | `TestRideRerunFatalNaturalResumeRejectsIncompleteProofAndNormalRun` |

The applicable expired-continuation caller is separately proven by
`TestRideDurableDomainEngineExpiredIngressCommitsPayable` and
`TestRideRerunDurableExpiredFatalCommitResponseLostReplaysExactClass`; both
operate on a committed queue/execution lease, preserve its original class, and
do not use a source scan as a substitute. The dropoff point implementation is
`runRideDurableDropoffPayableContinuationPointIngress`, which starts from the
committed `R-DROPOFF` continuation and enters `DomainEngine` through the point
candidate query.

The post-pickup representative regression runs one actual retained fatal
`R-START` case through each ingress. It starts with committed
admission -> offer -> accept -> trip state, resumes through the existing
natural chain, and passes terminal aggregate verification. The common registry
tests cover all source states and the normal-run exclusion without expanding
identical error-class mechanics.

## Resource, authority, and time proofs

`VerifyRideRerunStateReady` is aggregate and read-only. It requires the
prepared driver/availability/capacity baseline, authority-scoped capacity delta
net zero, and zero open payment holds. It does not reset historical ledger,
identity, promotion/redemption, cursor, or payment history.

`ReadRidePersistentLogicalWatermark` reads the maximum persisted Ride timeline
coordinate for the selected user authority. It includes the persisted business,
queue, execution, progression, capacity, payment, ledger, offer, assignment,
trip, and settlement coordinates. It does not add a guessed duration; the
selected-run coordinator establishes the next safe epoch from that watermark.
A foreign authority's future row is excluded by the query scope.

## Focused production-path verification

The following tests are intentionally compact and cover the reusable mechanism
rather than a Cartesian state × error-class expansion:

- `TestRideRerunCleanupStateCoverage`: all 13 residual states map to the
  registered cancellation or natural route.
- `TestRideAllNonterminalSourcesRejectRetainedFatalClassInNormalRun`: every
  residual state rejects rerun-only behavior during normal measurement.
- `TestRideDurableNormalOfferFeedsRerunCancellationTerminalVerify` and
  `TestRideDurable{Scheduled,Immediate}RootRerunCleanupCancelsFromAdmission`:
  committed admission reaches legal cancellation, releases resources, and
  passes the read-only ready proof without allocating a new offer.
- `TestRideDurableActualRootOfferThenAcceptCommitsProductionRows`:
  one committed post-pickup fatal source enters through ready, point, and
  expired `DomainEngine` cleanup, retains its diagnosis, reaches terminal
  state, and verifies resource baselines and foreign-authority isolation.
- `TestRideRerunFatalNaturalResumeProofAndExecutionAcrossPostPickupStates`:
  each post-pickup registry edge uses the same read-only zero-effect proof and
  exact sealed operation identity.
- `TestRideRerunReadyProofScopesCapacityAndPaymentToCurrentAuthority` and
  `TestReadRidePersistentLogicalWatermarkScopesRawDurableRowsByAuthority`:
  aggregate resource proof and logical time are authority-scoped reads.
- `TestRideDurableRegistryRerunIngress` and
  `TestRideDurableRerunSystemCancellationCommitResponseLostReplaysAllCancellableSources`:
  all 13 durable sources use actual normal mutation prefixes; the cancellation
  family additionally proves a post-publication response-loss replays one
  `R-CANCEL-S` fact, then reaches a terminal ready proof.

No E2E or QPS conclusion follows from these local tests. They establish that a
reused Internet dataset has a legal, verified Ride starting state before a
benchmark run begins.

## D1 historical success matrix (44/44 registry tuples)

`rerun_historical_outcome_matrix_test.go` is the D1 success-only proof. Each
row below enters through the production normal admission/progression/queue
transaction, reads the committed successor from the same durable store, and
then uses `DomainEngine` pre-run cleanup until the production aggregate ready
verifier succeeds. It intentionally does not claim commit-loss, stale-winner,
or retained-fatal coverage; those are separate outcome evidence.

| Exact normal registry tuples | Count | Production success test |
| --- | ---: | --- |
| `R-SCH-CREATE/none`, `R-REQ/none` | 2 | `TestRideHistoricalOutcomeMatrixDurableRootAdmissionVerticals` |
| `R-SCH-ACT/ride_scheduled` | 1 | `TestRideHistoricalOutcomeMatrixDurableScheduledActivationVertical` |
| `R-OFFER-NEW/ride_requested`, `R-OFFER-NEW/ride_reassign_pending`, `R-OFFER-ACCEPT/driver_offer_pending`, `R-OFFER-REJECT/driver_offer_pending`, `R-OFFER-TIMEOUT-RETRY/driver_offer_pending`, `R-OFFER-TIMEOUT/driver_offer_pending`, `R-SUPPLY-REJECT/ride_reassign_pending` | 7 | `TestRideHistoricalOutcomeMatrixDurableOffer{New,Accept,Branch,Timeout}*`, `TestRideHistoricalOutcomeMatrixDurableSupplyRejectVertical` |
| `R-DEADLINE/{ride_scheduled,ride_requested,ride_reassign_pending}` | 3 | `TestRideHistoricalOutcomeMatrixDurableDeadlineTerminalVerticals` |
| `R-ENROUTE/ride_assigned`, `R-ARRIVE/driver_enroute_pickup`, `R-ONBOARD/driver_arrived_pickup`, `R-START/rider_onboard`, `R-DROPOFF/trip_in_progress` | 5 | `TestRideHistoricalOutcomeMatrixDurableTripVerticals` |
| `R-PAYABLE/dropoff_completed` | 1 | `TestRideHistoricalOutcomeMatrixDurablePayableVertical` |
| `R-PAY-OK/{ride_payable,ride_payment_failed_retryable}`, `R-PAY-RETRY/{ride_payable,ride_payment_failed_retryable}`, `R-PAY-FAIL/{ride_payable,ride_payment_failed_retryable}`, `R-SETTLE/ride_completed` | 7 | `TestRideHistoricalOutcomeMatrixDurableFinanceVerticals` |
| `R-CANCEL-C/{ride_scheduled,ride_requested,driver_offer_pending,ride_reassign_pending,ride_assigned,driver_enroute_pickup,driver_arrived_pickup}`, `R-CANCEL-P/{ride_assigned,driver_enroute_pickup,driver_arrived_pickup}`, `R-CANCEL-S/{ride_scheduled,ride_requested,driver_offer_pending,ride_reassign_pending,ride_assigned,driver_enroute_pickup,driver_arrived_pickup}`, `R-NOSHOW/driver_arrived_pickup` | 18 | `TestRideHistoricalOutcomeMatrixDurableFeedbackTerminalVerticals` |

Every D1 test asserts the exact resulting order state and affected capacity or
payment resource before terminal verification. In particular the scheduled
activation test continues through its normal `R-OFFER-NEW` successor before
beginning rerun cleanup; it does not claim a nonterminal `ride_requested`
state is ready.

## D2 success ingress matrix

The D2 tests reuse each D1 normal committed source by cloning its durable raw
store and constructing a fresh `DomainEngine` for each applicable entrance.
The clone is only the transaction-store adapter; the cleanup itself still
enters production `DrainOneBatch` and ends at `VerifyRideRerunStateReady`.

| Success source kind | scan | point | expired-sweep | N/A reason |
| --- | ---: | ---: | ---: | --- |
| 18 active registry successors | 18/18 | 16/16 | 18/18 | Point is N/A only for `R-SCH-ACT` (root-owned direct activation has no progression point identity) and `R-DROPOFF` (its legal point identity is the committed `R-PAYABLE` continuation). |
| 26 terminal registry successors | N/A | N/A | N/A | The normal transaction already reaches a terminal aggregate; no residual source may be re-entered for cleanup. |

`TestRideScheduledActivationRerunCancellationStalesReadyOfferContinuation`
is the boundary regression for the activation exception: it proves a normal
`R-SCH-ACT` leaves a real ready `R-OFFER-NEW`, then each applicable cleanup
entrance selects `R-CANCEL-S` and stale-closes that exact offer queue and
execution before the aggregate proof. Normal Run is not invoked after the
clone is made.