# Historical Empty Window 5m Default + Local A/B Replay Guide

Historical note:
- This document describes an earlier implementation where empty-window bypass was decided in `thanosquery` by `max_empty_window_age`.
- Current code uses taskqueue-owned retry bypass based on `firstFailedAt` and no longer uses `max_empty_window_age`.
- For current behavior, read:
  - `docs/reference/2026-05-25-pipeline-metric-failure-policy.md`
  - `docs/reference/2026-05-25-pipeline-task-lifecycle-and-thread-model.md`
- The rest of this document is kept only as historical rollout and replay context.

Scope:
- Use time-based bypass only (no retry-budget bypass).
- Code default: if `max_empty_window_age` is NOT configured, use 5 minutes.
- If configured, use configured value.

## 1) Current code behavior

In `thanosquery`:
- Default max empty window age is 5 minutes.
- Empty window handling:
  - if `now - task.EndTimestamp >= max_empty_window_age` => bypass this window and continue.
  - else => requeue and wait.

Equivalent policy:
- configured => config value
- not configured => 5m

## 2) Quick local unit verification

Run (from repo root):

```bash
cd pipeline
GOROOT=/opt/homebrew/Cellar/go/1.25.0/libexec PATH=/opt/homebrew/Cellar/go/1.25.0/libexec/bin:$PATH \
  go test ./internal/otel_collector/processor/thanos \
  -run 'TestDoQuery_NoDatapoint_(SwitchesToRequeueStrategy|BypassOnWindowAge|RequeueWhenWindowAgeNotReached)|TestNewQuerier_(DefaultMaxEmptyWindowAgeIsFiveMinutes|UsesConfiguredMaxEmptyWindowAge)' -v
```

Expected:
- all tests PASS

## 3) Local A/B replay plan (requeue-like vs regenerate-like)

Goal:
- Reproduce your previous comparison setup and verify new behavior still matches expected requeue semantics under empty windows.

A (baseline-ish / regenerate-like reference):
- Use existing replay flow as historical reference run (artifact/log-based), or checkout previous commit behavior if needed.

B (current behavior / 5m age bypass):
- Run current code with mock phases that keep empty longer than 5m to trigger bypass and forward progress.

Recommended replay phases for B:
- `REPLAY_PHASES=empty:360,error:60,data:120`
- This guarantees empty phase > 5m.

Run B:

```bash
cd pipeline/tools/local-e2e/scripts
REPLAY_PHASES='empty:360,error:60,data:120' \
REPLAY_WAIT_SECS=600 \
RUN_ID='chaos_empty5m_current' \
DD_NAMESPACE='chaos.test' \
bash run-dd-chaos-replay.sh
```

Artifacts:
- `pipeline/tools/local-e2e/generated/results/<round>/chaos-dd/<RUN_ID>/`
  - `pipeline.log`
  - `summary.txt`
  - `mock-o11y.log`

## 4) What to check in logs

In `pipeline.log`:
- requeue path messages should appear during early empty stage:
  - `no datapoint exists, should requeue and wait.`
- once empty age crosses 5m, should see bypass message:
  - `no datapoint exists, bypass empty window and continue`
- then observe ack/progress logs in later phases.

Practical grep:

```bash
grep -n 'requeue and wait\|bypass empty window and continue\|timestamp acked\|exporter send succeed, should ack' pipeline.log
```

## 5) Notes

- You do NOT need to change runtime config for existing clusters to get 5m policy now (default is in code).
- If later you can edit config, setting `max_empty_window_age` will override the default.
- No code commit is done in this task.