# Workspace Lark Timeline Ingestion Design

Status: Initial implementation behind disabled feature flags; retention and
production rollout controls remain follow-up work.

## 1. Background

Ticket Manager currently builds a curated workspace timeline from Jira issues,
completed workspace tasks, and DevOps operations or configuration changes. Raw
facts are persisted in `workspace_timeline_evidence`, incremental collection
checkpoints are persisted in `workspace_timeline_sync_states`, and AI or
deterministic rules turn evidence into `workspace_timeline_entries` and a
workspace timeline summary.

Lark group chats are an important source of incident facts, including customer
impact, investigation findings, mitigation actions, decisions, and recovery
confirmation. They are also materially different from the existing sources:

- messages arrive continuously and may be duplicated or out of order;
- a busy chat can produce many more records than Jira or DevOps;
- messages are frequently replies or thread fragments whose meaning depends on
  nearby context;
- most messages are not timeline-worthy;
- messages may be edited or deleted after the initial event;
- rich text, mentions, cards, files, images, and bot-generated messages require
  type-specific handling;
- chat content has stronger privacy and retention requirements than synthesized
  timeline entries.

This design extends the existing evidence model without making synchronous
workspace timeline sync responsible for processing the full chat stream.

## 2. Goals

- Ingest messages only from Lark chats bound to an eligible workspace.
- Preserve enough source information to trace a curated timeline entry back to
  its Lark messages.
- Keep ingestion idempotent under duplicate and out-of-order delivery.
- Recover missed new-message events through bounded reconciliation.
- Keep event acknowledgement fast and keep AI work out of the callback path.
- Process evidence in bounded, oldest-first batches without starving older
  messages.
- Group related messages by thread and time proximity before synthesis.
- Avoid regenerating the summary for every chat message.
- Preserve human-created and human-edited timeline content.
- Bound API usage, database growth, model tokens, retries, and retention.

## 3. Non-goals

- Mirroring the complete history of every Lark chat.
- Providing a general-purpose Lark message search product.
- Downloading or storing attachment binaries as timeline evidence.
- Replacing Lark as the source of truth for chat history.
- Automatically changing confirmed or human-created timeline entries.
- Inferring incident facts from reactions, read receipts, or chat membership in
  the first version.
- Guaranteeing that edits or deletions in an already completed time window
  trigger timeline regeneration.
- Fine-grained retry or audit state for every message.
- Automatically migrating prior output when the prompt or output schema changes.

## 4. Decision Summary

1. Keep `workspace_timeline_evidence` as the workspace-scoped fact and audit
   layer.
2. Reuse `workspace_timeline_sync_states` for two coarse checkpoints: one Lark
   collection checkpoint per workspace/chat and one synthesis state per
   non-empty 10-minute window. Never create one sync-state row per message.
3. Use `im.message.receive_v1` as the low-latency signal for new messages.
4. Add periodic, paginated Lark history reconciliation for missed events and
   restart recovery. Edits and deletions may update evidence, but completed
   synthesis windows are not automatically reopened.
5. Do not add `workspace_timeline_evidence_processing` or
   `workspace_timeline_dirty_scopes`. An in-memory notification may wake the
   worker early; the periodic watcher is the reliable fallback.
6. Discover the oldest evidence not covered by a window state, derive its fixed
   UTC window, and create a state row only after finding evidence in that
   window. Empty windows create no rows.
7. Synthesize each window asynchronously. If a window exceeds the evidence or
   token limit, process it in internal chunks while advancing one window cursor.
8. Update the workspace summary at most once after a completed window and only
   when visible timeline entries materially changed. Never overwrite a
   human-edited summary.
9. Backfill only the eligible workspace lifetime, capped at 72 hours. Do not
   read the full chat history.
10. Store normalized text and whitelisted metadata. Keep binaries and full raw
    payloads outside the timeline database.
11. Accept coarse recovery semantics: a failed window is retried as a whole,
    prompt upgrades require an explicit full rerun, and edits in completed
    windows are not guaranteed to regenerate output.

## 5. Proposed Architecture

```text
                         +-----------------------------+
Lark message event ----> | Lark timeline event handler |
                         +--------------+--------------+
                                        |
                                        | validate binding, normalize,
                                        | idempotent upsert
                                        v
                         +-----------------------------+
                         | workspace_timeline_evidence |
                         +--------------+--------------+
                                        |
                                        | best-effort in-memory wakeup
                                        v
                         +-----------------------------+
                         | periodic synthesis watcher  |
                         +--------------+--------------+
                                        |
                                        | oldest non-empty UTC window;
                                        | one coarse sync-state row
                                        v
                         +-----------------------------+
                         | timeline synthesis worker   |
                         +--------------+--------------+
                                        |
                            +-----------+-----------+
                            |                       |
                            v                       v
              workspace_timeline_entries   timeline summary

Periodic reconciliation ----> chat collection sync state
        |
        | paginated Lark history fetch with overlap
        v
evidence upsert path above
```

The event path provides low latency. Reconciliation provides correctness. Both
paths call the same normalization and evidence-upsert code.

## 6. Eligibility and Scope

A Lark message is eligible only when all of the following are true:

- the message belongs to a group or topic-group chat;
- the chat is represented by a Lark chat collaboration resource;
- that resource is bound to the workspace;
- the workspace is `firing`;
- the workspace was created within the last 72 hours;
- the message occurred no earlier than the workspace creation time;
- Lark timeline ingestion is enabled for the environment and workspace type.

Manual `Refresh from sources` may reconcile an explicitly selected workspace
even when it is no longer eligible for automatic scanning, but it must still
respect the workspace creation time and retention boundary.

### 6.1 Chat-to-workspace mapping

The existing collaboration resource binding is workspace-scoped and does not
make a Lark chat globally unique. If one chat is intentionally bound to several
eligible workspaces, the same source message is stored once per workspace. This
duplication is intentional because permissions, retention, curation, and AI
processing are workspace-specific.

The handler must resolve bindings from persisted state or use a cache with a
database fallback. It must not depend only on startup-time advisor
registration, otherwise a newly bound chat can lose messages until restart.

## 7. Lark Event Path

### 7.1 Event type

Use the existing `im.message.receive_v1` callback and shared
`LarkEventDispatcher`. Runtime code receives the Lark Go SDK event shape, where
message content is message-type-specific JSON. It must not assume the
human-readable content transformation performed by `lark-cli event consume`.

The callback provides the sender's `open_id`, but not a guaranteed display
name. Persist `open_id` immediately. Resolve display names through a bounded
cache or batch lookup; do not call the contact API synchronously for every
message.

### 7.2 Callback responsibilities

The event handler should do only bounded work:

1. Validate the event and required identifiers.
2. Resolve eligible workspace bindings for `chat_id`.
3. Reject unsupported, self-generated, or out-of-scope messages.
4. Normalize supported content.
5. Upsert evidence idempotently.
6. Best-effort notify an in-memory channel so the periodic synthesis watcher may
   run early.

It must not invoke OpenCode, regenerate the timeline, update the summary, fetch
attachments, or perform unbounded contact lookups.

The current dispatcher acknowledges the HTTP callback before asynchronously
executing handlers. Therefore callback delivery alone is not a durable queue.
The periodic reconciliation and synthesis watchers are required even after
event ingestion is enabled. Losing an in-memory notification only adds latency;
the next default two-minute scan rediscovers the evidence. A future shared
dispatcher enhancement may add synchronous durable inbox handlers, but this
proposal does not require that change for the first release.

### 7.3 Message type policy

| Message type | Evidence behavior | AI behavior |
|---|---|---|
| `text` | Normalize and store text | Eligible |
| `post` | Normalize title and text blocks | Eligible |
| `image` | Store message and image key metadata only | Ineligible unless nearby text supplies context |
| `file` / `media` / `audio` | Store filename/type/key metadata only | Ineligible in the first version |
| `interactive` | Store minimal card metadata when needed | Ignore by default |
| `system` | Store only when needed for reconciliation | Ignore |
| unsupported | Count and skip | Ignore |

Interactive cards and messages sent by the Ticket Manager bot must be excluded
from synthesis by default. This prevents workspace overview cards, task
notifications, and AI replies from feeding back into the timeline.

### 7.4 Normalization

Normalization should live in a reusable Lark package, not in an HTTP handler.
It should return a stable structure such as:

```go
type NormalizedLarkMessage struct {
    MessageID   string
    ChatID      string
    ThreadID    string
    RootID      string
    ParentID    string
    SenderOpenID string
    MessageType string
    Text         string
    CreatedAt    time.Time
    UpdatedAt    time.Time
    Deleted      bool
    Metadata     map[string]any
}
```

Only fields required for traceability and synthesis should be copied into
`metadata_json`. Full callback envelopes, access tokens, attachment bytes, and
unrelated user profile data must not be stored.

## 8. Reconciliation Path

Events cover new-message latency; reconciliation covers missed delivery,
service downtime, and backfill. A bounded overlap may also refresh edited or
deleted evidence, but this design does not guarantee regeneration of a window
that has already completed.

### 8.1 Initial backfill

When a Lark chat is first bound or a workspace first becomes eligible:

```text
start = max(workspace.created_at, now - 72h)
end   = now
```

Fetch messages in ascending creation-time order and follow every page. The
existing `IM.ListMessages` implementation currently requests only 20 messages
and does not paginate; it must be extended before it can be used for timeline
reconciliation.

### 8.2 Incremental reconciliation

Persist one source checkpoint per workspace/chat:

```text
source      = lark
source_type = lark_messages
source_key  = chat/<chat_id>
```

Recommended checkpoint semantics:

- `last_seen_updated_at`: highest safely committed message creation/update time;
- `last_seen_cursor`: encoded durable high-water mark containing
  `create_time_ms` and `message_id`, or a Lark page token only if the API
  contract guarantees that it remains valid across runs;
- `last_seen_content_hash`: unused for the chat-level checkpoint;
- `metadata_json`: reconciliation version, last successful full-window scan,
  and bounded diagnostic information.

Normal reconciliation starts slightly before the committed high-water mark to
handle late and out-of-order delivery. An optional less-frequent scan of a
larger recent window can refresh edits and deletions in evidence. Both windows
must be configurable and capped by workspace creation time and the 72-hour
ingestion boundary.

The checkpoint advances only after all pages in the covered range have been
normalized and committed. A failed page must not advance the checkpoint past
uncommitted data.

### 8.3 Idempotency

Lark evidence uses the existing unique key:

```text
workspace_id + source + source_type + source_key
```

Recommended values:

```text
source      = lark
source_type = lark_message
source_key  = chat/<chat_id>/message/<message_id>
```

Duplicate events and reconciliation results update the same row. A message edit
changes `source_updated_at`, normalized content, metadata, and `content_hash`.
A deleted message keeps its identity but replaces retained content according to
the deletion policy and marks `deleted=true` in metadata.

## 9. Evidence Schema Changes

Extend the shared entity constants with:

```go
TimelineEvidenceSourceLark             = "lark"
TimelineEvidenceSourceTypeLarkMessage  = "lark_message"
```

No structural change to `workspace_timeline_evidence` is required for the first
version. Populate fields as follows:

| Evidence field | Lark value |
|---|---|
| `workspace_id` | Bound workspace ID |
| `source` | `lark` |
| `source_type` | `lark_message` |
| `source_key` | `chat/<chat_id>/message/<message_id>` |
| `occurred_at` | Message creation time |
| `source_updated_at` | Message update time, otherwise creation time |
| `observed_at` | Ticket Manager ingestion time |
| `actor_id` | Sender `open_id` |
| `actor_display_name` | Cached value when available |
| `content` | Normalized bounded text |
| `content_hash` | Hash of normalized semantic content and deletion state |
| `metadata_json` | Whitelisted chat/thread/reply/type/attachment references |

Add or verify a composite index supporting the hot query:

```text
(workspace_id, source, source_type, occurred_at)
```

The primary key supplies the stable `id` tie-breaker for keyset pagination.

Timeline evidence list APIs must use keyset pagination before Lark evidence is
exposed in the UI. They must not load every evidence row for a workspace.

## 10. Sync State Model

No new processing or dirty-scope table is required. The existing
`workspace_timeline_sync_states` table stores coarse-grained state in two
separate namespaces for Lark. It must not create one AI processing row per Lark
message.

The current generic AI path uses `source=system`,
`source_type=ai_timeline_synthesis`, and `source_key=evidence/<evidence_id>`.
That behavior may remain unchanged for Jira, task, and DevOps evidence during
the first rollout, but Lark evidence must be excluded from that path and routed
through the window namespace below. This avoids an unrelated migration of
existing sources while preventing per-message state growth for chat.

### 10.1 Chat collection checkpoint

Collection continues to use one checkpoint per workspace and chat:

```text
source      = lark
source_type = lark_messages
source_key  = chat/<chat_id>
```

This state answers only "how far has chat history been collected?" Its cursor
and update time follow the reconciliation semantics in section 8.

### 10.2 Non-empty synthesis window

Timeline synthesis uses fixed 10-minute UTC windows:

```text
window_start = floor(message.occurred_at / 10 minutes) * 10 minutes
window_end   = window_start + 10 minutes

source      = system
source_type = lark_timeline_window
source_key  = chat/<chat_id>/window/<window_start_epoch>
```

If an explicit full regeneration is required after a prompt or schema upgrade,
the rerun may use a versioned namespace:

```text
source_key = chat/<chat_id>/processor/<version>/window/<window_start_epoch>
```

The existing columns are used as follows:

- `last_seen_cursor`: the last committed `occurred_at + evidence_id` inside the
  window; this advances after each internal chunk;
- `last_seen_updated_at`: the last successful processing time;
- `last_seen_content_hash`: an aggregate hash of the ordered
  `(evidence_id, content_hash)` pairs that were included;
- `metadata_json`: window start/end, `processing | completed | ignored | failed`
  status, evidence count, chunk count, prompt version, attempt count, current
  chunk start, and bounded error diagnostics.

The aggregate hash provides idempotency and audit context. It does not imply
that a completed window is automatically reopened when an old message changes.

Most importantly, a window state row is created only after the watcher finds at
least one eligible evidence row in that window. A chat with no messages for 10
minutes creates no evidence row, no synthesis state row, and no AI request.

At the theoretical maximum, a continuously active chat produces 144 window
rows per day, or 432 within the 72-hour automatic-ingestion period. Real usage
is lower because empty windows are never materialized.

## 11. Window Discovery and Scheduling

The event handler may send a best-effort in-memory wakeup keyed by workspace and
chat. Wakeups may be deduplicated or dropped because they are only a latency
optimization. The periodic synthesis watcher, running every two minutes by
default, is the durable discovery mechanism.

For every eligible workspace/chat, the watcher:

1. Reads evidence in ascending `(occurred_at, id)` order within
   `max(workspace.created_at, now - 72h)` and `now` using keyset pagination.
2. Maps candidate evidence to fixed 10-minute UTC window keys.
3. Bulk-loads the corresponding existing window states and skips terminal
   `completed`, `ignored`, and exhausted `failed` windows.
4. Selects the oldest non-empty uncovered or retryable window.
5. Waits until `window_end + close_grace` and, when reconciliation is enabled,
   until collection has successfully covered the window. This reduces the
   chance that late delivery is omitted.
6. Atomically creates or transitions the one window state to `processing`, then
   invokes synthesis.

Discovery starts from evidence, not from a clock-generated list of buckets.
Therefore ten minutes with no chat messages never inserts a row. The 72-hour
bound and keyset pagination keep discovery bounded without a per-message
processing index.

Only one synthesis run per workspace should be active in one process. Competing
instances must use the sync-state row's unique key and an atomic state
transition to avoid duplicate work. If a process dies in `processing`, the
watcher treats the state as stale after a configured timeout and retries that
window as a whole. This is intentionally coarser than per-evidence leases.

## 12. Timeline Synthesis

### 12.1 Candidate selection

The current implementation loads all workspace evidence and then keeps the
newest configured number. This is unsuitable for a continuous high-volume
stream because it is O(total history) per run and can starve old evidence.

After the watcher selects `[window_start, window_end)`, the worker queries only
eligible evidence in that chat and range, ordered by `(occurred_at, id)`. It
applies both a hard evidence-count limit and an estimated-token limit.

A window may contain more than one model batch. After every successfully
committed chunk, the worker advances the same window state's
`last_seen_cursor`. It continues until all evidence in the window is covered,
then records the final aggregate content hash and marks the window `completed`
or `ignored`. No per-evidence processing state is written.

Failed model calls or invalid output retry the current chunk with bounded
attempts. After the limit, the whole window is marked `failed` with bounded
diagnostics. Operators may explicitly replay that window, but there is no
fine-grained retry for selected individual messages.

### 12.2 Conversation grouping

The fixed window is the scheduling and checkpoint boundary. Within a chunk,
group evidence using the strongest available conversational boundary:

1. `thread_id`;
2. `root_id`;
3. top-level message plus replies;
4. otherwise message proximity within the selected window.

Preserve message order inside each group. The primary evidence must always be
inside `[window_start, window_end)`. The model may receive the previous five
minutes and messages up to `window_end` with the same thread/root as read-only
context, but only current-window evidence contributes to the window cursor and
aggregate hash.

### 12.3 Timeline-worthy output

The model should create or update drafts only for new facts such as:

- customer impact or affected scope;
- a new investigation finding;
- a decision or ownership change;
- a mitigation or operational action;
- a root-cause conclusion with supporting evidence;
- recovery or verification;
- an important next step or blocker.

Acknowledgements, mentions, repeated status, scheduling chatter, bot messages,
links without explanation, and social conversation should produce no timeline
entry while remaining traceable as evidence. A window that produces no
material entry is marked `ignored`.

### 12.4 Entry binding and deduplication

Generated entries remain `source=ai`, `status=draft` and bind every supporting
Lark evidence row through `workspace_timeline_entry_evidence`.

The existing draft fingerprint remains the first deduplication layer. Extend AI
metadata to record:

- processor and prompt version;
- evidence IDs and their input content hashes;
- chat/thread/root identifiers;
- synthesis batch ID.

Confirmed, hidden, and human-created entries keep their current protection
rules. New evidence may be attached to a confirmed entry as supporting evidence
when the fingerprint matches, but its text must not be overwritten.

## 13. Summary Update Policy

Do not regenerate the workspace summary for every message or every internal
chunk. A completed 10-minute window is the summary coalescing boundary.

After the window reaches a terminal state, attempt summary regeneration at most
once and only when:

- the window created or materially updated at least one visible draft;
- the workspace has no human-edited summary;
- the window state does not already record `summary_attempted=true`.

Use the current visible entries as the primary summary input. Newly processed
evidence may provide supporting context, but messages marked `ignored` must not
change the summary.

The existing rule that a `source=human` summary is never automatically
overwritten remains unchanged.

## 14. Resource Controls

All values should be configurable. Suggested initial controls are:

- automatic ingestion lookback: 72 hours;
- synthesis watcher interval: 2 minutes;
- fixed synthesis window: 10 minutes;
- window close grace: 2-5 minutes;
- context lookback: 5 minutes, plus matching thread/root context;
- maximum evidence count per internal chunk: 40;
- maximum prompt size: explicit estimated-token budget;
- maximum whole-window attempts: 3;
- stale processing timeout: configurable and longer than the model timeout;
- maximum concurrent Lark synthesis per Ticket Manager instance: 2;
- maximum concurrent job per workspace: 1;
- normal reconciliation: frequent small overlap;
- optional edit/delete evidence refresh: less-frequent larger overlap;
- bounded exponential retry with jitter;
- per-workspace and global window backlog limits with metrics and alerts.

Count limits alone are insufficient. Token limits must account for normalized
text, metadata, and nearby entry context.

## 15. Retention, Privacy, and Access

- Ingest only explicitly bound chats and eligible workspace time ranges.
- Use bot identity and the minimum required Lark scopes.
- Do not log message bodies or full callback payloads.
- Do not persist tokens, secrets, attachment bytes, or unrelated profiles.
- Store attachment identifiers and basic metadata only.
- Apply the existing Ticket Manager workspace authorization to evidence APIs.
- Hide Lark evidence from the UI until evidence endpoints support pagination and
  access checks are verified.
- Stop automatic ingestion when the workspace leaves `firing` or ages out.
- Define deletion or redaction behavior for recalled messages.
- Apply the organization's approved ticket/chat retention policy. Do not create
  a new indefinite chat archive by default.
- Add a cleanup or redaction job before enabling the feature in production.

## 16. Failure Handling

| Failure | Required behavior |
|---|---|
| Duplicate event | Idempotent evidence upsert; no duplicate window state |
| Event handler failure after callback ACK | Reconciliation recovers the message |
| In-memory wakeup lost | Default two-minute watcher discovers the evidence |
| Lark page fetch failure | Keep prior checkpoint; retry from the last committed range |
| Evidence write failure | Do not advance reconciliation checkpoint |
| AI call failure | Preserve evidence; retry the current chunk/window with backoff |
| Invalid AI output | Bounded whole-window retry, then mark the window failed |
| Worker crash | Stale `processing` timeout makes the whole window retryable |
| Message edit | Upsert the evidence hash; do not automatically reopen a completed window |
| Message deletion | Redact/mark evidence; completed-window regeneration is best effort only |
| Prompt/schema upgrade | Run an explicit bounded full regeneration with a versioned key namespace |
| Human summary/entry exists | Never overwrite human content |

## 17. Observability

Metrics should include:

- Lark events received, accepted, ignored, and failed by reason;
- evidence inserts, updates, duplicates, and redactions;
- reconciliation duration, pages, messages, lag, and failures;
- in-memory wakeups sent, dropped, and coalesced;
- non-empty windows by status, oldest uncovered evidence age, stale processing
  count, and whole-window retries;
- AI chunks, evidence per chunk, tokens, latency, retries, and validation errors;
- timeline drafts created, updated, ignored, and bound;
- summary regeneration count, skipped count, and lag;
- messages excluded as bot/self-generated;
- retention cleanup rows and failures.

Logs should carry workspace ID, chat ID hash or approved identifier, message ID,
batch ID, and processing version, but never raw message content.

## 18. Configuration

Suggested configuration shape:

```yaml
services:
  workspace:
    timeline:
      lark:
        enabled: false
        eventIngestionEnabled: false
        reconciliationEnabled: false
        ingestionLookback: 72h
        synthesisScanInterval: 2m
        synthesisWindow: 10m
        synthesisWindowCloseGrace: 3m
        synthesisContextLookback: 5m
        synthesisStaleTimeout: 15m
        synthesisWindowMaxAttempts: 3
        reconcileInterval: 5m
        reconcileOverlap: 10m
        evidenceRefreshInterval: 1h
        evidenceRefreshLookback: 24h
        evidenceBatchLimit: 40
        maxConcurrentSynthesis: 2
```

Production values must be tuned from observed group traffic and Lark API quota,
not copied blindly from the suggested defaults.

## 19. Rollout Plan

### Phase 0: Contracts and storage

- Add Lark evidence source/type constants.
- Extend the existing sync-state repository for chat checkpoints and non-empty
  synthesis window keys, including an atomic compare-and-set transition; add no
  new table.
- Exclude Lark evidence from the existing `evidence/<evidence_id>` AI
  sync-state path.
- Add the composite evidence query index.
- Add paginated evidence query APIs internally.
- Add feature flags, all disabled by default.

### Phase 1: Shadow event ingestion

- Register the Lark timeline message handler.
- Normalize `text` and `post` messages.
- Filter bot/self-generated messages.
- Write evidence and send a best-effort in-memory wakeup without calling AI.
- Enable for development and selected test workspaces.
- Verify duplicate, ordering, volume, privacy, and storage behavior.

### Phase 2: Reconciliation

- Extend the shared Lark IM client with paginated history listing.
- Implement bounded initial backfill and incremental checkpoints.
- Optionally refresh edits and deletions in a bounded recent window.
- Verify recovery after dropped events and restarts.

### Phase 3: Windowed timeline drafts

- Add the default two-minute watcher and oldest-first non-empty window discovery.
- Add atomic window-state transitions and stale-window recovery.
- Process large windows in evidence/token-bounded chunks.
- Group context by thread/root within the fixed time window.
- Generate and bind AI drafts.
- Keep summary generation disabled initially.
- Compare generated drafts against manual incident timelines.

### Phase 4: Summary and production rollout

- Enable at-most-once summary regeneration after a materially changed window.
- Confirm human-summary protection.
- Add retention cleanup/redaction.
- Roll out by workspace type and environment.
- Tune batching, concurrency, and reconciliation from metrics.

### Phase 5: Optional improvements

- Add attachment text extraction through an explicitly approved pipeline.
- Add message-edit-specific event handling if supported and reliable.
- Add atomic facts or suggestions when direct draft merging is insufficient.
- Move complete raw payloads to an approved object store if audit requirements
  exceed the normalized evidence model.

## 20. Test Plan

### Unit tests

- normalize every supported Lark message type;
- reject missing identifiers and unsupported chat types;
- exclude Ticket Manager bot messages and overview cards;
- map one chat event to every eligible bound workspace;
- exclude recovered, archived, or older-than-72-hour workspaces;
- idempotently upsert duplicate messages;
- derive stable 10-minute UTC window keys;
- create no window state for an empty 10-minute period;
- preserve ordering within a thread;
- select the oldest uncovered non-empty window before newer windows;
- advance one window cursor across multiple count/token-bounded chunks;
- respect atomic state transitions, stale timeouts, bounded whole-window retries,
  and versioned full-rerun keys;
- never overwrite human entries or summaries.

### Repository tests

- evidence uniqueness and edit upsert behavior;
- chat-level collection checkpoint updates;
- non-empty window-state uniqueness and atomic transitions;
- no sync-state insert when a candidate time range has no evidence;
- stale processing-window recovery and bounded attempt transitions;
- checkpoint advancement only after committed pages;
- keyset pagination with equal timestamps and different IDs.

### Integration tests

- more than one Lark API page of messages;
- duplicate and out-of-order event delivery;
- event loss recovered by reconciliation;
- restart in the middle of a paginated backfill;
- edit and deletion evidence refresh without reopening a completed window;
- a window containing more than 40 messages processed across multiple chunks;
- continuous traffic across multiple windows without starving the oldest window;
- lost in-memory wakeup recovered by the default two-minute watcher;
- the same chat bound to multiple workspaces;
- workspace aging past 72 hours while jobs are pending;
- manual refresh outside the automatic scan path;
- AI failure followed by a whole-window retry and successful binding.

### Load tests

- burst traffic into one chat;
- sustained traffic across many eligible workspaces;
- database query latency at expected retention volume;
- window backlog recovery after a prolonged AI outage;
- reconciliation API usage under configured overlap windows.

## 21. Acceptance Criteria

- A new supported message in an eligible bound chat is persisted idempotently.
- Duplicate events never create duplicate evidence or timeline entries.
- Missed events are recovered by reconciliation without a full-chat scan.
- A message edit changes the evidence hash, but a completed window is not
  required to regenerate automatically.
- A non-empty 10-minute window creates at most one state row for the active
  processor namespace; an empty window creates none.
- Window discovery is limited to the eligible 72-hour range and uses keyset
  pagination rather than loading all workspace evidence at once.
- A window larger than the model batch limit advances one durable window cursor
  across chunks.
- Continuous traffic cannot starve older uncovered windows.
- AI work never runs in the Lark callback request path.
- A burst of messages creates a bounded number of synthesis chunks and at most
  one summary attempt per completed window.
- Every generated Lark timeline entry is traceable to evidence IDs.
- Human entries and human-edited summaries are never overwritten.
- Automatic ingestion stops outside the workspace status and 72-hour scope.
- Evidence retention and deletion behavior is enabled before production rollout.

## 22. Decisions Requiring Confirmation

The following must be confirmed before production implementation:

- whether all bound Lark chats or only the primary chat participate;
- the approved Lark message and evidence retention period;
- whether recalled messages must be hard-deleted or redacted in place;
- whether file names and attachment keys may be retained;
- the exact Lark bot scopes and availability of message history APIs;
- environment-specific Lark API quota and expected peak message rate;
- whether `post` rich text is sufficient for the first release or text-only is
  required for a smaller MVP.
