# APAC monitor fan-out worker

Handler: `worker_function.lambda_handler`

## Responsibility

The worker consumes one tenant per SQS record, updates the APAC target database,
and sends a scoring request after the database transaction commits.

Three known APAC tenants with exceptionally large cluster inventories use a
forced Sales-owned fast path:

- `1372813089209260919`
- `1372813089209300444`
- `1372813089209301246`

For these tenants the Worker preserves any existing enrichment, upserts the
latest base tenant fields into `apac_sales_owned_tenants` with
`priority = 'P0'`, removes the Active row, and commits. This rule runs before
HubSpot and sales-owner matching. It does not call Domain Intelligence, DevOps
cluster APIs, Clinic, Billing, or Scoring.

For an active candidate it:

1. loads or classifies email-domain intelligence;
2. queries DevOps/Clinic for cluster counts and 30-day average QPS;
3. queries Billing for credit-card and active-credit signals;
4. matches the latest successful HubSpot target snapshot;
5. loads the message's exact Effective Owner snapshot row and writes
   `effective_sales_owner_email` and `effective_sales_owner_source`;
6. if unmatched, checks that Effective Owner against the configured APAC Sales
   set;
7. upserts `apac_active_tenants` or `apac_sales_owned_tenants` and deletes the
   same tenant ID from the other table in one transaction;
8. sends the tenant ID, bucket, and optional HubSpot P0 override to Scoring.

For a new signup it only performs domain intelligence, upserts the same
destination table with `source_type = 'new_signup'`, and sends a scoring
request. It does not call the cluster or Billing APIs. HubSpot matching always
precedes the sales-owner rule, so a tenant matching both stays Active and is
forced to P0 after its normal score is calculated.

## Failure behavior

- Enrichment failures are recorded in the selected tenant table's
  `error_message` and
  produce an `active_failed` or `new_signup_failed` scoring event.
- A scoring-queue send failure is not swallowed. The current SQS record is
  returned in `batchItemFailures` so Lambda retries it.
- A missing `effective_owner_sync_id` or missing row in that exact snapshot
  fails the SQS record. Worker never falls back to the legacy
  `sales_owner_email` field.
- Because the target-table writes are upserts, retrying after a scoring send
  failure is safe and does not create duplicate tenant rows.
- Partial cluster failures are retained in `error_message`; successfully
  resolved cluster counts are still written. QPS is written as `NULL` when
  any cluster QPS request fails, so unavailable or incomplete data is not
  mistaken for a real zero-QPS result.
- Domain Intelligence, cluster metrics, and Billing failures are isolated.
  A failure in one enrichment source preserves its previous value when
  available and does not prevent the other sources from being collected.
- When a cached Billing config is missing, inactive, or has no valid credit
  card, the Worker rediscovers all active Billing configs before concluding
  that the tenant has no valid card.
- A new-signup upsert preserves an existing tenant score and priority instead
  of replacing them with `NULL`.
- When `cluster_count >= 10`, the Worker keeps the DevOps cluster and service
  plan counts but skips all per-cluster Clinic QPS calls. It writes
  `tenant_avg_qps = NULL`; Scoring treats a non-individual tenant at this
  threshold as at least P1. At 50 or more clusters, Scoring makes every tenant
  P0 regardless of its domain-intelligence category.

## Configuration

Required for every record:

- `TARGET_DATABASE_URL` (preferred), or the `TARGET_DB_*` variables

Required for records that do not use the forced Sales-owned fast path:

- `SCORING_QUEUE_URL`

Required only for active-candidate enrichment:

- `DEVOPS_API_TOKEN`

Required only when an active candidate has fewer than 10 clusters:

- `CLINIC_API_KEY`

Optional:

- `OPERATOR_EMAIL`
- `OPENAI_API_KEY`
- `DEVOPS_BASE_URL`
- `CLINIC_BASE_URL`
- `OPENAI_WEB_SEARCH_MODEL`
- `DOMAIN_INTELLIGENCE_TTL_DAYS`
- `LLM_TIMEOUT_SECONDS`
- `BILLING_REQUEST_INTERVAL_SECONDS`
- `BILLING_RETRY_COUNT`
- `TARGET_TIDB_SSL_CA_PATH` or `TIDB_SSL_CA_PATH`
- `APAC_SALES_EMAILS` (comma/semicolon-separated exact addresses; empty disables
  Sales-owned routing)
- `HUBSPOT_TARGET_LIST_ID` (default `19544`)
- `HUBSPOT_SNAPSHOT_CACHE_TTL_SECONDS` (default `300`)

The role needs CloudWatch Logs, permission to consume the monitor queue, and
`sqs:SendMessage` for the scoring queue. Keep batch size 1 and reserved
concurrency 2 initially because the external Billing and Clinic APIs dominate
latency and have rate limits.

## Removed legacy code

The Worker package no longer contains a Dispatcher handler, DevOps tenant-list
queries, raw SQL overrides, or the old all-tenants-in-one-invocation handler.
It also no longer logs full tenant, payment, or credit objects.