# LATAM tenant signal pipeline

This project monitors LATAM TiDB Cloud tenants, enriches and scores them, and
sends daily and weekly summaries to Feishu. The five Python 3.12 Lambda
functions and their supporting AWS resources are managed as one SAM stack.

## Tenant scope

The regular active-tenant flow selects DevOps tenants that meet all of these
conditions:

- `status = 'active'`;
- `kind = 'external'`;
- `sales_force_region = 'LATAM'`;
- `plan NOT IN ('free_trial', 'new_user', 'closing')`.

The new-signup flow selects tenants created during the previous UTC natural
day with `country_region = 'LATAM'`. It intentionally does not filter plans,
because new signups commonly use `new_user` or `free_trial`.

## Data flow

```text
Daily EventBridge rule
        |
        v
   Dispatcher --> Monitor SQS --> Worker --> latam_active_tenants
                                      |               |
                                      v               |
                                 Scoring SQS          |
                                      |               |
                                      v               |
                                   Scoring <----------+
                                      |
                         +------------+------------+
                         v                         v
            score + daily score snapshot   HubSpot Contact priority

Daily Summary rule --------------------> LATAM DB --> Feishu
Weekly Metrics rule -------------------> LATAM DB --> Feishu
```

Dispatcher emits one message per tenant. Worker enriches and upserts that
tenant before requesting a score recalculation. Summary and Weekly Metrics are
read-only reporting functions and do not participate in the SQS pipeline.

## Components

| Function | Trigger | Responsibility | Runtime configuration |
| --- | --- | --- | --- |
| [Dispatcher](LATAM-Tenants-Monitor-Fanout-Dispatcher-Task/README.md) | Daily EventBridge or manual invocation | Select active tenants and previous-day signups, then publish one monitor message per tenant | 120 s, 256 MB |
| [Worker](LATAM-Tenants-Monitor-Fanout-Worker-Task/README.md) | Monitor SQS, batch size 1 | Collect domain, cluster, QPS, and billing signals; update the tenant row; publish a scoring message | 900 s, 512 MB, reserved concurrency 2 |
| [Scoring](LATAM-Tenants-Scoring-Task/README.md) | Scoring SQS, batch size 1, maximum concurrency 2 | Calculate score, priority, and key-lead status; write the daily score snapshot and existing HubSpot Contact priority | 180 s, 256 MB |
| [Weekly Metrics](LATAM-Weekly-Tenant-Metrics-Task/README.md) | Weekly EventBridge or manual invocation | Aggregate LATAM stock and previous-week signup metrics and send a Feishu card | 120 s, 256 MB |
| [Summary](LATAM-Tenants-Summary-Task/README.md) | Daily EventBridge or manual invocation | Send LATAM totals and up to 10 detail rows per section to Feishu | 120 s, 512 MB |

All five functions use active X-Ray tracing. Their configuration is supplied
through Lambda environment variables managed by CloudFormation. Sensitive
values are `NoEcho` stack parameters and must not be committed to source,
template defaults, or `samconfig.toml`.

Important processing behavior:

- Domain Intelligence assigns zero or more canonical ICP categories from
  `fintech`, `retail`, `e_commerce`, `ai`, and `igaming`. Scoring raises a
  recognized organization's Fit score to at least 25 when any ICP matches;
  ICP alone does not force P0 or P1.
- Domain Intelligence uses Qwen `qwen3.8-max` with built-in web search and
  non-thinking Responses mode. `API_KEY`, `WORKSPACE_ID`,
  `DASHSCOPE_ENDPOINT`, and `REGION` are injected directly into the Worker
  Lambda environment through `NoEcho`/deployment-time stack parameters; the
  function does not read them from AWS Secrets Manager.
- Worker always retrieves every DevOps cluster page so service-plan counts are
  accurate. At `cluster_count >= 10`, it skips per-cluster Clinic QPS requests
  and stores QPS as unknown.
- A Worker scoring-message failure uses the existing SQS retry behavior, so a
  retry runs the complete tenant operation again. Database writes are upserts.
- After its database commit, Scoring writes P0/P1/P2 to
  `latam_pql_priority_tier` on the existing HubSpot Contact matched by exact
  tenant email. It never creates a Contact; an empty token disables only this
  writeback.
- Summary shows accurate totals while limiting each P0, P1, and recent-import
  detail section to 10 rows.

## Database

The target database is `latam_regional_support`. `TargetDatabaseURL` must use
`latam_regional_support` as its URL database path. [`schema.sql`](schema.sql)
defines the isolated LATAM tables required by this pipeline:

| Table | Purpose |
| --- | --- |
| `latam_active_tenants` | Current unified tenant record, ICP categories, enrichment signals, score, priority, and tracking state |
| `latam_key_leads` | LATAM company list used for key-lead matching |
| `latam_tenant_events` | Import/event records used by the recent-import summary section |
| `latam_tenant_score_snapshots` | One score, priority, and ICP snapshot per tenant per UTC day |
| `domain_intelligence` | Cached email-domain, entity, and multi-select ICP classification |

There is no separate `latam_new_signup` table. New signups are stored in
`latam_active_tenants` with `source_type = 'new_signup'`. None of these five
Lambda functions currently writes `latam_tenant_events`; its data must come
from an external event producer.

## AWS infrastructure

[`template.yaml`](template.yaml) is the source of truth for:

- five `AWS::Serverless::Function` resources;
- monitor and scoring queues, each with a dead-letter queue;
- Worker and Scoring SQS event-source mappings;
- three EventBridge schedule rules;
- least-privilege Lambda IAM roles;
- retained CloudWatch log groups with configurable retention;
- Lambda error and DLQ-visible-message alarms.

Deployment target:

| Setting | Value |
| --- | --- |
| AWS account | `552185537300` |
| Region | `us-west-2` |
| CloudFormation stack | `latam-tenant-monitoring` |
| AWS CLI profile | `552185537300_DBaaS-DevUser-Role` |

This repository change prepares the stack but does not deploy it. Validate the
target account, parameters, and schedules before the first deployment.

## Schedules

The stack creates all schedules with `ScheduleState=DISABLED` by default.
The configured times favor China morning hours. Dispatcher starts five minutes
after the UTC day boundary so its previous-day query reads the complete prior
UTC day, while giving the Worker 1 hour 55 minutes to drain the monitor queue
before the daily Summary:

| Task | EventBridge expression | UTC | Mexico City | Bogota/Lima | Sao Paulo/Buenos Aires | Asia/Shanghai |
| --- | --- | --- | --- | --- | --- | --- |
| Dispatcher | `cron(5 0 * * ? *)` | Daily 00:05 | Previous day 18:05 | Previous day 19:05 | Previous day 21:05 | Daily 08:05 |
| Summary | `cron(0 2 * * ? *)` | Daily 02:00 | Previous day 20:00 | Previous day 21:00 | Previous day 23:00 | Daily 10:00 |
| Weekly Metrics | `cron(0 4 ? * MON *)` | Monday 04:00 | Sunday 22:00 | Sunday 23:00 | Monday 01:00 | Monday 12:00 |

Enable schedules through a CloudFormation/SAM update only after the smoke test
succeeds; do not change the rules directly and create stack drift.

## Validate and build

Run these commands from this directory:

```bash
python3 -m unittest discover -s tests -p 'test_*.py' -v
sam validate --lint --template-file template.yaml
sam build --template-file template.yaml
```

Deploy the built template through the approved secret-injection workflow:

```bash
sam deploy \
  --template-file .aws-sam/build/template.yaml \
  --stack-name latam-tenant-monitoring \
  --region us-west-2 \
  --profile 552185537300_DBaaS-DevUser-Role \
  --capabilities CAPABILITY_IAM \
  --resolve-s3
```

Required stack parameter values must be supplied securely at deploy time. Do
not add them to this command, shell history, repository files, or documentation
as plain text.

## Rollout checklist

1. Keep all three schedules disabled.
2. Invoke Dispatcher with a small `limit` and verify that both SQS queues drain.
3. Confirm that the monitor and scoring DLQs remain empty.
4. Validate the resulting `latam_active_tenants` row, ICP categories, score
   snapshot, and HubSpot `latam_pql_priority_tier` value.
5. Manually invoke Summary and Weekly Metrics and verify their Feishu cards.
6. Enable the schedules with a SAM stack update and monitor Lambda errors and
   DLQ alarms during the first complete run.