# TiPOC Node And Time Planning

Use this page to decide what EC2 nodes you need, then go to
[environment-setup.md](environment-setup.md) to create them. All numbers are
rough estimates — the POC can complete even when the target QPS is not reached.

**Baseline instance type**: `c6g.2xlarge` (ARM, 8 vCPU, 16 GB RAM). The node
counts and QPS estimates below are based on testing with this type. Performance
and cost will differ with other instance types — adjust node counts accordingly.

## 1. Generation Nodes (rough guide)

How many EC2 nodes you need to generate CSV source objects. More nodes = faster
generation, but 1–2 nodes are enough for most POCs.

| Data size | Generation nodes | Typical time |
| --- | ---: | --- |
| 10G – 50G | 1 | 30 min – 2 h |
| 100G | 1 – 2 | 30 min – 2 h |
| 1T | 4 | 30 min – 2 h |
| 10T+ | run a sample first | — |

Node list format: `node-id=address:capacity`. Capacity = how many generation
workers that node runs.

```bash
TIPOC_GENERATION_NODES='node-a=10.0.0.10:15,node-b=10.0.0.11:15'
```

If you reuse existing source objects, remove `generate` from `TIPOC_PREPARE_OPS`.

## 2. Pressure Nodes (rough guide)

How many EC2 nodes and what concurrency to start with. These are ballpark
starting points, not guarantees. Assumes the baseline `c6g.2xlarge` instance type.

| Target QPS | Nodes | Starting concurrency |
| ---: | ---: | ---: |
| ≤ 35k | 1 | 64 – 256 |
| 35k – 75k | 2 | 256 – 512 |
| 75k – 125k | 4 | 1k – 4k |
| 125k – 200k | 8 | up to 8k |
| > 200k | 8+ | staged ceiling search |

Node list format:

```bash
TIPOC_PRESSURE_NODES='node-a=10.0.0.10:1:512,node-b=10.0.0.11:1:512'
TIPOC_PRESSURE_CONCURRENCY=1024
```

Fields: `node-id=address:capacity-units:sql-pool-size`.

For an 8-node first pass at 8192 total workers, start with 1024 workers and a
1024 SQL pool per node. Keep `capacity-units=1` for an even split:

```bash
TIPOC_PRESSURE_CONCURRENCY=8192
TIPOC_PRESSURE_NODES='node-a=10.0.0.10:1:1024,node-b=10.0.0.11:1:1024,node-c=10.0.0.12:1:1024,node-d=10.0.0.13:1:1024,node-e=10.0.0.14:1:1024,node-f=10.0.0.15:1:1024,node-g=10.0.0.16:1:1024,node-h=10.0.0.17:1:1024'
```

If the compiled manifest still says `4096` or keeps `512` SQL pool per node,
fix it before the first run. The rerun wrapper is for follow-up tries after a
completed run, not for correcting the initial manifest.

## 3. First-run Duration

Start short, then extend once the control window passes.

| Purpose | Duration |
| --- | --- |
| Connectivity / command validation | 300 s (5 min) |
| Standard POC run | 1800 s (30 min) |
| Ceiling-search step | at least 3 min measured per step |
| Long stability run | only after a shorter window passes |

For staged concurrency in one attempt:
`scripts/rerun/run-again.sh --concurrency-steps 1024,2048,4096`

## 4. Multi-round Planning (summary)

After the first run, use `scripts/rerun/run-again.sh` to change concurrency,
duration, or workload mix in follow-up tries.

| Change | How |
| --- | --- |
| Concurrency or duration | `run-again.sh --concurrency N --duration-seconds N` |
| Switch tpce → exchange | `run-again.sh --workload-mix exchange_matching` |
| Exchange matching ratio | `run-again.sh --exchange-matching-write-percent N --exchange-matching-sql-write-percent N` |
| Query-cost paired mode | `run-again.sh --query-cost-mode MODE --query-cost-attempt-ru-source SOURCE` |
| Query-cost RU side polling | `run-again.sh --query-cost-poll-interval-seconds N --query-cost-poll-timeout-seconds N` |

Exchange matching ratio is controlled by `exchange_matching_write_percent`
(integers from 20 through 90) and `exchange_matching_sql_write_percent` (`0`, or
10 through 90). Higher values mean more writes vs reads. Set them during render
or in a rerun:

```bash
# During init render:
TIPOC_EXCHANGE_MATCHING_WRITE_PERCENT=85 \
TIPOC_EXCHANGE_MATCHING_SQL_WRITE_PERCENT=70 \
scripts/env/premium/init-attempt.sh ... --render-only

# In a follow-up rerun try:
scripts/rerun/run-again.sh \
  --exchange-matching-write-percent 85 \
  --exchange-matching-sql-write-percent 70 \
  ...
```

Label each round as `continuation` (same attempt, same target) or
`clean comparison` (restored or reprepared target). For a real A/B benchmark,
restore the prepare-end baseline or run a fresh prepare before the comparison
round.

---

**Next**: Go to [environment-setup.md](environment-setup.md) to create the nodes you planned, then [deployment-poc-sop.md](deployment-poc-sop.md) to execute.