# AGENTS.md

This file provides guidance to AI coding assistants (Claude Code, Cursor, Copilot, etc.) when working with code in this repository.

## Project Overview

FinOps multi-cloud billing analytics platform that transforms raw billing data from AWS, GCP, Azure, and Alibaba Cloud into unified analysis-ready data marts using dbt and BigQuery.

## Build/Development Commands

### dbt (primary data layer)
```bash
cd dbt
dbt build                    # RECOMMENDED - runs seeds + models + tests in order
dbt seed --full-refresh      # Reload account_mapping after CSV changes
dbt run --select marts       # Rebuild marts only
dbt test                     # Run data quality tests
dbt docs generate && dbt docs serve  # Generate and view documentation
```

**Critical**: Always use `dbt build` instead of separate commands. Running `dbt run` without `dbt seed` first will fail because intermediate models depend on account_mapping.

### data-agent-v1 (Next.js web app — v1 archived)
```bash
cd data-agent-v1
make dev      # Start dev server at http://localhost:3000
make build    # Production build
make lint     # ESLint checks
make install  # Install dependencies
```
Prefer Makefile commands over npm.
**Note**: v1 code is archived in `data-agent-v1/`. Design docs for v2 remain in `data-agent/design/`.

### Docker deployment
```bash
docker-compose up -d --build data-agent  # Fast redeploy
docker-compose logs -f data-agent        # View logs
```

## Workflow Requirements

**After making code changes, ALWAYS:**
1. Run `make lint` and `make build` for data-agent changes
2. Run `dbt build` for dbt model changes
3. Update `work.md` if any TODO items are completed or new tasks are identified
4. Update `dbt/design/*.md` if architecture or schema changes are made

## Architecture

### dbt Three-Layer Model
```
Raw (gcp-digital-bi.aws_billing_no_payer.aws_billing_*)
  → Staging (stg_aws_cost) - VIEW, 1:1 clean, all rows preserved
    → Intermediate (int_aws_billing) - VIEW, usage rows only, account mapping joined
      → Marts - TABLE, partitioned by billing_period_month
        • mart_cost_monthly_by_region
        • mart_cost_monthly_by_service
        • mart_cost_monthly_by_region_service
```

### Target Database
- Project: `gcp-rd-research-dev`
- Dataset: `finops_cloud_billing`
- Location: `us-west2`

### Account Mapping (SCD Type 2)
Business attributes for cloud accounts are managed via `dbt/seeds/account_mapping.csv`. Uses effective_from/effective_to date ranges for historical tracking. Run `dbt seed --full-refresh` after CSV changes.

## Key Directories

- `BigQuery/`: Raw SQL queries organized by cloud provider (cloud_aws/, cloud_gcp/, cloud_azure/, cloud_alibaba/)
- `dbt/`: dbt transformation project (models/, seeds/, tests/, analyses/)
- `dbt/design/`: Architecture and deployment documentation - **update when schema changes**
- `data-agent-v1/`: Next.js 14 web agent for SQL generation via LLM (v1, archived)
- `data-agent/design/`: v2 platform design docs and task breakdowns
- `docs/`: Schema references and billing guides
- `work.md`: Implementation TODOs - **update when tasks change**

## Superset Dashboard Conventions

Every Superset virtual dataset (VDS) and its associated dashboard/charts **must** have a corresponding design document at `dbt/design/superset-<dashboard-slug>.md`. The design doc must include:

1. **Data Semantics** — source table, primary dimensions, metrics, and any field mapping notes
2. **Virtual Dataset SQL** — full SQL template with placeholder variables
3. **Chart Specifications** — each chart's viz type, group-by, metrics, and purpose
4. **Native Filter Design** — filter list, cascade relationships, and defaults
5. **Dashboard Metadata** — title, slug, label colors
6. **Bootstrap Script** — reference to `dbt/scripts/create_superset_<slug>.sh`

Account filter convention:
- Any dashboard with an Account native filter **must** use an `account_label` column (`CONCAT(account_id, ' | ', account_display_name)`) so users see both ID and name in the dropdown.
- The VDS SQL must include `account_label` as a computed column and the native filter target must point to `account_label` instead of `account_display_name`.

When creating, modifying, or deleting a Superset dashboard:
- **Create**: Write the design doc first, then implement the bootstrap script
- **Modify**: Update the design doc to reflect all changes
- **Delete**: Remove the design doc, bootstrap script, and clean up VDS/charts/dashboard from Superset

Superset API operations (create/update/delete datasets, charts, dashboards, filters) are documented in the `/superset` skill. Use it for any Superset management task.

Current inventory:
- `vds_aws_tag_compliance` → `dbt/design/superset-tag-compliance-dashboard.md` + `dbt/scripts/create_superset_tag_compliance.sh`
- `vds_aws_tenant_cost_trend` → `dbt/design/superset-tenant-cost-trend.md` + `dbt/scripts/create_superset_tenant_cost_trend.sh`
- `vds_aws_dev_account_cost_trend` → `dbt/design/superset-aws-dev-account-cost-trend.md` + `dbt/scripts/create_superset_aws_dev_account_cost_trend.sh`
- `vds_prod_region_cost_trend` → `dbt/design/superset-prod-region-cost-trend.md` + `dbt/scripts/create_superset_prod_region_cost_trend.sh`
- `vds_prod_cost_anomaly` → `dbt/design/superset-prod-cost-anomaly.md` + `dbt/scripts/create_superset_prod_cost_anomaly.sh`
- `vds_dev_cost_anomaly` → `dbt/design/superset-dev-cost-anomaly.md` + `dbt/scripts/create_superset_dev_cost_anomaly.sh`

## Browser Screenshots (Playwright MCP)

When taking browser screenshots for dashboard verification, save files to `.playwright-mcp/` directory (already in `.gitignore`). Never save screenshots to the repo root.

```bash
# Playwright MCP screenshot filename examples
.playwright-mcp/dashboard-tenant-cost-trend.png
.playwright-mcp/dashboard-tag-compliance.png
```

## Coding Conventions

- **Commits**: Conventional Commits format (feat:, docs:, chore:, fix:)
- **TypeScript**: 2-space indentation, single quotes
- **SQL**: snake_case file and column names
- **Tailwind**: Keep class ordering consistent with nearby code

## Key Data Semantics

- **Usage rows only in marts**: Tax/Fee/Discount/Credit rows excluded from analysis marts
- **savings_rate**: `(public_ondemand_cost - cost_usd) / public_ondemand_cost` (range 0.0-1.0)
- **charge_type**: Normalized to Usage, Tax, Discount, Credit, Fee, Refund, Other
- **pricing_model**: OnDemand, Reserved, SavingsPlan, Spot, Discount, Fee

## Authentication

```bash
gcloud auth application-default login
gcloud config set project gcp-rd-research-dev
```
Never commit secrets. Use `.env.local` for data-agent API keys.