# daily-oncall-metrics

Unified cronjob for daily oncall metrics synchronization.

## Overview

This cronjob aggregates multiple data sources and syncs them to Feishu Bitable for tracking and monitoring.

## Architecture

```
daily-oncall-metrics/
├── README.md                   # This file
├── main.py                    # Unified entry point
├── base.py                    # Common utilities (token, bitable, notifications)
├── failed_job_sync.py         # Failed import/DM job sync to Bitable
├── jira_ticket_sync.py       # TCOC & Alert ticket sync
└── ...
```

## Execution Flow

### Step 1: main.py - Entry Point
```bash
python3 cronjobs/team-workflow/daily-oncall-metrics/main.py
```

1. Loads configuration from environment (via base.py)
2. Calls each sync module sequentially:
   - failed_job_sync.py - Syncs failed import/DM jobs to Bitable
   - jira_ticket_sync.py - Syncs TCOC & Alert tickets
3. Sends summary notification to Feishu chat

### Step 2: failed_job_sync.py - Data Sync
1. Queries Grafana for failed import jobs
2. Writes records to Feishu Bitable (deduplication by Job ID + Failed Tasks)
3. Returns statistics: `{import: count, dm: count, added: count}`

`Failed Reason` is synced as the full normalized text from Grafana so downstream wiki reports can preserve the original error details.

**Bitable**: `Q3ahbd2doaPwVVsItRfclxZJnCc` / `tblGuNt6JW9eVGvx`

### Step 3: main.py - Send Summary
1. Receives sync results from failed_job_sync.py
2. Sends summary message to Feishu chat

## Data Flow Diagram

```
┌─────────────────────────────────────────────────────────────┐
│                     main.py                                 │
│  1. Load config                                            │
│  2. Call failed_job_sync.py                               │
│     └─► Query Grafana → Write Bitable → Return stats       │
│  3. Send summary message                                   │
└─────────────────────────────────────────────────────────────┘
```

## Current Status

| Module | Status |
|--------|--------|
| failed_job_sync.py (Import) | ✅ Working |
| failed_job_sync.py (DM) | ✅ Working |
| jira_ticket_sync.py (TCOC) | ✅ Working |
| jira_ticket_sync.py (Alert) | ✅ Working |

**Note:** Dataflow failed job notification moved to separate cronjob: `dataflow-failed-job-analyze`

## Usage

### Run daily (for previous day data)
```bash
python3 cronjobs/team-workflow/daily-oncall-metrics/main.py
```

The Jira sync keeps two TCOC paths in sync:
1. New TCOC tickets created on the target date are added to Bitable.
2. Existing TCOC rows in Bitable whose status is not terminal are re-queried from Jira so status changes do not stay stuck at `UNDER INVESTIGATION`.

### Run for specific date
```bash
python3 cronjobs/team-workflow/daily-oncall-metrics/failed_job_sync.py 20260316
```

### Run failed job sync only
```bash
python3 cronjobs/team-workflow/daily-oncall-metrics/failed_job_sync.py
```

## Required Environment Variables

Add to `~/.bashrc`:

```bash
# Feishu App
export CRONJOB_TEAM_WORKFLOW_APP_ID="cli_a92c6d7546f85cc6"
export CRONJOB_TEAM_WORKFLOW_APP_SECRET="your-secret"

# Bitable - Failed Jobs
export FAILED_JOBS_APP_TOKEN="Q3ahbd2doaPwVVsItRfclxZJnCc"
export FAILED_JOBS_TABLE_ID="tblGuNt6JW9eVGvx"

# Bitable - Oncall Records (for Jira sync)
export ONCALL_RECORDS_APP_TOKEN="your-token"
export ONCALL_RECORDS_TABLE_ID="your-table-id"

# Grafana
export DATASERVICE_GRAFANA_TOKEN="your-grafana-token"

# Notification (optional - defaults to user's chat_id)
export ONCALL_NOTIFY_CHAT_ID="oc_xxx"
```

## Debugging

### Skip sending notification
Set `SKIP_SEND_NOTIFICATION=true` to skip sending Feishu message:
```bash
SKIP_SEND_NOTIFICATION=true python3 cronjobs/team-workflow/daily-oncall-metrics/main.py
```

## Schedule

Daily at 1:00 AM UTC (9:00 AM Beijing time).

```bash
0 1 * * * cd /path/to/repo && python3 cronjobs/team-workflow/daily-oncall-metrics/main.py
```