# Problem Domain Classification Standard

## 1. Introduction

This document defines the standardized Problem Domain classification system for Jira tickets. The purpose of this system is to identify the **business scenario or operational context** in which the user encountered a problem, answering the question: **"What was the user trying to do when they encountered this issue?"**

Problem Domain is designed to be scenario-oriented rather than component-oriented, providing insights that are immediately actionable for product prioritization and management reporting.

## 2. Design Philosophy

The core design philosophy of Problem Domain is:

* **Scenario-Oriented**: Focus on what the user was doing (business activity) rather than which component failed (technical detail).
* **Business-Understandable**: Categories should be clear to executives, product managers, and customers, not just engineers.
* **Actionable for Product**: Each domain should directly map to product improvement opportunities or investment areas.
* **Flat Structure**: Single-level classification ideal for pie charts and executive dashboards.
* **Array Field Support**: A ticket can involve multiple domains (e.g., a scaling operation that encounters network issues).

## 3. Comparison with Problem Pattern

Problem Domain and Problem Pattern are complementary dimensions that provide different perspectives:

| Aspect | Problem Pattern L1 | Problem Domain |
|--------|-------------------|----------------|
| **Perspective** | Technical component/symptom | Business scenario/activity |
| **Example** | `kernel`, `operation`, `data_utils` | `query_optimization`, `data_import`, `cluster_scaling` |
| **Audience** | Engineers, Support | Management, Product, Customers |
| **Purpose** | Technical classification | Business intelligence |
| **Actionability** | Guides troubleshooting | Guides product investment |

**Why Scenario-Oriented?**

Problem Domain focuses on **what the user was trying to do** rather than **which component had issues**. For example:
- "Query Optimization: 30%" tells us users struggle with query optimization → Invest in better query diagnostics, SQL optimization tools, auto-tuning features
- This is more actionable for product planning than "Kernel component: 30%"

## 4. Classification Structure

Problem Domain uses a **flat, single-level structure** with **21 categories** organized into 6 logical groups. This is an **ARRAY field** in BigQuery, allowing multiple domains per ticket.

## 5. Usage Guidelines

1. **Classify at Creation or Early Triage**: Problem Domain should be identifiable from the user's description of what they were trying to do.
2. **Think from User Perspective**: Ask "What business activity or operation was the user performing?"
3. **Multiple Domains Allowed**: If a ticket involves multiple scenarios (e.g., scaling + network issues), tag both domains.
4. **Keep It Simple**: Choose the most specific domain that describes the user's activity.

## 6. Classification Table

The following is the complete Problem Domain classification definition table, organized by category.

### Category 1: Database Query & Transaction

| Problem Domain | Code | Description | Typical User Activities | Corresponding Problem Pattern |
| :------------- | :--- | :---------- | :---------------------- | :---------------------------- |
| **Query Optimization** | `query_optimization` | SQL query performance issues requiring optimization of execution plans, indexes, or statistics. | - Analyzing and optimizing slow queries<br>- Adding or modifying indexes<br>- Updating statistics<br>- Rewriting inefficient SQL<br>- Using hints to guide optimizer<br>- Troubleshooting full table scans | `kernel/slow_query`<br>`kernel/high_resource_usage` (query-induced) |
| **Slow Query Analysis** | `slow_query_analysis` | Slow query diagnosis, root cause analysis and optimization recommendations. | - Diagnose slow query root causes<br>- Analyze slow query logs<br>- Get SQL optimization recommendations<br>- Slow query pattern recognition | `kernel/slow_query`<br>`kernel/high_resource_usage` (query-induced) |
| **Write Performance** | `write_performance` | DML write operation performance issues (INSERT, UPDATE, DELETE, LOAD DATA). | - Batch inserting data<br>- Updating large tables<br>- Deleting expired data<br>- Loading data files (LOAD DATA)<br>- Handling write hotspots<br>- Dealing with write stalls | `kernel/slow_write`<br>`kernel/high_resource_usage` (write-induced) |
| **DDL Execution** | `ddl_execution` | DDL operation issues (CREATE/DROP/ALTER TABLE, CREATE INDEX, etc.). | - Adding columns to large tables<br>- Creating indexes on production tables<br>- Dropping tables or indexes<br>- Modifying table schemas<br>- Online DDL operations<br>- DDL performance tuning | `kernel/ddl` |
| **Transaction & Locking** | `transaction_locking` | Transaction conflicts, lock waits, deadlocks, and consistency issues. | - Resolving lock wait timeouts<br>- Diagnosing deadlocks<br>- Handling write conflicts<br>- Understanding transaction isolation<br>- Optimizing concurrent workloads<br>- TSO-related issues | `kernel/query_error` (lock-related)<br>`kernel/inconsistent_behavior` |

### Category 2: Data Migration & Sync

| Problem Domain | Code | Description | Typical User Activities | Corresponding Problem Pattern |
| :------------- | :--- | :---------- | :---------------------- | :---------------------------- |
| **Data Import** | `data_import` | Data import scenarios using TiDB Lightning, LOAD DATA, or CSV import. | - Importing TB-scale data with Lightning<br>- Loading CSV files into tables<br>- Handling data format issues during import<br>- Optimizing import performance<br>- Managing resources during import<br>- Refreshing statistics after import | `data_utils/tidb_lightning`<br>`kernel/slow_write` (LOAD DATA) |
| **Data Migration** | `data_migration` | Cross-database data migration, primarily using DM (Data Migration) tool. | - Migrating from MySQL/MariaDB to TiDB<br>- Configuring DM migration tasks<br>- Handling compatibility issues<br>- Resolving migration interruptions<br>- Managing migration lag<br>- Syncing DDL changes | `data_utils/dm` |
| **Real-time Replication** | `realtime_replication` | Real-time data synchronization using TiCDC (Change Data Capture). | - Setting up TiCDC changefeeds<br>- Monitoring replication lag<br>- Troubleshooting changefeed failures<br>- Syncing to Kafka/MySQL/other sinks<br>- Managing TiCDC checkpoints<br>- Handling downstream issues | `data_utils/cdc_latency`<br>`data_utils/cdc_interruption` |
| **Backup & Recovery** | `backup_recovery` | Backup and restore operations using BR (Backup & Restore) or PITR. | - Backing up clusters to S3/GCS<br>- Restoring from backup files<br>- Point-in-time recovery (PITR)<br>- Managing backup storage<br>- Validating backup integrity<br>- Disaster recovery operations | `data_utils/br` |

### Category 3: Cluster Operations

| Problem Domain | Code | Description | Typical User Activities | Corresponding Problem Pattern |
| :------------- | :--- | :---------- | :---------------------- | :---------------------------- |
| **Cluster Scaling** | `cluster_scaling` | Cluster scaling operations (scale-out, scale-in, scale-up, scale-down). | - Adding TiKV/TiDB/TiFlash nodes<br>- Removing nodes from cluster<br>- Increasing node specifications<br>- Decreasing node specifications<br>- Rebalancing regions after scaling<br>- Handling scaling-induced performance issues | `operation/scaling_out`<br>`operation/scaling_in`<br>`operation/scaling_up`<br>`operation/scaling_down`<br>`operation/manual_scheduling` |
| **Version Upgrade** | `version_upgrade` | Cluster version upgrade operations. | - Planning version upgrades<br>- Checking upgrade compatibility<br>- Performing rolling upgrades<br>- Handling upgrade failures<br>- Rolling back failed upgrades<br>- Testing new version features | `operation/upgrade` |
| **Configuration Management** | `configuration_management` | Cluster configuration parameter tuning and management. | - Tuning memory parameters<br>- Adjusting concurrency settings<br>- Modifying cache configurations<br>- Optimizing performance parameters<br>- Understanding configuration impacts<br>- Rolling back configuration changes | `operation/config_change` |
| **Node Crashes** | `node_crashes` | Node crashes, panics and unexpected restarts. | - Analyzing crash stacks<br>- Diagnosing crash root causes<br>- Handling frequent restarts<br>- Coordinating repair measures | `kernel/panic`<br>`operation/node_restart` |
| **Leader Election** | `leader_election` | PD Leader election, Region Leader switching issues. | - Leader election failures<br>- Frequent leader switching<br>- Leader imbalance<br>- Election performance impact | `kernel/performance_jitter` |
| **Performance Jitter** | `performance_jitter` | Performance jitter, fluctuations and instability. | - Diagnose performance jitter root causes<br>- Analyze jitter time patterns<br>- Correlate multi-dimensional metrics<br>- Jitter impact assessment | `kernel/performance_jitter` |

### Category 4: Resource Management

| Problem Domain | Code | Description | Typical User Activities | Corresponding Problem Pattern |
| :------------- | :--- | :---------- | :---------------------- | :---------------------------- |
| **Memory Management** | `memory_management` | Memory usage optimization and OOM prevention. | - OOM error handling<br>- Memory parameter tuning<br>- Memory usage analysis<br>- Memory leak detection | `kernel/oom`<br>`kernel/high_resource_usage` (memory) |
| **Disk I/O Optimization** | `disk_io_optimization` | Disk I/O performance and storage optimization. | - I/O performance tuning<br>- Disk space management<br>- Storage parameter optimization<br>- I/O-intensive workload optimization | `kernel/high_resource_usage` (disk)<br>`kernel/stuck_component` |
| **CPU Saturation** | `cpu_saturation` | CPU usage optimization and saturation handling. | - CPU saturation handling<br>- CPU-intensive query optimization<br>- Concurrency control tuning<br>- CPU load balancing | `kernel/high_resource_usage` (CPU) |
| **Network Connectivity** | `network_connectivity` | Network connection performance and latency optimization. | - Network latency optimization<br>- Connection timeout handling<br>- Network bandwidth management<br>- Cross-region network optimization | `connectivity/vpc_peering`<br>`cloud_vendor/network_latency`<br>`connectivity/timeout_or_refused` |

### Category 5: Platform & Infrastructure

| Problem Domain | Code | Description | Typical User Activities | Corresponding Problem Pattern |
| :------------- | :--- | :---------- | :---------------------- | :---------------------------- |
| **Monitoring Dashboards** | `monitoring_dashboards` | Monitoring dashboards, alert configuration and metrics interpretation. | - Configure custom alerts<br>- Create monitoring dashboards<br>- Interpret monitoring metrics<br>- Set alert rules | `monitoring/alert`<br>`monitoring/metrics_interpretation`<br>`monitoring/custom_rule` |
| **Platform Console** | `platform_console` | Cloud platform console, API operations and automation. | - Console operation optimization<br>- API calls and automation<br>- Batch operation management<br>- Console performance optimization | `operation/whitelist_request`<br>`operation/quota_request` |
| **Quota Limits** | `quota_limits` | Quota management, limit adjustments and resource requests. | - Quota application and approval<br>- Limit parameter tuning<br>- Resource usage monitoring<br>- Quota alert configuration | `operation/quota_request` |

### Category 6: Support & Consulting

| Problem Domain | Code | Description | Typical User Activities | Corresponding Problem Pattern |
| :------------- | :--- | :---------- | :---------------------- | :---------------------------- |
| **Product Consulting** | `product_consulting` | Product features, architecture design and capability consulting. | - Architecture design consulting<br>- Feature usage guidance<br>- Capability boundary assessment<br>- Product selection recommendations | `consultation/how_to`<br>`consultation/best_practice`<br>`feature_request/*` (capability inquiries) |
| **Best Practices** | `best_practices` | Best practices, usage guidelines and optimization recommendations. | - Best practices sharing<br>- Usage guideline development<br>- Performance optimization recommendations<br>- Architecture best practices | `consultation/best_practice` |
| **Security Configuration** | `security_configuration` | Security configuration, permission management and compliance settings. | - Security policy configuration<br>- Permission management setup<br>- Compliance requirement fulfillment<br>- Security best practices | `consultation/how_to` |
| **Cost Optimization** | `cost_optimization` | Cost analysis, resource optimization and expense control. | - Cost analysis and reporting<br>- Resource usage optimization<br>- Expense budget management<br>- Cost savings recommendations | `consultation/best_practice` |

## 7. Relationship to Other Dimensions

### Problem Domain vs Problem Pattern

**Problem Pattern** is **technical/symptom-oriented**:
- "The symptom is a slow query in the kernel component"
- Perspective: Support engineer looking at technical manifestation

**Problem Domain** is **scenario/business-oriented**:
- "The user was performing query optimization"
- Perspective: Product manager understanding user activity

**Example Mapping:**

| User Activity (Problem Domain) | Technical Symptom (Problem Pattern) |
|-------------------------------|-------------------------------------|
| `query_optimization` | `kernel/slow_query`, `kernel/high_resource_usage` |
| `data_import` | `data_utils/tidb_lightning`, `kernel/slow_write` |
| `cluster_scaling` | `operation/scaling_out`, `operation/manual_scheduling` |
| `memory_management` | `kernel/oom`, `kernel/high_resource_usage` (memory) |
| `disk_io_optimization` | `kernel/high_resource_usage` (disk), `kernel/stuck_component` |
| `cpu_saturation` | `kernel/high_resource_usage` (CPU) |
| `node_crashes` | `kernel/panic`, `operation/node_restart` |
| `performance_jitter` | `kernel/performance_jitter` |

### Problem Domain vs Issue Type

**Issue Type** is **user intent**:
- "Why did the user contact us?" (Bug, Performance, Feature Gap, etc.)

**Problem Domain** is **user activity**:
- "What was the user doing?" (Query Optimization, Data Import, etc.)

**Cross-Analysis Example:**

| Problem Domain | Issue Type | Interpretation |
|---------------|-----------|----------------|
| `data_import` | `Performance` | Data import is slow (performance issue) |
| `data_import` | `User Experience` | Data import tools are hard to use |
| `data_import` | `Bug` | Data import has a defect |
| `query_optimization` | `Performance` | Queries are slow |
| `query_optimization` | `User Experience` | Query optimization is complex/unclear |

This cross-analysis reveals: Is the problem the product itself (Bug/Performance) or how users interact with it (User Experience)?

### Problem Domain vs Service Type

**Problem Domain** describes **what the user was doing**:
- "User was performing cluster scaling"

**Service Type** describes **what service we delivered**:
- "We provided operational guidance"

**Example Mapping:**

| Problem Domain | Service Type | Meaning |
|---------------|--------------|---------|
| `query_optimization` | `deep_troubleshooting` | User optimizing queries; we did deep kernel analysis |
| `query_optimization` | `performance_tuning` | User optimizing queries; we provided SQL tuning |
| `data_import` | `data_tool_support` | User importing data; we helped with Lightning |
| `cluster_scaling` | `operational_guidance` | User scaling cluster; we provided operation guidance |
| `product_consulting` | `technical_consulting` | User learning product; we provided consultation |
| `best_practices` | `technical_consulting` | User seeking best practices; we provided recommendations |
| `security_configuration` | `technical_consulting` | User configuring security; we provided guidance |
| `cost_optimization` | `technical_consulting` | User optimizing costs; we provided analysis |

## 8. Analysis Examples

### Example 1: Product Investment Priorities

**Query:** "Which business scenarios have the most issues?"

```sql
SELECT
  domain,
  COUNT(*) as ticket_count,
  ROUND(COUNT(*) * 100.0 / (SELECT COUNT(*) FROM jira.ticket, UNNEST(ai_predicted_problem_domains) WHERE created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 MONTH)), 2) as percentage
FROM jira.ticket,
UNNEST(ai_predicted_problem_domains) as domain
WHERE created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 MONTH)
GROUP BY domain
ORDER BY ticket_count DESC
LIMIT 10;
```

**Sample Result:**
```
query_optimization: 450 (28%)    ← Top priority: improve query diagnostics
data_import: 280 (17%)           ← Second: enhance Lightning tools
cluster_scaling: 180 (11%)       ← Third: smoother scaling experience
realtime_replication: 150 (9%)   ← Fourth: TiCDC stability
```

**Insight:** "Query optimization accounts for 28% of tickets. We should invest in: better query diagnostics, auto-tuning capabilities, and SQL optimization recommendations."

### Example 2: Scenario + Issue Type

**Query:** "For data import, are issues mostly bugs or UX problems?"

```sql
SELECT
  ai_predicted_issue_type,
  COUNT(*) as count,
  ROUND(COUNT(*) * 100.0 / SUM(COUNT(*)) OVER(), 2) as percentage
FROM jira.ticket,
UNNEST(ai_predicted_problem_domains) as domain
WHERE domain = 'data_import'
  AND created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 MONTH)
GROUP BY ai_predicted_issue_type
ORDER BY count DESC;
```

**Sample Result:**
```
Performance: 120 (43%)           ← Import is slow
User Experience: 85 (30%)        ← Import tools are hard to use
Bug: 45 (16%)                    ← Import has defects
Operational Support: 30 (11%)    ← Users need help importing
```

**Insight:** "Data import issues are 43% performance and 30% UX. Priority actions: 1) Optimize Lightning performance, 2) Simplify Lightning configuration and improve error messages."

### Example 3: Customer Pain Points

**Query:** "What scenarios does customer Wetech struggle with most?"

```sql
SELECT
  domain,
  COUNT(*) as count
FROM jira.ticket,
UNNEST(ai_predicted_problem_domains) as domain
WHERE customer = 'wetech'
  AND created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 MONTH)
GROUP BY domain
ORDER BY count DESC
LIMIT 5;
```

**Insight:** Understand customer-specific pain points to provide targeted support, training, or product improvements.

### Example 4: Scenario Complexity

**Query:** "Which scenarios require the most expert-level services?"

```sql
SELECT
  domain,
  ai_predicted_service_type,
  COUNT(*) as count
FROM jira.ticket,
UNNEST(ai_predicted_problem_domains) as domain
WHERE ai_predicted_service_type IN ('deep_troubleshooting', 'bug_investigation')
  AND created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 3 MONTH)
GROUP BY domain, ai_predicted_service_type
ORDER BY count DESC
LIMIT 10;
```

**Sample Result:**
```
query_optimization + deep_troubleshooting: 180
node_crashes + deep_troubleshooting: 95
performance_jitter + deep_troubleshooting: 45
realtime_replication + bug_investigation: 67
```

**Insight:** "Query optimization, node crashes and performance jitter consistently require deep troubleshooting. These areas need: better diagnostic tools, automation, or product hardening."

### Example 5: Trend Over Time

**Query:** "Are problem domain distributions changing over time?"

```sql
SELECT
  DATE_TRUNC(created_at, MONTH) as month,
  domain,
  COUNT(*) as count
FROM jira.ticket,
UNNEST(ai_predicted_problem_domains) as domain
WHERE created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 6 MONTH)
GROUP BY month, domain
ORDER BY month, domain;
```

**Insight:** Track if product improvements are reducing issues in specific domains (e.g., fewer "data_import" tickets after Lightning optimization).

## 9. Best Practices

### For Support Team
- ✅ Think from user's perspective: "What were they trying to do?"
- ✅ Tag multiple domains if ticket involves multiple activities
- ✅ Use specific domains when possible (prefer `query_optimization` over generic categories)
- ✅ Update Problem Domain if user's actual activity becomes clearer during investigation

### For Product Team
- ✅ Use Problem Domain for product roadmap prioritization
- ✅ Cross-analyze with Issue Type: Are issues bugs or UX?
- ✅ Track trends: Are improvements reducing issues in specific domains?
- ✅ Focus on high-frequency + high-complexity domains

### For Management
- ✅ Use Problem Domain for executive reporting (simple pie charts)
- ✅ Communicate priorities with data: "Query optimization is our #1 pain point"
- ✅ Set measurable goals: "Reduce data import tickets by 30% next quarter"
- ✅ Allocate resources based on domain distribution

## 10. Conclusion

Problem Domain provides a business-oriented view of ticket distribution by focusing on **what users are trying to do** rather than which technical components are involved. This classification:

1. **Actionable for Product**: Directly guides product investment and prioritization
2. **Understandable by All**: Clear to executives, PMs, and customers, not just engineers
3. **Complementary**: Works with Problem Pattern (technical) and Issue Type (intent)
4. **Flexible**: Array field allows capturing complex scenarios

Problem Domain is most valuable when combined with other dimensions:
- **Problem Domain + Issue Type**: Understand both activity and problem nature
- **Problem Domain + Service Type**: Understand both activity and service depth
- **Problem Domain + Root Cause**: Link business scenarios to technical causes
- **Problem Domain + Customer**: Identify customer-specific pain points

This multi-dimensional approach provides comprehensive insights for improving product quality, user experience, and business outcomes.