# App Module

## Overview

The App module is the core part of the dataflow-service project, adopting the classic MVC architecture pattern to implement the project's main business logic. This module contains components such as controllers, models, repositories, and workers, responsible for handling HTTP and gRPC requests, executing business logic, and managing background tasks.

## Directory Structure

```
app/
├── config/              # Configuration-related code
├── container/           # Dependency injection container
├── controllers/         # API and gRPC entry points
├── event_bus/           # Event bus related code
├── internal/            # Internal helper packages
├── metrics/             # Metrics monitoring related code
├── models/              # Business logic layer
├── repo/                # Data access layer, encapsulating gorm
├── utils/               # Utility functions
├── utils_test/          # Utility function tests
├── worker/              # Background worker tasks
├── workflow/            # Workflow management
└── README.md            # This file
```

## Core Components

### 1. config/

Configuration-related code, including constant definitions and error code definitions.

### 2. container/

Dependency injection container, responsible for initializing and managing various objects and services in the application.

### 3. controllers/

API and gRPC entry points, responsible for receiving requests, performing parameter validation, calling model layer methods to handle business logic, and returning responses.

#### Subdirectories:
- **console/**: Console API, used by the management console
- **inner/**: Internal API, used by other internal services
- **interceptor/**: Interceptors, used as middleware for request processing
- **open/**: Open API, used by external users
- **ops/**: Operations API, used by operations personnel
- **v1beta1/**: v1beta1 version of the API

### 4. event_bus/

Event bus related code, used for publishing and subscribing to system events.

#### Subdirectories:
- **consumer/**: Event consumers, handling received events
- **publisher/**: Event publishers, publishing system events

### 5. internal/

Internal helper packages, containing various internally used tools and services.

### 6. metrics/

Metrics monitoring related code, used for collecting and reporting system metrics.

### 7. models/

Business logic layer, implementing the core business logic of the system.

#### Main Submodules:
- **backup/**: Backup-related business logic
- **biz_changefeed/**: Business change data capture related logic
- **bucket/**: Storage bucket related business logic
- **domain/**: Domain model definitions
- **import/**: Data import related business logic
- **restore/**: Restore related business logic
- **ticdc/**: TiCDC related business logic

### 8. repo/

Data access layer, encapsulating gorm operations and providing database access interfaces.

### 9. utils/

Utility functions, providing various common utility methods.

### 10. worker/

Background worker tasks, responsible for handling time-consuming operations such as backup and restore.

### 11. workflow/

Workflow management, using fastflow engine to manage asynchronous task workflows.

## Architecture Description

The App module adopts the classic MVC architecture pattern:

1. **Controllers**: Handle HTTP and gRPC requests, perform parameter validation, call model layer methods to handle business logic, and return responses.

2. **Models**: Implement core business logic, which is the core and most heavyweight layer of the system. The model layer uses methods provided by the repository layer to access the database.

3. **Repo (Repository)**: Encapsulate database access logic, providing reusable database operation methods. The repository layer uses gorm for database operations.

4. **Worker**: Handle time-consuming operations, such as backup, restore, and other asynchronous tasks.

5. **Workflow**: Manage worker workflows, using fastflow engine to implement complex workflow logic.

## Main Features

- **Backup and Restore**: Provide data backup and restore capabilities for TiDB clusters.
- **Data Import**: Support importing data from multiple data sources to TiDB.
- **Data Migration**: Support migrating data from other database systems to TiDB.
- **Change Data Capture**: Capture data changes in TiDB clusters and synchronize them to other systems in real-time.
- **Storage Management**: Manage storage buckets and storage-related resources.

## Development Guide

### 1. Adding New API Interfaces

1. Add new controller methods in the corresponding subdirectory under `controllers/`.
2. Add corresponding business logic under `models/`.
3. Add corresponding data access methods under `repo/` (if needed).

### 2. Adding New Background Tasks

1. Add new workers under `worker/`.
2. Add corresponding workflow definitions under `workflow/` (if needed).
3. Add corresponding business logic under `models/`.

### 3. Testing

- Add unit tests for controllers to test API interface parameter validation and error handling.
- Add unit tests for models to test the correctness of business logic.
- Add unit tests for repositories to test the correctness of data access.

## Notes

- Do not write business logic in controllers; all business logic should be implemented in the model layer.
- Model layer methods should have good testability and be defined using interfaces.
- The repository layer should encapsulate database operations and provide reusable methods.
- Workers and workflows should handle time-consuming operations to avoid blocking request processing threads.
- The event bus should be used for internal system event notifications to avoid direct synchronous calls.