# Changefeed OpenAPI Review Round 2 Design

## 1. Conclusion

**We will implement a backward-compatible fix set for Review Round 2 in the current branch, plus MySQL PrivateLink request semantics, without request-payload breaking changes.**

The implementation focuses on routing correctness, pagination correctness, validation correctness, and API documentation/field-level clarity.

## 2. Scope

### 2.1 In Scope

1. **List RCUs 404**: expose `/v1beta1/changefeedRCUs` through HTTP gateway routing.
2. **List changefeeds `pageSize` optionality**: allow omitted `page_size` while preserving `[1,100]` validation when provided.
3. **List changefeeds `nextPageToken` correctness**: return empty token on last page.
4. **S3 flush interval default mismatch**: align OpenAPI doc default with runtime behavior (`1m`).
5. **Kafka/public error clarity**: replace legacy `kafka_connection_info`-style ambiguity with OpenAPI field-oriented validation messages.
6. **MySQL + PRIVATE_LINK semantics**: allow empty `mysql.connection.endpoint` when using PrivateLink; require `network_info.ports` for endpoint assembly.

### 2.2 Out of Scope (Explicitly Confirmed)

1. New profile API (account id / external id).
2. Payload shape redesign to unify create/edit request body style.
3. Kafka/MySQL PrivateLink end-to-end environment verification.

## 3. Design

### 3.1 HTTP Routing Fix for ListChangefeedRCUs

Add explicit gateway mapping for:

- `GET /v1beta1/changefeedRCUs`

to the v1beta1 OpenAPI grpc-gateway mux in bootstrap routing.

This removes the current pre-gateway 404 ("no Route matched with those values") and lets the existing gRPC handler execute.

### 3.2 Pagination Contract Fixes

#### 3.2.1 `page_size` Optional in Request Validation

For `ListChangefeedsRequest.page_size`, update proto validation to allow empty values:

- Keep numeric constraints: `gte: 1`, `lte: 100`
- Add `ignore_empty: true`

Result:

- Omitted/zero-value request path uses controller default page size (`20`).
- Explicit values are still validated.

#### 3.2.2 `next_page_token` on Last Page

Current behavior can emit `next_page_token` even on terminal pages.

New behavior:

- Determine `hasNext` using page index, page size, and total count from connector list response.
- Emit token only when a real next page exists.
- Last page must return empty `next_page_token`.

### 3.3 MySQL PrivateLink Input Semantics

#### 3.3.1 API Contract

For `network_info.network_type = PRIVATE_LINK`:

1. `mysql.connection.endpoint` may be empty in user request.
2. `network_info.ports` must carry the downstream port(s), and for MySQL exactly one port is expected by existing downstream logic.

#### 3.3.2 Validation/Assembly

Keep existing downstream assembly behavior in console adaptor:

- Use sink endpoint address + provided port to assemble final endpoint.

Update OpenAPI/proto validation contract so this flow is not rejected before reaching adapter logic.

### 3.4 Kafka/Public Error Message Clarification

In OpenAPI create path, add a pre-check for Kafka with PUBLIC/VPC_PEERING network:

- `kafka.broker` must exist
- `kafka.broker.broker_endpoints` must be non-empty

If violated, return parameter error with OpenAPI field wording (e.g. `kafka.broker.broker_endpoints`) instead of legacy `kafka_connection_info` naming.

### 3.5 S3 Flush Default Documentation Alignment

Update OpenAPI proto docs for `StorageFlushConfig.interval` default from `5s` to `1m` to match existing runtime defaults.

No runtime behavior change in this item.

### 3.6 Output-only `changefeed.state`

`changefeed.state` is already output-only (`readOnly` in generated swagger). No structural create-request change is introduced in this round.

## 4. Components and File Impact

### 4.1 Expected Code Touch Points

1. `internal/cmd/bootstrap/dataflow_http.go` (gateway route mapping)
2. `app/controllers/open/changefeed/adapter_model.go` (pagination + Kafka pre-check)
3. `idl/idl/dataflow/openapi/v1beta1/changefeed.proto` (validation/doc updates)

### 4.2 Generated Artifacts

Regenerated by existing repository workflow:

1. `idl/pbgen/proto/openapi/v1beta1/*`
2. `api/changefeed.swagger.json`

## 5. Error Handling Design

1. Use existing `bizerr`/`bizerr_wrapper` patterns.
2. Do not swallow downstream errors; wrap with context where applicable.
3. Prefer explicit parameter messages that map to OpenAPI field paths for customer-facing troubleshooting.

## 6. Test Strategy

### 6.1 Unit Tests

Update/add tests in OpenAPI changefeed controller test suite to cover:

1. Last page returns empty `next_page_token`.
2. Omitted `page_size` passes request validation and defaults correctly.
3. Kafka public request missing broker endpoints returns clear OpenAPI-style parameter error.

### 6.2 Regeneration Verification

After proto changes, regenerate IDL outputs and ensure:

1. Build/tests compile with generated code.
2. Swagger reflects the updated validation/doc semantics.

## 7. Risks and Mitigation

### 7.1 Risk: Route Regression

- **Mitigation**: keep route addition surgical (single endpoint mapping) and do not alter existing route groups.

### 7.2 Risk: Pagination Behavior Drift

- **Mitigation**: add explicit token behavior tests for terminal and non-terminal pages.

### 7.3 Risk: Validation Semantics Change

- **Mitigation**: restrict to targeted fields (`page_size`, MySQL endpoint semantics for PrivateLink) and preserve existing non-PrivateLink behavior.

## 8. Rollout Note

This design intentionally avoids request-payload breaking changes. It can ship in the current branch as a compatibility-preserving review-fix iteration.