# Slow Log Explorer MaxCompute timeout fix

## Background

Slow Log Explorer can return a gateway timeout for Alibaba Cloud Premium
clusters. The list endpoint currently selects every list column, including the
full query text, even when the UI displays only a small subset. The initial UI
load also starts the list and three summary queries concurrently. The list SSE
response does not write a byte until the first warehouse query completes, so a
slow MaxCompute query can cross the gateway idle-read boundary before the first
batch is delivered.

## Goal

Make the first Slow Log Explorer page reliable for MaxCompute-backed clusters
while preserving the existing API and behavior for other providers and older
clients.

## Current behavior

- `GET /clinic/api/v1/slowlog-explorer/orgs/:oid/clusters/:cid/slowlogs`
  always selects the complete `slowlogColumns` list.
- The UI `cols` setting controls rendering only and is not sent to the list
  endpoint.
- The default UI page size is 50, which makes the backend execute a 30-row
  first query and a second query for the remainder.
- Summary statistics, trend, and top-digest queries start alongside the list.
- The SSE handler sets streaming headers but does not flush an event or comment
  before waiting for Data Proxy.

## Target contract

- Add an optional comma-separated `fields` query parameter to the list route.
- For MaxCompute only, `fields` contains allowlisted raw slow-log list column
  names. Unknown names and an explicitly blank selection return `400` rather
  than being interpolated into SQL; a valid value narrows the SELECT
  projection. Other dialects completely ignore this optimization parameter to
  preserve their old behavior.
  The server always adds `time`, `digest`, and `conn_id`, which are required
  for event identity and cursoring, plus the active `order_by` column required
  by MaxCompute. The UI requests `query` only when its query preview column is
  active.
- When `fields` is absent, and for every non-MaxCompute dialect, the existing
  full projection remains unchanged.
- The SSE response sends an initial comment and periodic comments while a Data
  Proxy query is pending. Only the handler goroutine writes to the response.
- The list route emits structured authorization, validation, upstream, cancel,
  and completion lifecycle logs with a trace ID and safe counts, without
  logging SQL, filters, field values, query text, or response payloads.
- The UI sends the raw fields needed by its pinned and visible columns. For
  Alibaba Cloud and unresolved provider metadata, it delays Summary loading
  until the list request settles and caps list pages at 30 rows. Other known
  providers keep their current page-size and concurrent Summary behavior.

## Affected modules and routes

- Root Go module `github.com/pingcap-inc/clinic`:
  `pkg/api/slowlog_explorer`.
- Clinic API executable: `cmd/server/main.go`.
- Customer-facing route: Slow Log Explorer list SSE route only. Existing
  stats, trend, top-digest, detail, and raw-detail contracts are unchanged.
- Clinic UI repository: `packages/slow-log-explorer`.

No module ownership, executable, image, workflow, or app-delivery mapping is
changed, so `docs/clinic-module-overview.md` does not require an update.

## Implementation steps

1. Parse and validate the optional list `fields` parameter.
2. Extend the slow-log list SQL builder to accept an explicit projection and
   use it only for the MaxCompute dialect.
3. Add a handler helper that waits for a Data Proxy result while emitting SSE
   comments from the handler goroutine and respecting request cancellation.
4. Add route-local structured lifecycle logging with span trace ID and request
   header fallback.
5. Extend focused SQL-builder and handler tests for compatibility, validation,
   projection, initial flush, heartbeat, and cancellation behavior.
6. Map UI column keys to raw list fields and include them in the stream URL.
7. Gate Summary requests on settlement of the current list request and cap
   Alibaba Cloud (or unresolved-provider) list pages at 30 rows.
8. Run focused Go tests and Clinic UI lint, type-check, and build checks.

## Exclusions

- No global Kong timeout change.
- No MaxCompute engine polling or `Terminate` change.
- No table-manager, `MSCK REPAIR`, Delta catalog, partitioning, or storage
  layout change.
- No pagination-cursor redesign.
- No stats, trend, top-digest, detail, or export SQL change.

## Compatibility and impact

- Older clients that omit `fields` receive the existing response and SQL
  projection.
- AWS, GCP, Azure/Synapse, and other non-MaxCompute dialects retain the existing
  projection even when a new UI sends `fields`.
- The transformed list response keeps its existing shape; unselected raw
  values remain zero or null as they do when a column is absent.
- Summary values may appear later on Alibaba Cloud because they wait for the
  list request, but their calculations and endpoints are unchanged.
- The UI continues to support existing page sizes for known non-Alibaba
  providers. Alibaba Cloud list and Load More requests are capped at 30 rows to
  avoid the expensive immediate second scan.

## Test plan

From the Clinic repository root:

```text
go test ./pkg/api/slowlog_explorer
```

From the Clinic UI repository root:

```text
pnpm --filter @clinic/slow-log-explorer build
pnpm build
```

Focused assertions include:

- missing `fields` preserves the legacy projection;
- MaxCompute accepts, deduplicates, and narrows an allowlisted projection;
- non-MaxCompute ignores the optimization and preserves legacy SQL;
- invalid fields fail before any Data Proxy request;
- mandatory identity/cursor fields are always selected and query text is
  selected when the preview column is active;
- a hidden active sort column remains in the MaxCompute SELECT projection;
- SSE sends an initial comment and heartbeat without concurrent response writes;
- request cancellation stops waiting and does not write a stale result;
- the UI URL changes when visible columns change;
- Summary does not start before the list settles;
- Alibaba Cloud does not start a 50-row automatic request before provider
  metadata is known.

## Staging verification

1. Deploy the Clinic feature branch to staging with Clinic CD component
   `clinic` and deploy the matching Clinic UI build through its normal staging
   workflow.
2. Open a MaxCompute-backed Slow Log Explorer page and verify that the first
   list batch appears without a gateway timeout, requested `fields` are present,
   and Summary loads only after the list request settles.
3. Verify an AWS or GCP page still uses its existing page size and behavior.
4. If approved access is available, confirm the Clinic Pod image, readiness,
   restart count, and relevant gateway/application logs. Record skipped checks
   explicitly.

## Release boundary

Affected source module: Clinic root Go module plus the separate Clinic UI
package. The backend produces the `clinic` image and requires Clinic CD
`components=clinic`, app-delivery application `clinic-api`, and the Clinic
Deployment. The UI requires its own normal Clinic UI image/release path. No
`clinic-data-service`, Kong, NGM, internal-api, task-manager, uploader, or
app-delivery configuration deployment is required.

Merging either source PR, deploying staging, and releasing production each
remain separate authorization and review boundaries.