# Object Storage Metrics

`dataplane-manager` exports object storage usage metrics from the existing controller-runtime metrics endpoint. The first implementation reads AWS S3 Storage Lens CSV exports and enriches bucket metrics from regional Crossplane bucket CRs.

## Runtime model

The collector runs in the dataplane-manager controller process:

```text
S3 Storage Lens report bucket
        |
        v
AWS report collector background refresh
        |
        v
cached snapshot + Crossplane enrichment
        |
        v
dataplane-manager /metrics
```

Prometheus scrapes the existing dataplane-manager metrics endpoint configured by `--metrics-bind-address`; there is no standalone exporter image, service, or release process.

## Configuration

Configure report sources in the dataplane-manager root config:

```yaml
objectStorageMetrics:
  refreshInterval: 1h
  awsStorageLensReports:
    - name: dev-us-west-2
      region: us-west-2
      accountID: "790636686964"
      configID: tidb-cluster-bucket
      reportBucket: s3-lens-790636686964-us-west-2-an
      reportPrefix: StorageLens/
      roleARN: arn:aws:iam::790636686964:role/ObjectStorageMetricsExporterReadRole
```

If `awsStorageLensReports` is empty, no object storage metrics collector starts. `reportPrefix` defaults to `StorageLens/` when omitted. `refreshInterval` defaults to `1h` when omitted.

`configID` is the AWS S3 Storage Lens configuration ID. The collector uses it to discover manifests under `<reportPrefix>/<accountID>/<configID>/...` and validates it against `manifest.configId` to avoid reading the wrong report stream when a report bucket contains multiple Storage Lens exports.

For deployment, prefer S3 account regional namespace buckets such as `s3-lens-<account-id>-<region>-an` for Storage Lens report buckets. For multiple AWS accounts, configure one report source per account and use `roleARN` to assume an account-local read role before reading that account's report bucket.

## Permissions

The dataplane-manager runtime role needs `sts:AssumeRole` for each configured `roleARN`. Each assumed role needs read-only access to the report bucket:

- `s3:ListBucket` on the report bucket
- `s3:GetObject` on report objects

The process also lists Crossplane `buckets.s3.aws.upbound.io` CRs for ownership enrichment. Regional dataplane-manager RBAC already grants broad access to AWS Crossplane resources; if this is narrowed later, keep `list` access for `buckets.s3.aws.upbound.io`.

## Metrics

Implemented metrics:

```text
object_storage_bucket_size_bytes
object_storage_object_count
object_storage_storage_class_size_bytes
object_storage_report_last_success_timestamp_seconds
object_storage_report_age_seconds
object_storage_collector_refresh_errors_total
```

Common labels:

```text
provider
region
account_id
bucket
storage_class      # only for object_storage_storage_class_size_bytes
tenant_id
shared_pool_id
```

Report-level metrics (`object_storage_report_last_success_timestamp_seconds` and `object_storage_report_age_seconds`) are labeled by `provider`, `region`, `account_id`, and `collector` so multiple Storage Lens report sources from the same account do not produce duplicate Prometheus label sets. The collector label is `aws-storage-lens-report/<name>` when a source name is configured, otherwise it falls back to `aws-storage-lens-report/<accountID>/<region>/<configID>`.

Crossplane enrichment always runs for configured report sources. It joins Storage Lens `bucket_name` to `metadata.annotations["crossplane.io/external-name"]`, falling back to `metadata.name`, and reads:

- `tenant_id` from `metadata.labels["tags.tidbcloud.com/tenant"]`
- `shared_pool_id` from `metadata.labels["tags.tidbcloud.com/shared-pool"]`

If labels are missing, it falls back to `spec.forProvider.tags["tenant"]` and `spec.forProvider.tags["shared-pool"]`. Buckets missing from Crossplane are expected for unmanaged buckets; metrics are emitted with empty enrichment labels and the collector logs a warning without failing the refresh.

## AWS Storage Lens granularity

Storage Lens `storage_class` is coarser than the S3 CloudWatch `BucketSizeBytes` `StorageType` dimension. For example, Storage Lens can report `INTELLIGENT_TIERING`, while CloudWatch may split usage into detailed storage types such as `IntelligentTieringFAStorage`, `IntelligentTieringIAStorage`, or `StandardStorage`.

This collector only exposes Storage Lens CSV `storage_class` values. If dashboards need detailed S3 `StorageType` breakdowns, add a separate CloudWatch-backed collector and metric rather than overloading the Storage Lens storage-class metric.