# Layout System Redesign — Implementation & Test Plan

**Date**: 2026-03-31
**Design Doc**: `docs/design/layout-system-research.md`
**PR**: #562

---

## Implementation Plan: Phase 1 Foundation

### Overview

6 Work Packages, 16 files, estimated 6-8 days.

```
WP1 Schema+Models (0.5d)
  → WP2 Repository (1.5d)
    → WP3 Service (2-3d) ←── role_hierarchy.go export
      → WP4 API (1.5d)
      → WP5 Bootstrap (0.5d)
      → WP6 Bug fixes (0.5d, parallel)
```

### WP1: Schema + Models + Constants (0.5 day)

| File | Change | Lines |
|------|--------|-------|
| `backend/internal/bootstrap/system_tables.json` | Add `_System_LayoutAssignment`, `_System_PageLayoutHistory`, `_System_Migration` tables. Add `Source`, `GenerationVersion` columns to `_System_PageLayout`. KEEP `IsCustomized`/`LayoutVersion` for backward compat. | +120 |
| `shared/pkg/models/metadata.go` | Add `Source string`, `GenerationVersion int` to `PageLayout`. Add `LayoutAssignment` and `PageLayoutHistoryEntry` structs. | +35 |
| Run `go run backend/cmd/codegen/main.go` | Regenerates `z_generated_tables.go`, `z_generated_fields.go`, `z_generated_columns.go` | +80 auto |

**Checkpoint**: `go build ./...` passes. Start server once → new tables created, new columns added.

### WP2: Repository Layer (1-1.5 days)

| File | Change | Lines |
|------|--------|-------|
| `backend/internal/domain/ports/repositories.go` | Add to `MetadataRepository` interface: `FindBestAssignment`, `CreateAssignment`, `DeleteAssignment`, `GetAssignmentsForObject`, `DeleteAssignmentsByLayoutID`, `WriteLayoutHistory`, `GetMigrationVersion`, `SetMigrationVersion` | +15 |
| `backend/internal/infrastructure/persistence/repo_layout.go` | Implement all new methods. Fix `SaveLayout` ON DUPLICATE KEY to include `IsDefault`/`IsDeleted`/`Source`/`GenerationVersion`. `FindBestAssignment` has two paths: with-role (IN clause) and no-role (`AND RoleId IS NULL`). | +180, ~10 mod |
| `backend/internal/infrastructure/persistence/metadata_scan_ui.go` | Update `scanLayout()` to scan `Source` + `GenerationVersion`. Add `scanAssignment()`. | +40 |

**Checkpoint**: `go build ./...` + `go test ./backend/internal/infrastructure/persistence/...`

### WP3: Service Layer (2-3 days) — highest risk

| File | Change | Lines |
|------|--------|-------|
| `backend/internal/features/permissions/role_hierarchy.go` | Export `getRoleAncestors` → `GetRoleAncestors` | ~5 |
| `backend/internal/features/metadata/metadata_layouts.go` | **Replace `GetLayout()` with `ResolveLayout()`**: single-query assignment lookup → fetch layout → stale version check → auto-generate fallback. Keep `GetLayout()` as thin wrapper for backward compat. Rename `currentLayoutVersion` → `currentGenerationVersion`. Fix `addFieldToLayout()` to iterate ALL layouts. Add `MigrateLayoutSystem()` function. | +200, -30 |
| `backend/internal/features/uiapi/primitives.go` | Update `GetLayout()` to compute role ancestors and pass to `ResolveLayout()`. Update cache key. | ~20 |
| `backend/internal/features/uiapi/service.go` | Add `AssignLayout()`, `DeleteAssignment()`, `GetAssignments()`. Deprecate `MarkLayoutCustomized`. | +40 |

**Checkpoint**: `go build ./...` + `go test ./backend/internal/features/metadata/...` + start server, verify `GET /ui-api/layout/Account` still works.

### WP4: API Layer (1-1.5 days)

| File | Change | Lines |
|------|--------|-------|
| `backend/internal/interfaces/rest/sobject_describe.go` | Fix EC-6: pass `recordTypeId` param to `ResolveLayout()`. Remove dead fallback at lines 414-462. Update `DescribeLayouts` to query real per-RT layout IDs instead of hardcoded string. | ~40 mod, -50 del |
| `backend/internal/interfaces/rest/ui_api_handler.go` | `HandleSaveLayout`: set `Source="admin_created"`, write history, add optimistic concurrency (EC-4). `HandleDeleteLayout`: cascade-delete assignments (EC-10), protect default (Decision 18). | ~30 |
| `backend/internal/interfaces/rest/tooling_metadata_handler.go` | `CreateProfileLayout`: write to `_System_LayoutAssignment` instead of `ProfileLayout`. | ~20 |

**Checkpoint**: `go test ./backend/internal/interfaces/rest/...` + integration test all layout endpoints.

### WP5: Bootstrap Cleanup (0.5 day)

| File | Change | Lines |
|------|--------|-------|
| `backend/cmd/server/main.go` | Remove `CleanupHardcodedLayouts()` call. Add `MigrateLayoutSystem()` after `InitializeSchema()`. | ~5 |
| `backend/internal/bootstrap/layouts.go` | Delete `CleanupHardcodedLayouts()` function. Keep `CleanupSystemListFields()`. | -15 |
| `backend/internal/bootstrap/standard_layouts.json` | DELETE file (extract section templates for Phase 2 first). | -500 |

**Checkpoint**: Full server restart against dev DB. Migration runs. All endpoints work.

### WP6: Bug Fixes (0.5 day, parallel with WP4/WP5)

| Fix | File | Change |
|-----|------|--------|
| EC-8: Ghost fields | `primitives.go` transformLayout | Filter fields not in schema | +5 |
| EC-7: Malformed JSON logging | `metadata_scan_ui.go` scanLayout | ERROR log with layout ID | ~3 |
| EC-9: Cache invalidation | `service.go` or `primitives.go` | Evict layoutCache in SaveLayout | ~5 |

---

## Test Plan

### Summary: 28 tests across 4 categories

| Category | Count | Priority | Requires |
|----------|-------|----------|----------|
| Unit: ResolveLayout core | 9 | P0 | Mock repo only |
| Unit: addFieldToLayout + generateSmartLayout | 4 | P2 | Mock repo only |
| Integration: DB operations | 8 | P1 | TiDB via .env |
| Contract: API compatibility | 4 | P3 | Running server |
| Migration | 4 | P1 | TiDB via .env |
| **Total** | **28** | | |

### Unit Tests — `metadata_layout_resolve_test.go`

| # | Test | What it verifies | Priority |
|---|------|------------------|----------|
| 1.1 | `TestResolveLayout_NoAssignments_AutoGenerates` | Zero assignments → auto-generate layout + create specificity-0 assignment | P0 |
| 1.2 | `TestResolveLayout_SingleSpec0_ReturnsLayout` | Single global default returns correct layout | P0 |
| 1.3 | `TestResolveLayout_MultipleSpecificities_HighestWins` | Spec 0, 1, 2 → spec 2 wins | P0 |
| 1.4 | `TestResolveLayout_SameSpecificity_TieBreaking` | RT > Profile > Role > CreatedDate | P0 |
| 1.5 | `TestResolveLayout_UserNoRole_WildcardOnly` | No Role → `AND RoleId IS NULL` → only wildcard matches | P0 |
| 1.6 | `TestResolveLayout_DeletedLayout_FallsBack` | EC-1: assignment → deleted layout → auto-generate | P0 |
| 1.7 | `TestResolveLayout_SchemaNotFound_ReturnsNil` | EC-2: non-existent object → nil, no panic | P0 |
| 1.8 | `TestResolveLayout_StaleAutoGen_Regenerates` | Source=auto_generated, version stale → in-place UPDATE | P0 |
| 1.9 | `TestResolveLayout_AdminCreated_NeverRegenerated` | Source=admin_created → never touched | P0 |
| 1.10 | `TestAddFieldToLayout_UpdatesAllLayouts` | 3 layouts → all 3 get new field | P2 |
| 1.11 | `TestGenerateSmartLayout_StandardObject_Sections` | Account → "Account Information" not "Information" | P2 |
| 1.12 | `TestGenerateSmartLayout_CustomObject_GenericSections` | custom__c → 5 generic sections | P2 |
| 1.13 | `TestGenerateSmartLayout_CompactLayout` | Up to 7 fields, name first, priority order | P2 |

### Integration Tests — `metadata_layout_integration_test.go`

Build tag: `//go:build integration`. Uses `TIDB_*` env vars from `.env`.

| # | Test | What it verifies | Priority |
|---|------|------------------|----------|
| 2.1 | `TestInteg_ResolveLayout_CreatesAndFinds` | Full path: insert layout + assignment, resolve, verify | P1 |
| 2.2 | `TestInteg_ResolveLayout_HighestSpecWins` | 3 specificities in real DB, correct winner | P1 |
| 2.3 | `TestInteg_ResolveLayout_RoleHierarchy` | Ancestor traversal matches APAC VP assignment | P2 |
| 2.4 | `TestInteg_Migration_ExistingLayouts` | 8 layouts get Source/GenerationVersion + assignments | P1 |
| 2.5 | `TestInteg_Migration_Idempotent` | Run twice, same result | P1 |
| 2.6 | `TestInteg_UniqueConstraint_RejectsDuplicate` | Same dimensions → duplicate key error | P1 |
| 2.7 | `TestInteg_CascadeDelete_LayoutToAssignment` | Hard-delete layout → assignments gone | P1 |
| 2.8 | `TestInteg_GeneratedColumns_Computed` | Specificity + sentinel columns auto-computed by TiDB | P1 |

### Contract Tests — `tests/contract/`

Require running server against the TiDB database.

| # | Test | What it verifies | Priority |
|---|------|------------------|----------|
| 3.1 | `TestContract_DescribeLayouts_RecordTypeMappings` | `recordTypeMappings[]` with per-RT layout IDs | P3 |
| 3.2 | `TestContract_DescribeLayoutByRT_FullBody` | rtId used in resolution, full layout body returned | P3 |
| 3.3 | `TestContract_UIAPILayout_UserContext` | Layout resolved for current user's context | P3 |
| 3.4 | `TestContract_RecordUI_LayoutEmbedded` | Layout in aggregated record-ui response | P3 |

### Migration Tests — `metadata_migration_test.go`

Build tag: `//go:build integration`.

| # | Test | What it verifies | Priority |
|---|------|------------------|----------|
| 4.1 | `TestInteg_Migration_FreshDB_NoOp` | Empty DB: migration records version=1, no data changes | P1 |
| 4.2 | `TestInteg_Migration_Existing8_FullMigration` | 8 layouts → all get Source + assignments | P1 |
| 4.3 | `TestInteg_Migration_RunTwice_Idempotent` | Second run is no-op | P1 |
| 4.4 | `TestInteg_Migration_ValidationQueries` | All 4 post-migration validation queries pass | P1 |

### Test Execution

```bash
# Unit tests (fast, no DB)
cd backend && go test -v -run TestResolveLayout ./internal/features/metadata/
cd backend && go test -v -run TestAddFieldToLayout ./internal/features/metadata/
cd backend && go test -v -run TestGenerateSmartLayout ./internal/features/metadata/

# Integration tests (requires .env with TiDB connection)
source .env
cd backend && go test -v -tags=integration -run TestInteg ./internal/features/metadata/

# Contract tests (requires running server)
cd tests/contract && go test -v -run TestContract ./rest_api/sobjects/ ./ui_api/records/

# All tests
cd backend && make test-unit && make test-integration
```