# L0 Neutral Metadata IR ## Abstract NexusCRM is metadata-driven: application objects, fields, and automation are described as metadata and then interpreted by runtime services. Analysis code such as validators and planners needs to inspect that metadata, but it should not have to depend on the runtime Go structs or caches used to execute it. The neutral metadata intermediate representation, or IR, is the small data format used for that analysis boundary. Each IR node describes one authored metadata fact by recording four pieces of information: what the fact is, who authored it, where it came from in source metadata, and which source-to-IR translation produced it. An IR document gives generic analysis code a source-backed index of authored facts: object and field facts from pack components, and executable Flow element facts from Flow metadata. The IR is checked for ownership, provenance, duplicate identities, and valid lowering boundaries. It is not a replacement metadata store and does not change the runtime read path. ## 1. Introduction Metadata in NexusCRM is authored outside the generic engines. L1 product packs define product shape such as objects, fields, layouts, and Flows. L2 tenant overlays can add tenant-specific shape. L0 is the application-neutral platform: metadata services, validators, planners, records, permissions, automation, and transport mechanisms. L0 must be able to inspect metadata facts without hardcoding product objects or tenant rules. A metadata fact is one authored statement about platform shape or behavior. The statement "there is an object named `WorkItem`" is a fact. The statement "that object has field `Status`" is another fact. Runtime services need full metadata payloads to execute behavior, but validators and planners often need only a smaller view: the fact identity, author, source coordinate, and lowering edge. Using runtime structs directly as the analysis contract would confuse source authority. A Go struct such as `models.ObjectMetadata` is a loaded projection of authored source metadata. It is useful input to a lowerer, but it is not the source component itself. The IR keeps those roles separate: source metadata is the authority, runtime structs are projections, and lowerers produce neutral IR headers for analysis. The rest of this article is structured as follows. Section 2 follows one object component from authored source metadata through the runtime projection, object lowerer, generated IR document, node header semantics, validation, and identity projection. Section 3 applies the same IR model to executable Flow elements. Section 4 describes validation failures, the runtime boundary, the generated example test, and the implementation points that define the contract. ## 2. Object Metadata Lowering This section follows an object component end to end. The example starts with the authored source component, shows the runtime projection passed to the lowerer, and then reads the generated IR document as a set of source-backed metadata facts. ### 2.1 Source Metadata Start with an L1 pack named `toy-workflow-pack`. It defines an object `WorkItem` with one field, `Status`. The authored pack component is identified by this source URI: ```text pack://toy-workflow/objects/WorkItem.json ``` A source URI is a logical identifier for an authored component. In this example it names the `WorkItem` object component in the `toy-workflow` source namespace. The component payload has this shape: ```json [ { "apiName": "WorkItem", "label": "Work Item", "pluralLabel": "Work Items", "tableType": "standard_object", "sharingModel": "Private", "enableHierarchySharing": false, "fields": [ { "apiName": "Status", "label": "Status", "type": "string" } ] } ] ``` The field has a component-local source path: ```text /fields/Status ``` The path is not a runtime lookup path. It is the coordinate of the authored field fact inside the object component. ### 2.2 Runtime Projection And Lowerer Input The object metadata source is loaded into a typed projection for runtime use. For this example, the lowerer sees this projection: ```go schema := &models.ObjectMetadata{ APIName: "WorkItem", Fields: []models.FieldMetadata{ {APIName: "Status"}, }, } ``` The lowerer also receives explicit source coordinates: ```go source := metadata.ObjectComponentIRSource{ Owner: metadatair.Owner{ Layer: metadatair.OwnerLayerL1, OwnerPack: "toy-workflow-pack", }, URI: "pack://toy-workflow/objects/WorkItem.json", Component: "objects/WorkItem", Dialect: "toy", } ``` `Owner.Layer` records the authoring layer. L0 facts have no owner pack. L1 and L2 facts must name the pack or overlay that authored them. Here the owner is the L1 pack `toy-workflow-pack`. The lowerer combines the runtime projection with the source coordinates: ```go doc, err := metadata.LowerObjectComponentHeaders(schema, source) ``` The projection supplies typed object and field names. The source coordinates remain explicit so the generated IR can point back to the authored component. ### 2.3 Generated IR Document An IR document is a versioned collection of node headers. A node header records one metadata fact. The object lowerer emits one node for the object and one node for the field: | Kind | ID | Owner | Source path | |---|---|---|---| | `object` | `WorkItem` | `L1:toy-workflow-pack` | `/` | | `field` | `WorkItem.Status` | `L1:toy-workflow-pack` | `/fields/Status` | The object node represents the object component root. The field node represents the field fact inside that object. The semantic identity of a node is the pair `(Kind, ID)`, so `object:WorkItem` and `field:WorkItem.Status` are different facts even though both come from the same component. For a generic verifier, the field row is sufficient to answer four questions: - What is the fact? `field:WorkItem.Status`. - Who authored it? `L1:toy-workflow-pack`. - Where did it come from? `pack://toy-workflow/objects/WorkItem.json` at `/fields/Status`. - Which lowering produced it? `pack_component_json` to `neutral_ir`. ### 2.4 One Node Header The generated field node expands to this header-level view: | Header field | Generated value | |---|---| | ID | `WorkItem.Status` | | Kind | `field` | | Owner | `L1:toy-workflow-pack` | | Source format | `pack_component_json` | | Source URI | `pack://toy-workflow/objects/WorkItem.json` | | Source component | `objects/WorkItem` | | Source path | `/fields/Status` | | Dialect | `toy` | | Lowering boundary | `pack_component_json -> neutral_ir` | The fields have distinct roles. `ID` and `Kind` define the fact identity. `Owner` records who authored the fact. `Source format`, `Source URI`, `Source component`, and `Source path` identify the authored source fact. `Dialect` is inert provenance; generic engines must not branch on it. The lowering boundary records the source-to-IR edge. The current target surface is `neutral_ir`. The source format must match the boundary source. A node that claims to be sourced from `pack_component_json` but has a `flow_metadata_json -> neutral_ir` lowering boundary is invalid. ### 2.5 Validation And Projection Document validation is fail-closed. A document is invalid if any node has an empty ID, an unknown kind, an invalid owner, an incomplete source coordinate, an invalid lowering boundary, a source format that does not match the boundary source, or a duplicate `(Kind, ID)`. The same IR document can also project back to the runtime-facing identity shape: ```go metadata.ObjectFieldIdentityProjection{ ObjectAPIName: "WorkItem", FieldAPINames: []string{"Status"}, } ``` This projection enables a shadow check. A shadow check compares the identity seen through IR with the identity seen through the current runtime projection. It is diagnostic only. It does not publish IR into `MetadataService`, persist IR as runtime metadata, or switch a runtime read path. ### 2.6 Object Components With Multiple Fields The same object lowerer scales from one field to multiple fields. With fields `Summary` and `Status`, the typed projection is: ```go schema := &models.ObjectMetadata{ APIName: "WorkItem", Fields: []models.FieldMetadata{ {APIName: "Summary"}, {APIName: "Status"}, }, } ``` The generated nodes are: | Kind | ID | Source format | Source path | |---|---|---|---| | `object` | `WorkItem` | `pack_component_json` | `/` | | `field` | `WorkItem.Summary` | `pack_component_json` | `/fields/Summary` | | `field` | `WorkItem.Status` | `pack_component_json` | `/fields/Status` | Object and field IDs follow these forms: ```text object: field:. ``` Field source paths follow this component-local form: ```text /fields/ ``` If a field API name contains `~` or `/`, the path segment is escaped as `~0` or `~1`. The generated projection back to runtime identity is: ```go metadata.ObjectFieldIdentityProjection{ ObjectAPIName: "WorkItem", FieldAPINames: []string{"Summary", "Status"}, } ``` The object projection accepts only object and field nodes. It requires exactly one object node, requires `pack_component_json` source format, and requires each field ID to be under the object prefix. ## 3. Flow Metadata Lowering Object metadata is not the only source surface lowered to IR. The same header model also represents executable Flow elements, with a Flow-specific source format and identity scheme. ### 3.1 Flow Elements Flow metadata uses the same IR model with a different source format and ID scheme. The Flow lowerer receives Flow definition/version context plus source coordinates for the Flow metadata JSON payload: ```go definition := &models.FlowDefinition{ Id: "fd-workflow", DeveloperName: "FulfillmentFlow", OwnerPack: "toy-workflow-pack", } version := &models.FlowVersion{ DefinitionId: "fd-workflow", VersionNumber: 3, Metadata: models.FlowMetadata{ Decisions: []models.FlowDecision{{Name: "CheckStatus"}}, Assignments: []models.FlowAssignment{{Name: "SetStatus"}}, ActionCalls: []models.FlowActionCall{{Name: "NotifyOwner"}}, }, } source := metadata.FlowComponentIRSource{ Owner: metadatair.Owner{ Layer: metadatair.OwnerLayerL1, OwnerPack: "toy-workflow-pack", }, URI: "pack://toy-workflow/flows/FulfillmentFlow.flow-meta.json", Component: "flows/FulfillmentFlow", Dialect: "toy", } doc, err := metadata.LowerFlowComponentHeaders(definition, version, source) ``` The Flow version identity is `FulfillmentFlow.v3`. The generated IR nodes are: | Kind | ID | Source format | Source path | |---|---|---|---| | `flow_element` | `FulfillmentFlow.v3.CheckStatus` | `flow_metadata_json` | `/decisions/0` | | `flow_element` | `FulfillmentFlow.v3.SetStatus` | `flow_metadata_json` | `/assignments/0` | | `flow_element` | `FulfillmentFlow.v3.NotifyOwner` | `flow_metadata_json` | `/actionCalls/0` | Flow element IDs follow this form: ```text flow_element:.v. ``` Flow element source paths follow this form: ```text // ``` The Flow lowerer emits executable named elements from these sections: - `decisions` - `assignments` - `recordCreates` - `recordUpdates` - `recordDeletes` - `recordLookups` - `actionCalls` - `subflows` - `loops` - `customErrors` - `waits` `FlowDefinition` and `FlowVersion` rows provide identity context. They are not emitted as `flow_metadata_json` nodes because their facts are not sourced from the Flow metadata JSON payload. Start metadata and screens are also not emitted by this lowerer; it emits executable named elements only. The generated projection back to runtime identity is: ```go metadata.FlowElementIdentityProjection{ VersionNodeID: "FulfillmentFlow.v3", ElementNames: []string{"CheckStatus", "SetStatus", "NotifyOwner"}, } ``` The Flow projection accepts only `flow_element` nodes, requires `flow_metadata_json` source format, and requires each element ID to be under the supplied Flow version prefix. ## 4. Verification, Runtime Boundary, And References The IR is useful only if its facts are valid and if its boundary with runtime metadata remains explicit. This section describes the fail-closed behavior, runtime non-ownership, generated example check, and implementation references. ### 4.1 Failure Modes The same validation rules apply to object and Flow IR. The generated failure examples are: ```text duplicate object field: metadata IR duplicate node identity field:WorkItem.Status duplicate flow element: metadata IR duplicate node identity flow_element:FulfillmentFlow.v2.SharedElement missing L1 owner pack: metadata IR node object:WorkItem owner invalid: metadata IR L1 owner requires an owner pack ``` These failures are intentional. Duplicate identity would create competing facts with the same read identity. A missing L1 owner pack would make the authoring surface ambiguous. A mismatched source format and lowering boundary would make a node claim the wrong source-to-IR edge. ### 4.2 Runtime Boundary The IR is not the active runtime read authority. The current runtime read paths remain: - `MetadataService` loads and serves object, field, relationship, validation, auto-number, identifier, and Flow definition data from existing repositories and cache snapshots. - Flow execution reads `models.FlowMetadata` from `FlowVersion` metadata JSON. - Authorization, query, automation, connector, and API handlers do not read IR documents as runtime metadata. The IR exists beside those paths as an analysis contract. It gives verifier and planner code source-backed identity and provenance without creating another runtime metadata store. ### 4.3 Reproducing The Generated Blocks The output tables, projections, and failure block in this report are generated from live lowerer code. Run this command to update them: ```bash (cd nexuscrm-metadata && UPDATE_METADATAIR_DOC=1 go test . -run TestNeutralIRDocExamplesMatchDocument -count=1) ``` Run this command to verify the report without modifying it: ```bash (cd nexuscrm-metadata && go test . -run TestNeutralIRDocExamplesMatchDocument -count=1) ``` The test calls the live lowerers and projection helpers, then compares the generated markdown blocks with the blocks checked into this report. ### 4.4 Implementation References - Contract: [github.com/pingcap-inc/nexuscrm-contract/pkg/metadatair](https://github.com/pingcap-inc/nexuscrm-contract/tree/v0.1.2/pkg/metadatair) - Doc example generator/check: [nexuscrm-metadata/metadatair_doc_examples_test.go](../../nexuscrm-metadata/metadatair_doc_examples_test.go) - Object/field lowerer: [nexuscrm-metadata/metadatair_object_lowering.go](../../nexuscrm-metadata/metadatair_object_lowering.go) - Object/field examples: [nexuscrm-metadata/metadatair_object_lowering_test.go](../../nexuscrm-metadata/metadatair_object_lowering_test.go) - Flow element lowerer: [nexuscrm-metadata/metadatair_flow_lowering.go](../../nexuscrm-metadata/metadatair_flow_lowering.go) - Flow element examples: [nexuscrm-metadata/metadatair_flow_lowering_test.go](../../nexuscrm-metadata/metadatair_flow_lowering_test.go) - Shadow checks: [nexuscrm-metadata/metadatair_shadow.go](../../nexuscrm-metadata/metadatair_shadow.go) - Runtime metadata boundary: [nexuscrm-metadata/metadata_service.go](../../nexuscrm-metadata/metadata_service.go) - Runtime Flow boundary: [nexuscrm-automation/flow_executor.go](../../nexuscrm-automation/flow_executor.go)