"""Azure OpenAI / Microsoft Foundry auto-grant — called from the API key Feishu callback. Mirrors the AWS Bedrock + Aliyun Bailian autogrant modules but provisions inside Microsoft Azure on the **Foundry** surface. When an :class:`ApiKeyAssignment` whose provider name matches Azure OpenAI (``azure openai``, ``azure-openai``, ``aoai``, normalised for whitespace quirks — see :func:`normalize_provider_match_name`) transitions to ``approved`` via the Feishu callback, we: 1. Resolve the **target subscription** via platform_setting ``cloud.azure_openai.default_subscription_id`` (defaults to the PingCAP "Azure OpenAI" subscription ``7bc8ab17-59fb-44c1-92ca-61fd2b814735``). Joins the local ``cmdb_cloud_scopes`` row of type ``azure_subscription`` with that ``external_id`` to discover the owning ``Integration`` (the SP credential bundle). 2. **Mint a dedicated Resource Group** under that subscription. RG names are scoped per-subscription so we use a deterministic pattern: ``helix-aoai-{request_suffix}``. Tagged with ``helix-*`` keys so Azure Cost Management queries can drill in by RG → applicant. 3. **Create a Microsoft Foundry resource** — a Cognitive Services account of ``kind=AIServices`` with ``allowProjectManagement=True`` so child Foundry projects can be nested. ``S0`` SKU, ``customSubDomainName`` set to a globally-unique account name, SystemAssigned identity. Polls until ``provisioningState=Succeeded`` so the next project / listKeys call doesn't 409. 4. **Create a Foundry project** under the AIServices account (``Microsoft.CognitiveServices/accounts/{name}/projects/{project}``) with displayName + description — this is what end users actually work against in the Foundry portal. Polls until Succeeded. 5. **Read KEY1** from the parent account (Foundry projects share keys with the account — the project name is part of the URL, not the auth material), store the credential bundle on the assignment. 6. **Mirror the RG into ``cmdb_cloud_scopes``** as a new ``azure_resource_group``-typed row whose ``parent_scope_id`` points at the subscription scope. AI cost classification is now service-name-based (PR #253) so the RG just needs to exist and AI-spending dashboard sums it natively. ``owner_person_id=`` the applicant — the RG belongs to them, no separate handover. 7. **Bind ``cloud_scope_id``** on the assignment to the RG scope so the admin "rebind" UI shows the binding without parsing JSON. 8. **Auto-create a FinOps allocation rule** keyed on ``match_integration_ids=[]`` AND ``match_resource_group=`` → ``assign_person_id=``, priority 10 (high — RG-targeted rules beat default catch-alls). The cost ingest cron stamps every line item with the applicant from the next sync onward. This replaces an earlier ``kind=OpenAI`` flow (the legacy "Azure OpenAI Service" account type). Microsoft is consolidating Azure OpenAI under the Foundry surface — the portal's "Create a project" wizard creates exactly the RG + AIServices account + project shape we mint here. SP role requirements (admin-side prep, set on the standing PingCAP Azure OpenAI subscription via Azure Portal → Access control → Add role assignment): * ``Cognitive Services Contributor`` at the subscription scope — creates AIServices accounts + Foundry projects + reads keys. * ``Contributor`` at the subscription scope — creates resource groups. If either is missing, the autogrant captures the 403 verbatim in ``detail.azure_openai.error`` and the assignment stays in ``approved`` with an empty ``key_value`` so an admin can fill it manually via ``/admin/assets/api-keys``. Errors leave the assignment in ``approved`` status with a note in ``detail.azure_openai`` so an admin can still fill the key manually. """ from __future__ import annotations import logging import re import uuid from datetime import UTC, datetime from typing import Any from sqlalchemy import select from sqlalchemy.ext.asyncio import AsyncSession from app.domains.apikey.autogrant_alerts import record_autogrant_failure from app.domains.apikey.models import ( ApiKeyAssignment, ApiKeyProvider, normalize_provider_match_name, ) from app.domains.apikey.service import _encrypt_key_value from app.domains.cmdb.credential_crypto import decrypt_fields from app.domains.cmdb.models import ( SCOPE_TYPE_AZURE_RESOURCE_GROUP, SCOPE_TYPE_AZURE_SUBSCRIPTION, CmdbCloudScope, CmdbCostAllocationRule, ) from app.identity.person_models import IdentityPerson from app.integrations.drivers.azure_cloud import AzureCloudDriver from app.integrations.models import Integration, PlatformSetting from app.notifications.service import NotificationEnvelope, NotificationService logger = logging.getLogger(__name__) # Default target subscription when the platform_setting override isn't # set. PingCAP's standing "Azure OpenAI" subscription — every approved # AI API key request lands here so finance has one billing surface. # Override via the ``cloud.azure_openai.default_subscription_id`` # platform_setting if a future split (e.g. APAC vs US) is needed. _DEFAULT_SUBSCRIPTION_ID = "7bc8ab17-59fb-44c1-92ca-61fd2b814735" # Default region for AOAI accounts. ``eastus`` has the broadest model # availability + GPT-4 / GPT-4o quota historically. Override via the # ``cloud.azure_openai.default_location`` platform_setting (e.g. # ``japaneast`` for APAC latency). _DEFAULT_LOCATION = "eastus" def _is_azure_openai_provider(provider: ApiKeyProvider | None) -> bool: """Return True for any provider name pointing at Azure OpenAI. Matches case-insensitively against a whitespace-normalised version of the user-facing provider name — admins occasionally rename "Azure OpenAI" to "AOAI" / "Azure OpenAI Service" / etc., and prod incident AK-20260509-EF8A showed that a stray double space ("Azure OpenAI") silently bypassed the matcher and skipped RG auto-grant. ``normalize_provider_match_name`` collapses whitespace runs (incl. NBSP / full-width / tabs) before substring testing. """ if provider is None: return False name = normalize_provider_match_name(provider.name) return any( token in name for token in ("azure openai", "azure-openai", "aoai", "azureopenai") ) _NAME_MAX_LEN = 60 # AIServices 64; reserve 4 for safety / suffix _RG_MAX_LEN = 90 # Azure RG names allow up to 90 chars _PROJECT_MAX_LEN = 60 # Foundry project name 64; reserve 4 def _slugify_for_azure(text: str) -> str: """Produce a lowercase, hyphenated, ARM-name-safe slug. Cognitive Services account names match ``[a-zA-Z0-9-]{2,64}`` with no leading / trailing hyphen. Foundry project names match ``[a-zA-Z0-9][a-zA-Z0-9_.-]*`` (2..64). Resource Group names allow more characters but we keep the same sanitiser so the RG, account and project share a readable prefix. """ text = (text or "").strip().lower() # Replace any run of non-[a-z0-9] with a single hyphen. text = re.sub(r"[^a-z0-9]+", "-", text) # Collapse repeated hyphens, trim ends. text = re.sub(r"-+", "-", text).strip("-") return text def _build_resource_names( project_name: str, request_number: str, *, assignment_id: uuid.UUID, ) -> tuple[str, str, str]: """Build (rg_name, foundry_account_name, foundry_project_name). All three names share an ``{request_suffix}-{slug}`` shape so an admin glancing at the Azure portal sees the request number front- and-centre. The 8-hex tail on the **account** name is deterministically derived from ``assignment.id`` so that two concurrent Feishu-callback retries for the same assignment compute the same globally-unique subdomain — ARM PUT on an existing AIServices account is idempotent, so the second create returns the existing resource instead of minting a duplicate billable account (the prior random-suffix design defeated Azure's global-unique-name protection and caused a real cost-leak race). Mirrors Bedrock's deterministic IAM-username pattern. The 8 hex chars (32 bits) of ``assignment.id`` give roughly 4.3 bn distinct names per (project, date) pair — collision-resistance is bounded by ``assignment.id``'s own UUID4 entropy, which is fine. Foundry project names only need to be unique **within the parent account**, not globally — and we mint exactly one project per account per request — so the project name skips the assignment-id tail in favour of readability. ``proj-{request_suffix}-{slug}`` surfaces the request number in the Foundry portal's project picker. """ parts = request_number.split("-", 1) request_suffix = parts[1] if len(parts) > 1 else request_number request_suffix = _slugify_for_azure(request_suffix) or "req" slug = _slugify_for_azure(project_name) # Deterministic from assignment.id — same retry → same name. # ``str()`` + ``UUID()`` round-trip tolerates callers that pass a # str instead of uuid.UUID (test convenience; production callers # always pass the typed value). aid_tail = uuid.UUID(str(assignment_id)).hex[:8] rg_base = f"helix-aoai-{request_suffix}" if slug: rg_base = f"{rg_base}-{slug}" rg_name = rg_base[:_RG_MAX_LEN].rstrip("-") # Account name shares the RG prefix so the Azure portal groups # them visually; the assignment-id tail ensures a stable global # subdomain across retries. if slug: account_base = f"aoai-{request_suffix}-{slug}" else: account_base = f"aoai-{request_suffix}" # Reserve room for "-{aid_tail}" tail. reserved = 1 + len(aid_tail) account_base = account_base[: _NAME_MAX_LEN - reserved].rstrip("-") account_name = f"{account_base}-{aid_tail}" # Project name: scoped to the account, so no aid_tail needed. if slug: project_base = f"proj-{request_suffix}-{slug}" else: project_base = f"proj-{request_suffix}" foundry_project_name = project_base[:_PROJECT_MAX_LEN].rstrip("-") return rg_name, account_name, foundry_project_name def _foundry_project_endpoint(account_name: str, project_name: str) -> str: """Construct the Foundry project's inference endpoint URL. Microsoft's Foundry SDK + REST clients hit: ``https://{accountSubdomain}.services.ai.azure.com/api/projects/{projectName}`` where the subdomain is the parent account's ``properties.customSubDomainName`` (which we set equal to ``account_name``). Constructing this client-side is more reliable than parsing the project's ``properties.endpoints`` dict — the dict schema has changed across api-versions and not every key is the inference endpoint. """ return ( f"https://{account_name}.services.ai.azure.com" f"/api/projects/{project_name}" ) async def _resolve_target_subscription( db: AsyncSession, ) -> tuple[str, CmdbCloudScope, Integration] | None: """Resolve (subscription_id, sub_scope_row, owning_integration). Lookup chain: 1. ``cloud.azure_openai.default_subscription_id`` platform_setting (defaults to the PingCAP Azure OpenAI sub GUID). 2. Find the ``cmdb_cloud_scopes`` row of type ``azure_subscription`` whose ``external_id`` matches. 3. Load that scope's ``Integration`` — which carries the SP credentials needed for ARM calls. Returns None when the sub isn't synced yet (operator hasn't run "Sync scopes" on the Azure integration since adding the sub) so the caller surfaces a clear error. """ setting = await db.get( PlatformSetting, "cloud.azure_openai.default_subscription_id", ) sub_id_raw = ( str(setting.value).strip() if setting and setting.value else _DEFAULT_SUBSCRIPTION_ID ) # Azure subscription GUIDs are case-insensitive — both ``7BC8AB17`` # and ``7bc8ab17`` resolve to the same sub. The cloud-scope sync # stores them lowercase per Azure docs convention but a hand-set # platform_setting could be uppercase. Lowercase both sides so the # equality check is robust. sub_id = sub_id_raw.lower() # Prefer scope rows on enabled integrations. Multiple rows are # legitimately possible — the same sub GUID can be visible from # several Azure cloud integrations (multi-tenant SP, or two # tenants with overlapping reseller billing). ``scalar_one_or_none`` # would raise ``MultipleResultsFound`` and crash the autogrant. # Pick the most recently synced enabled-integration row instead. # # ``last_synced_at IS NOT NULL`` filter is important — MySQL/TiDB # sort NULLs FIRST in DESC order. A freshly-inserted scope row # that hasn't completed its first sync would otherwise outrank # every successfully-synced row. NULL means "we haven't observed # this scope yet" and is not a credible target for provisioning. rows = (await db.execute( select(CmdbCloudScope).where( CmdbCloudScope.scope_type == SCOPE_TYPE_AZURE_SUBSCRIPTION, CmdbCloudScope.external_id == sub_id, CmdbCloudScope.last_synced_at.is_not(None), ).order_by(CmdbCloudScope.last_synced_at.desc()) )).scalars().all() for candidate in rows: integ = await db.get(Integration, candidate.integration_id) if integ is None or not integ.enabled: continue if integ.provider != "azure": continue if len(rows) > 1: logger.info( "Azure OpenAI auto-grant: %d scope rows match sub %s; " "picked integration %s (most recently synced enabled " "azure integration).", len(rows), sub_id, integ.id, ) return sub_id, candidate, integ return None async def _resolve_default_location(db: AsyncSession) -> str: setting = await db.get( PlatformSetting, "cloud.azure_openai.default_location", ) if setting is not None and setting.value: return str(setting.value).strip() or _DEFAULT_LOCATION return _DEFAULT_LOCATION def _build_driver(integration: Integration) -> AzureCloudDriver: """Decrypt the SP credential bundle and instantiate the driver.""" secret_fields: dict[str, Any] = {} if integration.secrets_encrypted: secret_fields = decrypt_fields( integration.secrets_encrypted, integration.secrets_nonce, integration_id=str(integration.id), ) merged_config = { **{ k: v for k, v in (integration.config_public or {}).items() if v is not None }, **secret_fields, } return AzureCloudDriver( integration_id=str(integration.id), config=merged_config, ) async def maybe_auto_grant_azure_openai( db: AsyncSession, assignment: ApiKeyAssignment, ) -> dict[str, Any]: """Invoked when an assignment becomes ``approved``. Safe no-op when the provider isn't Azure OpenAI or when auto-grant has already run (idempotent on ``detail.azure_openai.granted`` / ``detail.azure_openai.in_flight``). Mirrors the Bedrock contract: an in-flight tombstone is committed before any ARM call so a mid- callback crash refuses to mint a second AOAI account. """ provider_row = await db.get(ApiKeyProvider, assignment.provider_id) if not _is_azure_openai_provider(provider_row): return {"status": "not-azure-openai"} detail = dict(assignment.detail or {}) info = dict(detail.get("azure_openai") or {}) if info.get("granted"): return {"status": "already-granted"} if info.get("in_flight"): return {"status": "in-flight-needs-manual-recovery"} resolved = await _resolve_target_subscription(db) if resolved is None: info["granted"] = False info["error"] = ( "No Azure subscription scope row found for the configured " "default sub (cloud.azure_openai.default_subscription_id). " "Run 'Sync scopes' on the Azure integration first, or set the " "platform_setting to a synced subscription." ) detail["azure_openai"] = info assignment.detail = detail # Durably persist the diagnostic so the operator sees "why # didn't autogrant fire?" on /admin/assets/api-keys without waiting # for the outer Feishu-callback transaction to finalise. Every # other failure branch in this module commits — keeping this # one symmetric prevents the "log + lose the diagnostic if # the outer txn rolls back" footgun. await db.commit() await db.refresh(assignment) logger.warning( "Azure OpenAI auto-grant skipped (no subscription scope) for %s", assignment.request_number, ) await record_autogrant_failure( db, assignment, kind='azure_openai', stage='no-subscription', error=str(info.get('error') or 'no-subscription'), provider_label=getattr(provider_row, 'name', None), ) return {"status": "no-subscription"} subscription_id, sub_scope, integration = resolved location = await _resolve_default_location(db) person = await db.get(IdentityPerson, assignment.assigned_to) email = (person.work_email or "") if person else "" rg_name, account_name, project_name = _build_resource_names( assignment.project_name or "", assignment.request_number, assignment_id=assignment.id, ) # Tags applied to RG, AIServices account AND Foundry project so # finance can filter by any dimension. ``Owner`` follows the Azure # / AWS cost-allocation convention (capitalised); ``helix-*`` keys # are for our chargeback rule + audit trace. project = (assignment.project_name or "").strip()[:256] # ``Owner`` tag is keyed on the applicant's work email — matches the # Bedrock convention ([bedrock_autogrant.py] uses ``email``) so a # cross-cloud Cost Management report grouping by ``Owner`` sees the # same per-employee bucket on AWS and Azure. Prior value (``project # or "helix"``) made AOAI chargeback look like a project-level # aggregate even though Bedrock kept per-person fidelity. The # ``helix-project-name`` tag retains the project label for AOAI- # specific drilldown. tags = { "helix-assignment-id": str(assignment.id), "helix-request-number": assignment.request_number, "helix-owner-email": email or "unknown", "helix-project-name": project, "Owner": email or "unknown", } # Tombstone before any ARM call. A mid-call **process crash** leaves # the assignment with ``in_flight=true`` and the next callback # short-circuits with ``in-flight-needs-manual-recovery`` — no risk # of minting a second AIServices account / Foundry project. # # IMPORTANT: every non-crash failure branch below CLEARS # ``in_flight`` before returning. The deterministic-naming contract # (``_build_resource_names`` derives RG / account / project names # from ``assignment.id`` + ``request_number``, never random) makes # ARM PUT idempotent — a retry with the same names returns the # existing resources rather than minting duplicates. Without this # clear-on-error rule, a transient 503 from Cognitive Services in # a busy region permanently wedges the assignment because the # next callback hits the in-flight short-circuit. The "needs # manual recovery" path is reserved for the genuinely-stuck case # of a mid-call process crash, where the autogrant never reached # the error-write block (``detail.azure_openai.error`` stays # unset, so admins can distinguish the two paths). info["in_flight"] = True info["subscription_id"] = subscription_id info["resource_group_attempted"] = rg_name info["foundry_account_name_attempted"] = account_name info["foundry_project_name_attempted"] = project_name info["location"] = location info["integration_id"] = str(integration.id) detail["azure_openai"] = info assignment.detail = detail await db.commit() await db.refresh(assignment) detail = dict(assignment.detail or {}) info = dict(detail.get("azure_openai") or {}) # Build driver — wraps SP credential decryption. A decrypt failure # means an admin re-encrypt round (migration 125) didn't reach this # row, OR the integration's row has been re-encrypted with a key # this process can't unwrap. Surface as ``decrypt-failed`` so the # admin sees the right diagnostic instead of a misleading # ``rg-failed`` masking the underlying credential issue. try: driver = _build_driver(integration) except Exception as exc: # noqa: BLE001 logger.exception( "Azure OpenAI auto-grant: SP credential decrypt failed", ) info["granted"] = False info["in_flight"] = False # decrypt failure didn't touch ARM info["error"] = f"credential decryption failed: {exc}"[:500] detail["azure_openai"] = info assignment.detail = detail # Persist the diagnostic alongside the cleared tombstone so # the next Feishu callback can retry once the credential issue # is fixed (admin re-encrypt round, key rotation), and so # admins see the failure reason on /admin/assets/api-keys without # waiting for the outer Feishu-callback transaction to finalise # (which could roll back on an unrelated downstream error). await db.commit() await db.refresh(assignment) await record_autogrant_failure( db, assignment, kind='azure_openai', stage='decrypt-failed', error=str(info.get('error') or 'decrypt-failed'), provider_label=getattr(provider_row, 'name', None), ) return {"status": "decrypt-failed"} # All ARM-touching calls go through ``driver``. We do NOT need a # ``finally: await driver.close()`` here — ``AzureCloudDriver`` # constructs a fresh ``httpx.AsyncClient`` per method call (see # ``_http()``); no persistent state to release. Mirroring # Bedrock's ``provider.close()`` would be a no-op on this driver. # If a future driver change introduces a cached client / token, # add the finally-close then. # 1. Ensure RG. try: rg_payload = await driver.ensure_resource_group( subscription_id=subscription_id, name=rg_name, location=location, tags=tags, ) except Exception as exc: # noqa: BLE001 logger.exception("Azure OpenAI auto-grant: ensure_resource_group failed") info["granted"] = False info["in_flight"] = False # deterministic RG name → retry-safe info["error"] = f"ensure_resource_group: {exc}"[:500] detail["azure_openai"] = info assignment.detail = detail await db.commit() # durably persist the diagnostic await db.refresh(assignment) # avoid greenlet-on-lazy-load await record_autogrant_failure( db, assignment, kind='azure_openai', stage='rg-failed', error=str(info.get('error') or 'rg-failed'), provider_label=getattr(provider_row, 'name', None), ) return {"status": "rg-failed"} rg_arm_id = str(rg_payload.get("id") or "") # 2. Provision Foundry AIServices account (polls until Succeeded). try: account_payload = await driver.provision_foundry_account( subscription_id=subscription_id, resource_group=rg_name, account_name=account_name, location=location, custom_subdomain=account_name, tags=tags, ) except Exception as exc: # noqa: BLE001 logger.exception( "Azure OpenAI auto-grant: provision_foundry_account failed", ) info["granted"] = False # Deterministic account name → idempotent retry. ARM PUT on # an existing AIServices account returns the existing resource # (with the original customSubDomainName), so the next callback # safely picks up where this one left off. info["in_flight"] = False info["error"] = f"provision_foundry_account: {exc}"[:500] info["resource_group_arm_id"] = rg_arm_id detail["azure_openai"] = info assignment.detail = detail await db.commit() # durably persist the diagnostic await db.refresh(assignment) await record_autogrant_failure( db, assignment, kind='azure_openai', stage='account-failed', error=str(info.get('error') or 'account-failed'), provider_label=getattr(provider_row, 'name', None), ) return {"status": "account-failed"} account_endpoint_url = ( (account_payload.get("properties") or {}).get("endpoint") or f"https://{account_name}.cognitiveservices.azure.com/" ) foundry_account_arm_id = str(account_payload.get("id") or "") # 3. Provision Foundry project under the AIServices account. # Display name surfaces in the Foundry portal's project picker; # description carries the same context so an admin reading the # project blade sees the request number + applicant. project_display_name = ( f"{assignment.request_number} · " f"{(assignment.project_name or 'Helix API key').strip()[:80]}" ) project_description = ( f"Auto-provisioned by Helix for API key request " f"{assignment.request_number}. " f"Applicant: {email or 'unknown'}. " f"Project: {project or '-'}." )[:1024] try: project_payload = await driver.provision_foundry_project( subscription_id=subscription_id, resource_group=rg_name, account_name=account_name, project_name=project_name, location=location, display_name=project_display_name, description=project_description, tags=tags, ) except Exception as exc: # noqa: BLE001 logger.exception( "Azure OpenAI auto-grant: provision_foundry_project failed", ) info["granted"] = False # Deterministic project name → idempotent retry. ARM PUT on # an existing project returns the existing resource. The # account is already created and captured below, so the retry # path skips re-creating it (Azure returns the existing # account on PUT). info["in_flight"] = False info["error"] = f"provision_foundry_project: {exc}"[:500] info["foundry_account_arm_id"] = foundry_account_arm_id info["resource_group_arm_id"] = rg_arm_id detail["azure_openai"] = info assignment.detail = detail await db.commit() # durably persist the diagnostic await db.refresh(assignment) await record_autogrant_failure( db, assignment, kind='azure_openai', stage='project-failed', error=str(info.get('error') or 'project-failed'), provider_label=getattr(provider_row, 'name', None), ) return {"status": "project-failed"} foundry_project_arm_id = str(project_payload.get("id") or "") foundry_project_endpoint = _foundry_project_endpoint( account_name, project_name, ) # 4. Read KEY1 from the parent account (Foundry projects share # keys with the account). try: keys = await driver.list_foundry_account_keys( subscription_id=subscription_id, resource_group=rg_name, account_name=account_name, ) except Exception as exc: # noqa: BLE001 logger.exception( "Azure OpenAI auto-grant: list_foundry_account_keys failed", ) info["granted"] = False # listKeys is read-only and the account / project already exist # on Azure — the next callback retry hits the same resources. info["in_flight"] = False info["error"] = f"list_keys: {exc}"[:500] info["foundry_account_arm_id"] = foundry_account_arm_id info["foundry_project_arm_id"] = foundry_project_arm_id info["resource_group_arm_id"] = rg_arm_id detail["azure_openai"] = info assignment.detail = detail await db.commit() # durably persist the diagnostic await db.refresh(assignment) await record_autogrant_failure( db, assignment, kind='azure_openai', stage='list-keys-failed', error=str(info.get('error') or 'list-keys-failed'), provider_label=getattr(provider_row, 'name', None), ) return {"status": "list-keys-failed"} key1 = keys.get("key1") or "" if not key1: info["granted"] = False info["in_flight"] = False info["error"] = "list_keys returned empty key1" info["foundry_account_arm_id"] = foundry_account_arm_id info["foundry_project_arm_id"] = foundry_project_arm_id info["resource_group_arm_id"] = rg_arm_id detail["azure_openai"] = info assignment.detail = detail await db.commit() # durably persist the diagnostic await db.refresh(assignment) await record_autogrant_failure( db, assignment, kind='azure_openai', stage='list-keys-empty', error=str(info.get('error') or 'list-keys-empty'), provider_label=getattr(provider_row, 'name', None), ) return {"status": "list-keys-empty"} # 5. Mirror the RG into cmdb_cloud_scopes so cost-allocation rules # + the AI-spending dashboard pick it up natively. AI cost is # now identified by service-name classification (PR #253 / # migration 152 removed ``is_ai_dedicated``); the RG itself is # just a parent-scope-attached child for ownership / chargeback. # Generate the UUID up front so we can use it as a regular value # (assignment.cloud_scope_id FK + allocation rule cross-reference) # without an intermediate ``flush`` — flushing mid-transaction # against the test fixture's session triggers a MissingGreenlet # on commit, and is unnecessary because we hold the PK value. # Race-defence: ``cmdb_cloud_scopes`` carries a UNIQUE constraint # on ``(integration_id, external_id)``. A concurrent Feishu callback # (operator double-click, webhook retry mid-flight, out-of-band # cloud-scope sync that already imported the RG) can produce a # pre-existing row with the same key. Pre-check via SELECT and # reuse the existing id if found — without this, an IntegrityError # at flush time would propagate to ``handle_feishu_callback``'s # outer log-and-swallow and leave the assignment wedged with # ``in_flight=True`` (the round-2 audit caught this exact path). # Pre-check resolves >99% of races; the residual microsecond # window between SELECT and INSERT is documented below. pre_existing_scope = (await db.execute( select(CmdbCloudScope).where( CmdbCloudScope.integration_id == str(integration.id), CmdbCloudScope.external_id == rg_name, CmdbCloudScope.scope_type == SCOPE_TYPE_AZURE_RESOURCE_GROUP, ) )).scalar_one_or_none() if pre_existing_scope is not None: rg_scope_id = uuid.UUID(str(pre_existing_scope.id)) logger.info( "Azure OpenAI auto-grant: scope row already exists for " "(integration=%s, rg=%s) — reusing existing scope %s " "(concurrent-callback / out-of-band-sync race resolved at " "the pre-check).", integration.id, rg_name, rg_scope_id, ) else: rg_scope_id = uuid.uuid4() rg_scope = CmdbCloudScope( id=rg_scope_id, integration_id=str(integration.id), scope_type=SCOPE_TYPE_AZURE_RESOURCE_GROUP, external_id=rg_name, display_name=( f"{rg_name} · AI API · {assignment.request_number}" )[:255], parent_scope_id=str(sub_scope.id), owner_person_id=( str(assignment.assigned_to) if assignment.assigned_to else None ), budget_usd_monthly=assignment.monthly_budget_usd, environment="dev", notes=( f"Auto-provisioned by Helix for API key request " f"{assignment.request_number} (project: {project or '-'}). " f"Foundry account: {account_name}; project: {project_name}." ), raw_data={ "source": "helix_autogrant.azure_openai", "request_number": assignment.request_number, "foundry_account": { "name": account_name, "arm_id": foundry_account_arm_id, "endpoint": account_endpoint_url, }, "foundry_project": { "name": project_name, "arm_id": foundry_project_arm_id, "endpoint": foundry_project_endpoint, "display_name": project_display_name, }, "rg_arm_id": rg_arm_id, }, ) db.add(rg_scope) # Flush so the new scope row exists before the assignment # UPDATE auto-flushed at commit-time references it via FK. # Without this the auto-flush re-orders the assignment UPDATE # ahead of the cmdb_cloud_scopes INSERT and TiDB/MySQL rejects # with FK 1452. # # If a vanishingly-rare race wins between our pre-check and # this flush (microsecond window), an IntegrityError will # propagate up. The pre-check eliminates the realistic case # (concurrent callbacks separated by ARM-call latency, which # is hundreds of ms). The residual window can only be hit by # two co-scheduled processes inserting in lockstep — at which # point the assignment goes through ``handle_feishu_callback``'s # outer log path with in_flight=True; admin reconciliation is # the right escape hatch for that genuinely-stuck case. await db.flush() # 6. Bind cloud_scope_id on the assignment. assignment.cloud_scope_id = str(rg_scope_id) # 7. Persist the credential bundle. The reveal endpoint returns # plaintext; the help center page renders these as labelled # copy buttons (see /help#foundry-employee). # # Two SDK styles are supported: # # * ``azure-ai-projects`` (Foundry SDK, recommended): the # ``AIProjectClient`` constructor takes ``endpoint=`` + # ``credential=`` explicitly — there is NO env-var auto- # discovery in the SDK itself. The bundle ships # ``FOUNDRY_PROJECT_ENDPOINT`` + ``FOUNDRY_API_KEY`` matching # Microsoft's own README convention so users can copy-paste # the docs samples without renaming. # * ``openai`` python client (legacy ``AzureOpenAI`` class): # DOES auto-discover ``AZURE_OPENAI_API_KEY`` / # ``AZURE_OPENAI_ENDPOINT`` / ``AZURE_OPENAI_API_VERSION``, # so a ``source `` is enough to authenticate. # # ``AZURE_OPENAI_DEPLOYMENT_NAME`` is intentionally COMMENTED # so a ``source `` doesn't clobber a pre-existing value # with empty. Model deployments are provisioned by **IT**, not the # employee — the live deployment name(s) are surfaced in the # "如何使用" panel (see ``ApiKeyService.usage_hints`` → # ``list_foundry_deployments``) and the employee should contact IT # if none is available. The bundle no longer tells the employee to # self-deploy a model (that was wrong: employees have no portal # access to the Foundry account, and IT owns model rollout). plaintext_bundle = ( f"# ── Microsoft Foundry SDK (azure-ai-projects) ──\n" f"# Pass these explicitly to AIProjectClient(endpoint=...,\n" f"# credential=AzureKeyCredential(...)) — the SDK does NOT\n" f"# auto-discover env vars on its own.\n" f"FOUNDRY_PROJECT_ENDPOINT={foundry_project_endpoint}\n" f"FOUNDRY_API_KEY={key1}\n" f"FOUNDRY_PROJECT_NAME={project_name}\n" f"FOUNDRY_ACCOUNT_NAME={account_name}\n" f"FOUNDRY_ACCOUNT_ENDPOINT={account_endpoint_url}\n" f"\n" f"# ── openai python client (AzureOpenAI) ──\n" f"# These names ARE auto-discovered by the openai SDK —\n" f"# sourcing this block is enough for the client to\n" f"# authenticate.\n" f"AZURE_OPENAI_API_KEY={key1}\n" f"AZURE_OPENAI_ENDPOINT={account_endpoint_url}\n" f"AZURE_OPENAI_API_VERSION=2024-10-21\n" f"# AZURE_OPENAI_DEPLOYMENT_NAME=" f" # the model= / deployment to call; provisioned by IT\n" f"\n" f"# ── Azure plumbing (informational) ──\n" f"AZURE_RESOURCE_GROUP={rg_name}\n" f"AZURE_SUBSCRIPTION_ID={subscription_id}\n" f"\n" f"# NOTE: Model deployments on this Foundry project are managed\n" f"# by IT. The '如何使用 / How to use' panel in Helix lists the\n" f"# live deployment name(s) to use as your model= argument. If it\n" f"# shows none — or you get a DeploymentNotFound error — contact\n" f"# IT to have a model deployed; do NOT try to deploy one\n" f"# yourself (you don't have portal access to this account).\n" f"# See /help#foundry-employee for the full SDK walk-through." ) assignment.key_value = _encrypt_key_value( plaintext_bundle, assignment_id=assignment.id, ) assignment.status = "active" # 8. Auto-create allocation rule so the cost ingest cron stamps # every line item under (this RG × this subscription) with the # applicant as the cost owner. Priority 10 puts RG-targeted # rules ahead of broader catch-alls (default rules typically # sit at priority 100+). # # Idempotency: mirror bedrock_autogrant.py:467-506 — a defensive # name-uniqueness pre-check so an admin manually re-running the # autogrant via ``/admin/assets/api-keys`` (after clearing # ``detail.azure_openai.granted``) doesn't end up with a duplicate # rule pointing at the same RG. Rules with the same name+RG would # both match every fact and only the lower-id one (tiebreaker on # equal priority) would stamp the owner — silent duplication # bloating the rules table. Bedrock added this guard in PR #220; # AOAI was missing it (audit 2026-05-12). rule_name = ( f"Azure Foundry · {assignment.request_number} · " f"{email or 'unknown'}" )[:255] existing_rule = (await db.execute( select(CmdbCostAllocationRule).where( CmdbCostAllocationRule.name == rule_name, ) )).scalar_one_or_none() if existing_rule is not None: rule = existing_rule else: rule = CmdbCostAllocationRule( id=str(uuid.uuid4()), name=rule_name, priority=10, match_integration_ids=[str(integration.id)], match_provider="azure", match_resource_group=rg_name, match_service=None, match_tag_key=None, match_tag_value=None, assign_person_id=( str(assignment.assigned_to) if assignment.assigned_to else None ), # Department / cost-center deliberately left NULL on the rule # row. ``person.department`` is a hybrid property that walks # ``primary_department`` and may trigger a lazy-load in the # autogrant's async context (greenlet boundary error). The # cost-allocation pipeline already denormalises these onto # ``cmdb_cost_facts.cost_owner_department`` / # ``cost_center`` at ingest time by re-resolving from the # ``assign_person_id``, so the rule row doesn't need them # baked in. assign_department=None, assign_cost_center=None, enabled=True, description=( f"Auto-generated by helix_autogrant.azure_openai for " f"assignment {assignment.id}. Charges every Azure cost " f"line item in RG {rg_name} (sub {subscription_id}) to " f"the API key applicant." ), created_by_id=None, updated_by_id=None, ) db.add(rule) info.update({ "granted": True, "in_flight": False, "subscription_id": subscription_id, "resource_group": rg_name, "resource_group_arm_id": rg_arm_id, # Foundry hierarchy — account (parent kind=AIServices) + # project (child Foundry project that the assignee actually # uses). All namespaced under ``foundry_*``. An audit confirmed # no consumer (admin UI, MCP tools, dashboards, schemas) reads # ``account_name`` / ``endpoint`` / ``account_arm_id`` at the # top level of this dict — those legacy keys are intentionally # NOT aliased to keep the schema clean. The # ``test_autogrant_succeeds_for_azure_openai`` test asserts # their ABSENCE so a future merge can't reintroduce them. "foundry_account_name": account_name, "foundry_account_arm_id": foundry_account_arm_id, "foundry_account_endpoint": account_endpoint_url, "foundry_project_name": project_name, "foundry_project_arm_id": foundry_project_arm_id, "foundry_project_endpoint": foundry_project_endpoint, "foundry_project_display_name": project_display_name, "location": location, "integration_id": str(integration.id), "scope_id": str(rg_scope_id), "allocation_rule_id": rule.id, "granted_at": datetime.now(UTC).isoformat(), }) detail["azure_openai"] = info assignment.detail = detail # Durability: commit credentials + scope + allocation rule NOW so # a subsequent rollback in the Feishu-callback transaction cannot # leave us with an orphan Foundry account / project on Azure but no # Helix DB record. Mirrors the Bedrock pattern. try: await db.commit() await db.refresh(assignment) except Exception: # noqa: BLE001 logger.exception( "Azure OpenAI auto-grant: Foundry resources created " "(account=%s, project=%s, RG=%s) but DB commit failed. " "Manual reconciliation required — see cmdb-ops.md " "in-flight recovery section.", account_name, project_name, rg_name, ) raise # Notify the assignee. Done after the durable commit so a # notification-channel failure cannot roll back the Foundry # provision. if person is not None: provider_label = ( provider_row.name if provider_row is not None else "Azure OpenAI" ) foundry_portal_url = ( f"https://ai.azure.com/build/projects/{project_name}" ) # Sanitise free-text fields before splicing into a multi-line # body. ``project_name`` is admin-controlled at request time # and accepts arbitrary text; ``display_name`` / ``work_email`` # come from HRIS so are lower risk but not zero-trust. # Stripping CR/LF guards against: # * visual breakage of the body (a stray \n in project_name # would lift the "请到..." line up into the wrong section); # * any rich-card / markdown-aware downstream renderer # (Feishu cards, email clients) interpreting injected # control characters as formatting / link markers. # Truncation matches the existing tag-truncation length so the # body never balloons past a reasonable size. def _safe_text(value: str | None, *, limit: int = 80) -> str: return ( (value or "") .replace("\r\n", " ") .replace("\n", " ") .replace("\r", " ") .strip()[:limit] ) safe_project_name = _safe_text(assignment.project_name) safe_display_name = _safe_text(person.display_name) safe_email = _safe_text(person.work_email, limit=120) safe_recipient = safe_display_name or safe_email or "" try: await NotificationService(db).dispatch( NotificationEnvelope( recipient=person, title=( "[Helix] Azure OpenAI / Foundry API Key 已就绪 / " "API key ready" ), body_text=( f"Hi {safe_recipient},\n\n" f"你申请的 {provider_label} API Key (申请号 " f"{assignment.request_number},项目 " f"{safe_project_name or '-'}) 已自动开通。\n\n" f"请到 Helix 的「我的 API Keys」页面查看凭证和使用方法:\n" f" /me/api-keys\n\n" f"已自动为你在 Azure 上创建:\n" f" • 独立资源组 {rg_name}\n" f" • Microsoft Foundry 资源 {account_name}\n" f" • Foundry 项目 {project_name}\n\n" f"账号成本可在 Azure Cost Management 按 RG 维度查看。" f"模型部署由 IT 统一管理:「我的 API Keys」的" f"「如何使用」面板会列出当前可用的部署名(model= 参数)," f"若显示暂无或调用报 DeploymentNotFound,请联系 IT 部署," f"无需(也无法)自行部署。详细使用手册见:" f"/help#foundry-employee\n\n" f"— Helix IT 平台\n" f"---\n" f"Hi {safe_recipient},\n\n" f"Your {provider_label} API key (request " f"{assignment.request_number}, project " f"{safe_project_name or '-'}) is now active.\n\n" f"View the credential on your Helix \"My API Keys\" " f"page:\n /me/api-keys\n\n" f"Provisioned for you on Azure:\n" f" • Resource group {rg_name}\n" f" • Microsoft Foundry resource {account_name}\n" f" • Foundry project {project_name}\n\n" f"Cost shows up under that RG in Azure Cost " f"Management. Model deployments are managed by IT " f"— the \"How to use\" panel on your My API Keys " f"page lists the live deployment name to use as " f"your model= argument. If it shows none, or you " f"get a DeploymentNotFound error, contact IT to " f"have a model deployed (you don't have portal " f"access to this account). See " f"/help#foundry-employee for the SDK setup.\n\n" f"— Helix IT Platform" ), template_key="apikey.ready", context={ "person_name": safe_recipient, "person_email": safe_email, "provider_name": provider_label, "request_number": assignment.request_number, "project_name": safe_project_name, "resource_group": rg_name, "foundry_account": account_name, "foundry_project": project_name, "foundry_project_endpoint": foundry_project_endpoint, "foundry_portal_url": foundry_portal_url, "portal_url": "/me/api-keys", "help_anchor": "/help#foundry-employee", }, url="/me/api-keys", related_resource_type="api_key_assignment", related_resource_id=assignment.id, ), ) except Exception: # noqa: BLE001 — never fail the grant on notify logger.exception( "Azure OpenAI auto-grant: notification dispatch failed for " "assignment %s; credential remains active and is visible " "on /me/api-keys.", assignment.request_number, ) logger.info( "Azure OpenAI auto-grant succeeded: assignment=%s rg=%s " "foundry_account=%s foundry_project=%s", assignment.id, rg_name, account_name, project_name, ) return { "status": "granted", "resource_group": rg_name, "foundry_account_name": account_name, "foundry_project_name": project_name, "foundry_project_endpoint": foundry_project_endpoint, "scope_id": str(rg_scope_id), "allocation_rule_id": rule.id, } __all__ = ["maybe_auto_grant_azure_openai"]