schema_version: 0.1.0 kind: source id: source.nexuscrm.opportunity_s3_snapshot name: nexuscrm Opportunity S3 snapshot export version: 0.1.0 owner: type: team name: data-platform domain: name: crm purpose: Source-faithful per-snapshot Parquet export of nexuscrm.Opportunity from TiDB Cloud Essential to S3, consumed by the lakehouse load stage. usage: intended: - Bulk load into the raw lakehouse layer - Audit and provenance of exported snapshots prohibited: - Direct analytical querying (load into a contracted product first). - Treating a single snapshot as an intra-day event stream. source: kind: database system: tidbcloud description: TiDB Cloud Essential nexuscrm.Opportunity identifiers: database: nexuscrm table: Opportunity cluster_id: '10996361222721502359' assets: - id: opportunity_s3_snapshot kind: file ref: s3://pingcap-datalake-nexuscrm-export-247906201378-us-west-2/tidb-essential/snapshots/source=nexuscrm/ materialization: external produced_by: kind: python ref: ingestion.extract.tidbcloud.engine.export schema: kind: relational columns: - name: id type: varchar(255) nullable: false - name: ownerid type: varchar(255) nullable: true - name: createdbyid type: varchar(255) nullable: true - name: lastmodifiedbyid type: varchar(255) nullable: true - name: createddate type: datetime nullable: false - name: lastmodifieddate type: datetime nullable: false - name: systemmodstamp type: datetime nullable: false - name: isdeleted type: tinyint(1) nullable: false - name: name type: varchar(255) nullable: false - name: accountid type: varchar(255) nullable: true - name: stagename type: varchar(255) nullable: false - name: amount type: decimal(65,30) nullable: true - name: closedate type: date nullable: false - name: probability type: decimal(65,30) nullable: true - name: forecastcategory type: varchar(255) nullable: false - name: forecastcategoryname type: varchar(255) nullable: true - name: expectedrevenue type: decimal(65,30) nullable: true - name: hasopportunitylineitem type: tinyint(1) nullable: true - name: laststagechangedate type: datetime nullable: true - name: isclosed type: tinyint(1) nullable: true - name: hasopenactivity type: tinyint(1) nullable: true - name: type type: varchar(255) nullable: true - name: leadsource type: varchar(255) nullable: true - name: nextstep type: text nullable: true - name: campaignid type: varchar(255) nullable: true - name: description type: text nullable: true - name: recordtypeid type: varchar(255) nullable: true - name: currencyisocode type: varchar(255) nullable: true - name: abp_document__c type: varchar(255) nullable: true - name: acv_amount_usd__c type: decimal(65,30) nullable: true - name: acv_amount_do_not_use__c type: decimal(65,30) nullable: true - name: acv_usd_with_cd_rate__c type: decimal(65,30) nullable: true - name: arr_amount_usd__c type: decimal(65,30) nullable: true - name: aws_campaign_name__c type: varchar(255) nullable: true - name: aws_close_date__c type: date nullable: true - name: aws_contract_vehicle__c type: varchar(255) nullable: true - name: aws_delivery_model__c type: json nullable: true - name: aws_field_engagement__c type: varchar(255) nullable: true - name: aws_lost_reason__c type: varchar(255) nullable: true - name: aws_project_description__c type: text nullable: true - name: aws_project_title__c type: varchar(255) nullable: true - name: aws_rejection_reason__c type: varchar(255) nullable: true - name: aws_stage__c type: varchar(255) nullable: true - name: aws_status__c type: varchar(255) nullable: true - name: aws_use_case__c type: varchar(255) nullable: true - name: account_name_original__c type: varchar(255) nullable: true - name: accurate_acv_amount__c type: decimal(65,30) nullable: true - name: age__c type: decimal(65,30) nullable: true - name: alicloud_channel__c type: tinyint(1) nullable: true - name: allow_close__c type: tinyint(1) nullable: true - name: any_compelling_event__c type: varchar(255) nullable: true - name: application_map__c type: longtext nullable: true - name: application_scenario__c type: varchar(255) nullable: true - name: aurora_data_volume__c type: varchar(255) nullable: true - name: aurora_instance_number_1st__c type: decimal(65,30) nullable: true - name: aurora_instance_number_2nd__c type: decimal(65,30) nullable: true - name: aurora_instance_number_3rd__c type: decimal(65,30) nullable: true - name: aurora_instance_type_1st__c type: varchar(255) nullable: true - name: aurora_instance_type_2nd__c type: varchar(255) nullable: true - name: aurora_instance_type_3rd__c type: varchar(255) nullable: true - name: budget__c type: varchar(255) nullable: true - name: budget_allocated__c type: varchar(255) nullable: true - name: budget_approved_or_available_already__c type: varchar(255) nullable: true - name: business_case__c type: varchar(255) nullable: true - name: business_painpoint_impact__c type: varchar(255) nullable: true - name: cd_rate__c type: decimal(65,30) nullable: true - name: champion__c type: longtext nullable: true - name: close_date__c type: date nullable: true - name: close_date_same_as_forecast__c type: date nullable: true - name: closed_lost_date__c type: date nullable: true - name: closed_lost_from_stage__c type: varchar(255) nullable: true - name: cloud_credits_requested__c type: decimal(65,30) nullable: true - name: cloud_order_type__c type: varchar(255) nullable: true - name: cloud_vendor__c type: json nullable: true - name: company_name_for_aws__c type: varchar(255) nullable: true - name: compelling_event_date__c type: date nullable: true - name: competition__c type: longtext nullable: true - name: competitive_advantage_level__c type: varchar(255) nullable: true - name: competitor_details__c type: longtext nullable: true - name: contractid type: varchar(255) nullable: true - name: contract_archived_date__c type: date nullable: true - name: contract_end_date__c type: date nullable: true - name: contract_start_date__c type: date nullable: true - name: contracting_start_date__c type: date nullable: true - name: corporate_wide_infrastructure_team_or_ap__c type: varchar(255) nullable: true - name: country_for_aws__c type: varchar(255) nullable: true - name: customer_email_for_aws__c type: varchar(255) nullable: true - name: customer_first_name_for_aws__c type: varchar(255) nullable: true - name: customer_influence_textl__c type: text nullable: true - name: customer_influence_text__c type: varchar(255) nullable: true - name: customer_influence__c type: tinyint(1) nullable: true - name: customer_last_name_for_aws__c type: varchar(255) nullable: true - name: customer_phone_for_aws__c type: varchar(50) nullable: true - name: customer_software_value__c type: decimal(65,30) nullable: true - name: customer_title_for_aws__c type: varchar(255) nullable: true - name: data_consolidation__c type: varchar(255) nullable: true - name: data_volume_size__c type: varchar(255) nullable: true - name: database_maintenance_hc__c type: varchar(255) nullable: true - name: deadline__c type: date nullable: true - name: deal_registration__c type: varchar(255) nullable: true - name: decision_criteria__c type: longtext nullable: true - name: decision_maker__c type: text nullable: true - name: decision_process__c type: longtext nullable: true - name: discovery_completed__c type: tinyint(1) nullable: true - name: documents_folder_na_region_use_only__c type: varchar(255) nullable: true - name: economic_buyer__c type: longtext nullable: true - name: elvance_scorecard_score__c type: decimal(65,30) nullable: true - name: elvance__elvance_owner__c type: varchar(255) nullable: true - name: elvance__elvance_stage__c type: varchar(255) nullable: true - name: evaluatiodwelltime__c type: decimal(65,30) nullable: true - name: evaluatiotime__c type: datetime nullable: true - name: existing_database_deployment_remarks__c type: varchar(255) nullable: true - name: existing_database_deployment__c type: json nullable: true - name: expected_monthly_aws_revenue__c type: decimal(65,30) nullable: true - name: fy22_ib__c type: tinyint(1) nullable: true - name: fy25_to_fy26_transition__c type: tinyint(1) nullable: true - name: fiscal_period__c type: varchar(255) nullable: true - name: forecast_quarter__c type: varchar(255) nullable: true - name: forecast_type__c type: varchar(255) nullable: true - name: fulfillment_channel__c type: varchar(255) nullable: true - name: funding_process__c type: varchar(255) nullable: true - name: gclid__c type: varchar(255) nullable: true - name: hyperscaler_source__c type: json nullable: true - name: iqm_note_na_region_use_only__c type: varchar(255) nullable: true - name: identify_pain__c type: longtext nullable: true - name: implicate_pain__c type: longtext nullable: true - name: inthisfiscalyear__c type: tinyint(1) nullable: true - name: industry__c type: varchar(255) nullable: true - name: industry_for_aws__c type: varchar(255) nullable: true - name: infrastructure_planning__c type: text nullable: true - name: infrastructure_purchase_history__c type: text nullable: true - name: infrastructure_status__c type: text nullable: true - name: is_marketing_development_funded__c type: varchar(255) nullable: true - name: is_net_new_business_for_company__c type: varchar(255) nullable: true - name: is_opp_from_marketing_activity__c type: varchar(255) nullable: true - name: is_this_for_marketplace__c type: varchar(255) nullable: true - name: key_deal__c type: varchar(255) nullable: true - name: key_updates__c type: text nullable: true - name: last_modified_over_14_days__c type: tinyint(1) nullable: true - name: launch_date__c type: date nullable: true - name: lead_owner_id__c type: varchar(255) nullable: true - name: lead_source_detail__c type: varchar(255) nullable: true - name: lighthouse_logo__c type: varchar(255) nullable: true - name: link_to_influence_map__c type: varchar(255) nullable: true - name: loss_reason_notes__c type: text nullable: true - name: loss_reason__c type: varchar(255) nullable: true - name: metrics__c type: longtext nullable: true - name: mysql_family_capture_remarks__c type: varchar(255) nullable: true - name: mysql_family_capture__c type: varchar(255) nullable: true - name: negative_impact_if_not_now__c type: varchar(255) nullable: true - name: net_fees_usd__c type: decimal(65,30) nullable: true - name: next_key_action__c type: longtext nullable: true - name: next_milestone__c type: text nullable: true - name: non_standard_referral_fee_rate__c type: decimal(65,30) nullable: true - name: number__c type: varchar(255) nullable: true - name: op_to_cloud_migration_file_id__c type: varchar(255) nullable: true - name: op_to_cloud_migration_file__c type: tinyint(1) nullable: true - name: op_to_cloud_migration__c type: tinyint(1) nullable: true - name: op_to_cloud__c type: tinyint(1) nullable: true - name: opportunity_description__c type: text nullable: true - name: opportunity_number__c type: varchar(255) nullable: true - name: opportunity_owner_email__c type: varchar(255) nullable: true - name: opportunity_owner_name__c type: varchar(255) nullable: true - name: opportunity_ownership__c type: varchar(255) nullable: true - name: opportunity_registration_number__c type: varchar(255) nullable: true - name: opportunity_source__c type: varchar(255) nullable: true - name: opportunity_type__c type: varchar(255) nullable: true - name: opportunity_registration__c type: tinyint(1) nullable: true - name: opt_owner_role__c type: text nullable: true - name: order_archived_date__c type: date nullable: true - name: originalopportunity__c type: varchar(255) nullable: true - name: originalorder__c type: varchar(255) nullable: true - name: other_solution_offered_for_aws__c type: varchar(255) nullable: true - name: overalll_age__c type: decimal(65,30) nullable: true - name: plg_bd_pool__c type: tinyint(1) nullable: true - name: pain_point_selection__c type: varchar(255) nullable: true - name: pain_points_and_remarks__c type: text nullable: true - name: paper_process__c type: longtext nullable: true - name: parnter_status__c type: varchar(255) nullable: true - name: partneractsastechliaisonarchdesigntextl__c type: text nullable: true - name: partneractsastechliaisonarchdesigntext__c type: varchar(255) nullable: true - name: partnerconfigbuildpocfile__c type: tinyint(1) nullable: true - name: partnerintroducesnewclevelexecreltextl__c type: text nullable: true - name: partnerintroducesnewclevelexecreltext__c type: varchar(255) nullable: true - name: partnerleadsjointcustomereventstextl__c type: text nullable: true - name: partnerleadsjointcustomereventstext__c type: varchar(255) nullable: true - name: partner_attribution__c type: tinyint(1) nullable: true - name: partner_configure_build_poc__c type: tinyint(1) nullable: true - name: partner_influence_attribution__c type: json nullable: true - name: partner_name__c type: varchar(255) nullable: true - name: partner_notes__c type: varchar(255) nullable: true - name: partner_primary_need_from_aws_new__c type: json nullable: true - name: partner_primary_need_from_aws_remarks__c type: varchar(255) nullable: true - name: partner_primary_need_from_aws__c type: varchar(255) nullable: true - name: partner_technical_liaison_architectural__c type: tinyint(1) nullable: true - name: partner_introduces_new_c_level_customer__c type: tinyint(1) nullable: true - name: partner_leads_and_arranges_joint_custome__c type: tinyint(1) nullable: true - name: person_responsible__c type: text nullable: true - name: person_source_detail__c type: varchar(255) nullable: true - name: person_source__c type: varchar(255) nullable: true - name: pingcap_acceptance_status__c type: varchar(255) nullable: true - name: pingcap_contracting_entity_emea_only__c type: varchar(255) nullable: true - name: pingcap_source__c type: tinyint(1) nullable: true - name: pingcap_stage__c type: varchar(255) nullable: true - name: pipe_gen_date__c type: date nullable: true - name: planned_poc_completion_date__c type: date nullable: true - name: planned_poc_start_date__c type: date nullable: true - name: planning_for_tidb__c type: longtext nullable: true - name: poc_planning_document__c type: varchar(255) nullable: true - name: postal_code_for_aws__c type: varchar(255) nullable: true - name: pre_sales_owner__c type: varchar(255) nullable: true - name: pricebook2id type: varchar(255) nullable: true - name: primary_contact_email_for_aws__c type: varchar(255) nullable: true - name: primary_contact_first_name_for_aws__c type: varchar(255) nullable: true - name: primary_contact_last_name_for_aws__c type: varchar(255) nullable: true - name: primary_contact_phone_for_aws__c type: varchar(50) nullable: true - name: procurement_type__c type: varchar(255) nullable: true - name: product_family__c type: varchar(255) nullable: true - name: project_description__c type: longtext nullable: true - name: project_name__c type: varchar(255) nullable: true - name: project_schedule__c type: text nullable: true - name: rfx_solicitation_number__c type: varchar(255) nullable: true - name: roi_analysis_completed__c type: tinyint(1) nullable: true - name: rate__c type: decimal(65,30) nullable: true - name: recent_source_detail__c type: varchar(255) nullable: true - name: recent_source__c type: varchar(255) nullable: true - name: recordtypename__c type: varchar(255) nullable: true - name: referral_is_valid_until__c type: date nullable: true - name: referral_valid_period_is_extended__c type: tinyint(1) nullable: true - name: region__c type: varchar(255) nullable: true - name: replacing_database_remarks__c type: varchar(255) nullable: true - name: replacing_database__c type: json nullable: true - name: review_comments__c type: text nullable: true - name: risk_rating__c type: varchar(255) nullable: true - name: risk_and_next_step__c type: text nullable: true - name: risky_contact_in_powerbase__c type: varchar(255) nullable: true - name: sales_activities_for_aws__c type: json nullable: true - name: sales_channel_type__c type: varchar(255) nullable: true - name: segmentation__c type: varchar(255) nullable: true - name: self_serving_ondemand__c type: tinyint(1) nullable: true - name: solution_offered_for_aws__c type: text nullable: true - name: stage_1_email_attachments_id__c type: varchar(255) nullable: true - name: stage_3_file_id__c type: varchar(255) nullable: true - name: stage_duration__c type: decimal(65,30) nullable: true - name: state_for_aws__c type: varchar(255) nullable: true - name: submitted_by_partner__c type: tinyint(1) nullable: true - name: sync_with_aws_auto__c type: tinyint(1) nullable: true - name: syncedquoteid type: varchar(255) nullable: true - name: tse__c type: varchar(255) nullable: true - name: taking_aurora_s_share__c type: varchar(255) nullable: true - name: technical_decision_maker__c type: varchar(255) nullable: true - name: technical_progress__c type: text nullable: true - name: technical_scenario__c type: varchar(255) nullable: true - name: technical_success_criteria__c type: varchar(255) nullable: true - name: technical_validation_process__c type: varchar(255) nullable: true - name: territory2id type: varchar(255) nullable: true - name: tidb_deployment_estimate__c type: longtext nullable: true - name: tidb_icp_winning_scenario__c type: varchar(255) nullable: true - name: top_issue__c type: longtext nullable: true - name: total_of_mysql_clusters__c type: varchar(255) nullable: true - name: unit_net_price_yearly__c type: decimal(65,30) nullable: true - name: update_date__c type: date nullable: true - name: upload_email_attachments__c type: tinyint(1) nullable: true - name: vertical__c type: varchar(255) nullable: true - name: website_for_aws__c type: varchar(255) nullable: true - name: why_now_average_score__c type: decimal(65,30) nullable: true - name: why_now__c type: longtext nullable: true - name: why_tidb_average_score__c type: decimal(65,30) nullable: true - name: why_tidb__c type: longtext nullable: true - name: why_us__c type: longtext nullable: true - name: why_do_anything__c type: longtext nullable: true - name: winning_scenario__c type: tinyint(1) nullable: true - name: affectlayer__affectlayer_notes__c type: varchar(255) nullable: true - name: awsapn__apn_crm_id__c type: varchar(255) nullable: true - name: awsapn__eligible_to_sync_with_apn__c type: tinyint(1) nullable: true - name: awsapn__last_apn_sync_date__c type: datetime nullable: true - name: awsapn__sync_with_partner_central__c type: tinyint(1) nullable: true - name: discount__c type: decimal(65,30) nullable: true - name: net_fees__c type: decimal(65,30) nullable: true - name: overall_start_date__c type: date nullable: true - name: quantity__c type: decimal(65,30) nullable: true freshness: max_age: PT48H degrade_after: PT30H tests: - id: opportunity_source_smoke type: sql severity: informational description: Placeholder informational smoke check for the source contract of opportunity. Replace with concrete row-count, freshness, or key-uniqueness assertions. sql: select 1 governance: classifications: - customer_data - financial - pii - restricted access: default: restricted retention: period: P7D audit: required: true events: - export - agent_action agent_permissions: default: deny compatibility: change_policy: semver extensions: primary_key: - Id watermark_columns: - SystemModstamp - LastModifiedDate row_estimate: 3582 known_blind_spots: - Intra-snapshot changes are collapsed. - Insert-then-delete between snapshots is invisible. - Schema drift requires review before promotion.