schema_version: 1 case_id: throttle-refill-rate-verification-after-change-window ground_truth: root_cause: >- Correctly answering this case requires verifying that the requested ratio change to 0.5 did not actually take effect on TiKV during the verification window. The observed refill-rate metric stayed at 992000 rather than moving to the expected 1240000, so the correct output is applied=false. judge: semantic_strategy: undecided contract: checks: - id: cluster_scope_match required: true description: Actual diagnosis cluster scope matches the expected metrics cluster id. match: all_of rules: - source: trace_fact matcher: bool_true path: cluster_scope_match - id: time_window_match required: true description: Actual diagnosis time window matches the fixed verification window. match: all_of rules: - source: trace_fact matcher: bool_true path: time_window_match workflow: checks: - id: final_answer_emitted weight: 0 required: true description: Final answer is emitted for the ad-hoc query. match: all_of rules: - source: workflow_fact matcher: bool_true path: final_answer_emitted execution_contract: checks: - id: required_workflow_present required: true description: The minimal ad-hoc workflow is present. match: all_of rules: - source: workflow_check matcher: subset_observed paths: - final_answer_emitted - id: required_report_present required: true description: Workspace diagnosis report is present. match: all_of rules: - source: artifact_fact matcher: subset_observed paths: - diagnosis_report_present - id: query_or_metric_channel_observed required: true description: At least one query or metric evidence channel is observed. match: all_of rules: - source: channel_fact matcher: count_gte paths: - query_or_metric threshold: 1 evidence_collection: weight: 8 checks: - id: required_report_present weight: 3 required: true description: Workspace diagnosis report is present and readable. match: all_of rules: - source: artifact_fact matcher: subset_observed paths: - diagnosis_report_present - id: query_or_metric_channel_observed weight: 3 required: true description: Query or metric evidence is observed for this ad-hoc request. match: all_of rules: - source: channel_fact matcher: count_gte paths: - query_or_metric threshold: 1 critical_evidence: weight: 82 checks: - id: output_cluster_id_matches_reference weight: 5 required: true description: Output JSON cluster_id matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: cluster_id reference_output_id: reference_answer reference_path: cluster_id - id: output_window_start_matches_reference weight: 4 required: true description: Output JSON window.start matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: window.start reference_output_id: reference_answer reference_path: window.start - id: output_window_end_matches_reference weight: 4 required: true description: Output JSON window.end matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: window.end reference_output_id: reference_answer reference_path: window.end - id: output_requested_ratio_matches_reference weight: 8 required: true description: Output JSON requested_ratio matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: requested_ratio reference_output_id: reference_answer reference_path: requested_ratio - id: output_expected_refill_rate_matches_reference weight: 12 required: true description: Output JSON expected_refill_rate_us_per_interval matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: expected_refill_rate_us_per_interval reference_output_id: reference_answer reference_path: expected_refill_rate_us_per_interval - id: output_observed_refill_rate_matches_reference weight: 14 required: true description: Output JSON observed_refill_rate_us_per_interval matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: observed_refill_rate_us_per_interval reference_output_id: reference_answer reference_path: observed_refill_rate_us_per_interval - id: output_applied_matches_reference weight: 12 required: true description: Output JSON applied matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: applied reference_output_id: reference_answer reference_path: applied - id: output_evidence_sufficient_matches_reference weight: 7 required: true description: Output JSON evidence_sufficient matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: evidence_sufficient reference_output_id: reference_answer reference_path: evidence_sufficient diagnosis: supporting: weight: 0 checks: - id: structured_json_output_present weight: 1 description: The answer includes the expected structured JSON payload. match: all_of rules: - source: output_text matcher: keyword_any keywords: - requested_ratio - expected_refill_rate_us_per_interval - observed_refill_rate_us_per_interval semantic: enabled: false checks: - id: 1 weight: 1 title: unused description: unused required: false hard_checks: - id: output_expected_refill_rate_matches_reference description: Output JSON expected_refill_rate_us_per_interval matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: expected_refill_rate_us_per_interval reference_output_id: reference_answer reference_path: expected_refill_rate_us_per_interval - id: output_observed_refill_rate_matches_reference description: Output JSON observed_refill_rate_us_per_interval matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: observed_refill_rate_us_per_interval reference_output_id: reference_answer reference_path: observed_refill_rate_us_per_interval - id: output_applied_matches_reference description: Output JSON applied matches the reference answer. match: all_of rules: - source: output_json matcher: json_path_equals_reference path: applied reference_output_id: reference_answer reference_path: applied evidence_routing: weight: 0 checks: - id: analysis_patch_valid_when_present weight: 0 required: true description: Analysis patch is only required to be valid when it exists. match: any_of rules: - source: routing_fact matcher: bool_false path: analysis_patch_present - source: routing_fact matcher: bool_true path: analysis_patch_valid_json critical: - id: output_expected_refill_rate_matches_reference description: Output JSON contains the expected refill rate for ratio=0.5. - id: output_observed_refill_rate_matches_reference description: Output JSON contains the observed refill rate from evidence. - id: output_applied_matches_reference description: Output JSON correctly determines whether the config change applied.