{"record":{"id":"a34632ba0f93456f","repo":"nautechsystems/nautilus_trader","slug":"connect-verification-evidence-is-incomplete","errorCode":null,"errorMessage":"Connect verification evidence is incomplete","messagePattern":"Connect verification evidence is incomplete","errorType":"validation","errorClass":"anyhow::Error","httpStatus":null,"severity":"error","filePath":"crates/adapters/blockchain/src/cache/database.rs","lineNumber":3876,"sourceCode":"    ///\n    /// Returns an error if the schema, migration snapshot, historical reconstruction, or retained\n    /// evidence is inconsistent, or persistence fails.\n    pub(crate) async fn ensure_execution_verification_schema(\n        &self,\n        bootstrap: &ExecutionVerificationBootstrap<'_>,\n    ) -> anyhow::Result<()> {\n        let first_header = bootstrap\n            .finalized_headers\n            .first()\n            .ok_or_else(|| anyhow::anyhow!(\"Verified finalized header ledger is empty\"))?;\n        anyhow::ensure!(\n            bootstrap.finalized_headers.windows(2).all(|headers| {\n                headers[1].number == headers[0].number.saturating_add(1)\n                    && headers[1].parent_hash == headers[0].hash\n            }),\n            \"Verified finalized headers are not one continuous parent-linked chain\"\n        );\n        anyhow::ensure!(\n            bootstrap.provider_ids.len() == 3\n                && bootstrap.operator_ids.len() == 3\n                && bootstrap.failure_domain_ids.len() >= 3\n                && !bootstrap.decisions.is_empty(),\n            \"Connect verification evidence is incomplete\"\n        );\n        let chain_id = i32::try_from(bootstrap.chain_id)\n            .context(\"Verification chain ID exceeds PostgreSQL INTEGER\")?;\n        let checkpoint_number = i64::try_from(bootstrap.checkpoint_number)\n            .context(\"Verification checkpoint exceeds PostgreSQL BIGINT\")?;\n        let checkpoint_timestamp = i64::try_from(bootstrap.checkpoint_timestamp)\n            .context(\"Verification checkpoint timestamp exceeds PostgreSQL BIGINT\")?;\n        let next_canonical_nonce = i64::try_from(bootstrap.next_canonical_nonce)\n            .context(\"Canonical nonce exceeds PostgreSQL BIGINT\")?;\n        let observed_canonical_nonce = i64::try_from(bootstrap.observed_canonical_nonce)\n            .context(\"Observed canonical nonce exceeds PostgreSQL BIGINT\")?;\n        let base_fee = bootstrap\n            .checkpoint_base_fee_per_gas","sourceCodeStart":3858,"sourceCodeEnd":3894,"githubUrl":"https://github.com/nautechsystems/nautilus_trader/blob/18893faf8b356be3320add8de2f861b0b647cf06/crates/adapters/blockchain/src/cache/database.rs#L3858-L3894","documentation":"`ensure_execution_verification_schema` requires the bootstrap's connect-verification evidence to be complete: exactly 3 provider IDs, exactly 3 operator IDs, at least 3 failure-domain IDs, and at least one recorded decision. This `anyhow::ensure!` fires when any of those conditions is unmet, meaning the quorum evidence backing the verification ledger is missing or insufficient. The schema's own CHECK constraints (`cardinality(provider_ids) = 3`, etc.) mirror this requirement, so incomplete evidence would be rejected downstream anyway.","triggerScenarios":"Constructing `ExecutionVerificationBootstrap` where `provider_ids.len() != 3`, `operator_ids.len() != 3`, `failure_domain_ids.len() < 3`, or `decisions.is_empty()` — e.g. only one or two providers responded during connect verification, or no decisions were recorded before migration.","commonSituations":"A misconfigured provider list (fewer than 3 RPC endpoints); providers being unreachable during connect verification so quorum never completes; an operator/failure-domain topology not yet fully registered; running migration before any connect-verification decisions have been persisted.","solutions":["Ensure exactly three providers and three operators are configured and reachable, with at least three distinct failure domains, before building the bootstrap.","Complete the connect-verification flow so at least one decision is recorded before invoking `ensure_execution_verification_schema`.","Check provider connectivity — if fewer than three providers respond, the quorum evidence will be short and this error fires.","Review how provider/operator/failure-domain IDs are aggregated into the bootstrap; a filtering bug can silently drop entries below the required counts."],"exampleFix":"// before\nlet bootstrap = ExecutionVerificationBootstrap {\n    provider_ids: &responded_providers,   // maybe only 2 responded\n    operator_ids: &operators,\n    failure_domain_ids: &domains,\n    decisions: &decisions,\n    ..b\n};\n\n// after\nanyhow::ensure!(\n    responded_providers.len() == 3\n        && operators.len() == 3\n        && domains.len() >= 3\n        && !decisions.is_empty(),\n    \"connect verification evidence incomplete before migration\"\n);\nlet bootstrap = ExecutionVerificationBootstrap {\n    provider_ids: &responded_providers,\n    operator_ids: &operators,\n    failure_domain_ids: &domains,\n    decisions: &decisions,\n    ..b\n};","handlingStrategy":"validation","validationCode":"fn validate_connect_evidence(b: &ExecutionVerificationBootstrap<'_>) -> Result<(), String> {\n    match (\n        b.provider_ids.len() == 3,\n        b.operator_ids.len() == 3,\n        b.failure_domain_ids.len() >= 3,\n        !b.decisions.is_empty(),\n    ) {\n        (true, true, true, true) => Ok(()),\n        (p, o, f, d) => Err(format!(\n            \"incomplete evidence: providers={} (need 3), operators={} (need 3), failure_domains>=3={}, decisions_nonempty={}\",\n            b.provider_ids.len(), b.operator_ids.len(), f, d\n        )),\n    }\n}","typeGuard":"fn has_complete_connect_evidence(b: &ExecutionVerificationBootstrap<'_>) -> bool {\n    b.provider_ids.len() == 3\n        && b.operator_ids.len() == 3\n        && b.failure_domain_ids.len() >= 3\n        && !b.decisions.is_empty()\n}","tryCatchPattern":"match database.ensure_execution_verification_schema(&bootstrap).await {\n    Err(e) if e.to_string().contains(\"Connect verification evidence is incomplete\") => {\n        // check provider/operator configuration and connect-verification completion, rebuild evidence, retry\n    }\n    Err(e) => return Err(e),\n    Ok(()) => {}\n}","preventionTips":["Configure exactly three providers and three operators; validate the config at load time.","Gate migration on connect-verification completion — do not run it before at least one decision is recorded.","Health-check all providers before bootstrap so quorum evidence is never short due to unreachable endpoints.","Log provider/operator/failure-domain counts during bootstrap assembly to catch filtering bugs early."],"tags":["blockchain","validation","quorum","configuration"],"backgroundTag":"missing-required-config-field","analyzedSha":"18893faf8b356be3320add8de2f861b0b647cf06","analyzedAt":"2026-09-08T20:49:34.690Z","contentChangedAt":"2026-09-08T20:49:34.690Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}