{"record":{"id":"c94418cddddcc51a","repo":"BoundaryML/baml","slug":"baml-cache-verify-cached-test-list-testset-discovery-differs","errorCode":null,"errorMessage":"BAML_CACHE_VERIFY: cached `test --list` testset discovery differs from a fresh discovery ({} vs {} leaf tests). A testset generator is nondeterministic or reads uncached state (IO/env/LLM) — please report this.","messagePattern":"BAML_CACHE_VERIFY: cached `test --list` testset discovery differs from a fresh discovery \\((.+?) vs (.+?) leaf tests\\)\\. A testset generator is nondeterministic or reads uncached state \\(IO/env/LLM\\) — please report this\\.","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"baml_language/crates/baml_cli/src/bytecode_cache.rs","lineNumber":494,"sourceCode":"        // Read the on-disk blob directly, bypassing the `BAML_NO_DISCOVERY_CACHE`\n        // disable (verify must still compare against whatever is on disk).\n        let Some(cached) =\n            self.load_decoded::<TestDiscovery>(&self.test_discovery_key, \"test discovery\")\n        else {\n            return Ok(());\n        };\n        Self::compare_test_discovery(&cached, honest)\n    }\n\n    /// The env-independent core of [`Self::verify_test_discovery`], so the\n    /// oracle's discriminating power (pass on a faithful cache, bail on a stale\n    /// one) is unit-testable without mutating the process environment.\n    pub(crate) fn compare_test_discovery(\n        cached: &TestDiscovery,\n        honest: &TestDiscovery,\n    ) -> anyhow::Result<()> {\n        if cached.testset_leaf_names != honest.testset_leaf_names {\n            anyhow::bail!(\n                \"BAML_CACHE_VERIFY: cached `test --list` testset discovery differs from a fresh \\\n                 discovery ({} vs {} leaf tests). A testset generator is nondeterministic or reads \\\n                 uncached state (IO/env/LLM) — please report this.\",\n                cached.testset_leaf_names.len(),\n                honest.testset_leaf_names.len(),\n            );\n        }\n        Ok(())\n    }\n}\n\n/// Derive and borsh-serialize each stdlib package's `PackageInterface` from a\n/// compiled database, keyed by package name. On a warm database the query\n/// returns the seed verbatim, so re-serializing reproduces the same bytes\n/// (idempotent); on a cold database it materializes the interface once.\nfn extract_stdlib_interface(db: &ProjectDatabase) -> std::collections::BTreeMap<String, Vec<u8>> {\n    use baml_db::baml_compiler2_hir_ty::package_interface::export_interface;\n    let mut out = std::collections::BTreeMap::new();","sourceCodeStart":476,"sourceCodeEnd":512,"githubUrl":"https://github.com/BoundaryML/baml/blob/bd85ce9dee1463ff04d27efd20531013a4ff46c1/baml_language/crates/baml_cli/src/bytecode_cache.rs#L476-L512","documentation":"Error \"BAML_CACHE_VERIFY: cached `test --list` testset discovery differs from a fresh discovery ({} vs {} leaf tests). A testset generator is nondeterministic or reads uncached state (IO/env/LLM) — please report this.\" thrown in BoundaryML/baml.","triggerScenarios":"Thrown at baml_language/crates/baml_cli/src/bytecode_cache.rs:494 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Find the nondeterministic testset generator in your .baml sources and remove hidden state reads (file IO, environment variables, clock, or LLM calls) from generator code","Clear the test discovery cache and regenerate it once the generator is deterministic","If the generator looks clean, report the issue to the BAML maintainers with a minimal reproducible testset — a compile/discovery input may be missing from the cache key"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"bd85ce9dee1463ff04d27efd20531013a4ff46c1","analyzedAt":"2026-09-12T03:38:25.718Z","contentChangedAt":"2026-09-12T03:38:25.718Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}