{"record":{"id":"1417968a6ee28e64","repo":"tursodatabase/turso","slug":"index-contains-segment-bytes-below-requested-minimum","errorCode":null,"errorMessage":"index contains {} segment bytes, below requested minimum {}; increase documents or extra tokens","messagePattern":"index contains (.+?) segment bytes, below requested minimum (.+?); increase documents or extra tokens","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"perf/memory/src/fts.rs","lineNumber":132,"sourceCode":"    pub id_sum: i64,\n}\n\npub async fn run_fts(config: FtsConfig, observer: &mut dyn FtsObserver) -> Result<RunResult> {\n    let workload = FtsWorkload::prepare(config, observer).await?;\n    let result = workload.run(observer).await;\n    workload.finish(observer);\n    result\n}\n\nimpl FtsWorkload {\n    pub async fn prepare(config: FtsConfig, observer: &mut dyn FtsObserver) -> Result<Self> {\n        config.validate()?;\n        observer.on_phase(FtsPhase::Setup);\n        let fixture =\n            FtsFixture::create_in_mode(config.documents, config.mode, config.corpus).await?;\n        let stats = fixture.index_stats(config.corpus.cache_pages)?;\n        observer.on_index(&stats);\n        ensure!(\n            stats.segment_bytes >= config.corpus.min_index_bytes,\n            \"index contains {} segment bytes, below requested minimum {}; increase documents or extra tokens\",\n            stats.segment_bytes,\n            config.corpus.min_index_bytes\n        );\n        observer.on_phase(FtsPhase::Open);\n        let first = fixture.open().await?;\n        let mut sessions = Vec::with_capacity(config.connections);\n        for _ in 1..config.connections {\n            sessions.push(QuerySession {\n                conn: first._db.connect()?,\n                _db: first._db.clone(),\n            });\n        }\n        sessions.push(first);\n        if let Some(pages) = config.corpus.cache_pages {\n            for session in &sessions {\n                session","sourceCodeStart":114,"sourceCodeEnd":150,"githubUrl":"https://github.com/tursodatabase/turso/blob/492c4a71cd7c2649e7df83da1471b74f4b1c7aa9/perf/memory/src/fts.rs#L114-L150","documentation":"During FTS benchmark preparation, after indexing the corpus the tool checks that the resulting segment size meets the user-requested minimum (corpus.min_index_bytes). If the generated documents produced fewer index bytes than requested, the benchmark cannot measure memory at the requested scale, so it aborts. This is a configuration-vs-reality mismatch, not a bug in the indexer.","triggerScenarios":"Running the FTS memory benchmark with corpus.min_index_bytes set larger than the index produced by config.documents documents (plus any extra tokens). E.g. --min-index-bytes 50000000 with only a few hundred documents.","commonSituations":"Users raise min_index_bytes to benchmark larger working sets but forget to scale up the document count; changed corpus mode or tokenization produces smaller segments than before; copied a config from a different corpus shape.","solutions":["Increase config.documents so the indexed corpus produces at least min_index_bytes of segment data","Add extra tokens to the document fixture to grow per-document index size","Lower corpus.min_index_bytes to match what the current document count realistically produces"],"exampleFix":"// before\nFtsConfig { documents: 500, corpus: CorpusConfig { min_index_bytes: 50_000_000, .. } }\n// after\nFtsConfig { documents: 50_000, corpus: CorpusConfig { min_index_bytes: 50_000_000, .. } }","handlingStrategy":"validation","validationCode":"let stats = fixture.index_stats(cache_pages).await?;\nif stats.segment_bytes < config.corpus.min_index_bytes {\n    return Err(anyhow!(\"need >= {} segment bytes, got {}\", config.corpus.min_index_bytes, stats.segment_bytes));\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pick documents/min_index_bytes pairs you have verified empirically once","Scale documents proportionally when raising min_index_bytes","Log index_stats after indexing and before asserting the minimum"],"tags":["benchmark","fts","configuration","validation"],"backgroundTag":"invalid-config-value","analyzedSha":"492c4a71cd7c2649e7df83da1471b74f4b1c7aa9","analyzedAt":"2026-09-13T18:13:59.796Z","contentChangedAt":"2026-09-13T18:13:59.796Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}