{"record":{"id":"a19500183964e4af","repo":"tursodatabase/turso","slug":"missing-fts-segment-data","errorCode":null,"errorMessage":"missing FTS segment data","messagePattern":"missing FTS segment data","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"perf/memory/src/fts.rs","lineNumber":418,"sourceCode":"                turso_core::IOResult::Done(()) => break,\n                turso_core::IOResult::IO(completions) => {\n                    while !completions.finished() {\n                        io.step()?;\n                    }\n                }\n            }\n        }\n        let mut sizes = std::collections::BTreeMap::<&str, usize>::new();\n        let mut segments = 0;\n        for (path, _, bytes, _) in &dumper.rows {\n            if let Some(suffix) = path.strip_prefix(\"fts2/chunk/\") {\n                let (segment, _) = suffix.split_once('/').expect(\"segment chunk path\");\n                *sizes.entry(segment).or_default() += bytes;\n            } else if path.starts_with(\"fts2/seg/\") {\n                segments += 1;\n            }\n        }\n        ensure!(\n            segments > 0 && sizes.len() == segments,\n            \"missing FTS segment data\"\n        );\n        let mut page_size = 0;\n        conn.query(\"PRAGMA page_size\")?\n            .expect(\"page size query\")\n            .run_with_row_callback(|row| {\n                page_size = row.get::<i64>(0)? as usize;\n                Ok(())\n            })?;\n        let stats = IndexStats {\n            segment_bytes: sizes.values().sum(),\n            segments,\n            largest_segment_bytes: *sizes.values().max().unwrap(),\n            page_size,\n            configured_cache_pages: cache_pages,\n        };\n        drop(dumper);","sourceCodeStart":400,"sourceCodeEnd":436,"githubUrl":"https://github.com/tursodatabase/turso/blob/492c4a71cd7c2649e7df83da1471b74f4b1c7aa9/perf/memory/src/fts.rs#L400-L436","documentation":"After dumping the FTS index backing rows with FtsBackingRowDumper, index_stats inspects the dumped paths: segment files under `fts2/seg/` count the segments, and chunk files under `fts2/chunk/<segment>/...` accumulate per-segment byte sizes. The ensure! asserts that at least one segment exists AND every counted segment has at least one chunk contributing bytes. If either fails, the dump produced inconsistent or empty FTS segment metadata and memory stats cannot be computed, so it throws \"missing FTS segment data\".","triggerScenarios":"Calling FtsFixture::index_stats on a database where the FTS index `docs_fts` has no segments (index never populated, rows flushed differently than expected, or `fts2/seg/` rows absent from the dumper), or where segment chunk paths under `fts2/chunk/` don't line up 1:1 with counted segments (e.g. sizes.len() != segments because a segment has no chunk data or an unexpected path layout produced a duplicate/extra segment key).","commonSituations":"Running the memory benchmark against a fixture created with a changed/empty corpus so the index was never flushed to segment files; a Turso engine change renaming or reshaping the internal `fts2/seg/` / `fts2/chunk/` path layout; opening the wrong database file (e.g. path changed) so the dumper sees no index rows; running index_stats before any documents are committed.","solutions":["Verify the fixture was created with documents > 0 and the FTS index (docs_fts) actually exists and was built (FtsFixture::create already checks queries use the index).","Print/inspect dumper.rows paths to see what the index actually contains; if paths no longer match `fts2/seg/` and `fts2/chunk/<segment>/...`, update index_stats to the new internal layout after an engine change.","Make sure index_stats opens the same fts.db directory the fixture wrote to (self.directory.path().join(\"fts.db\")) and with_index_method(true) is enabled.","Reproduce with the known-good corpus from the test (1000 or 1001 documents) to confirm the segment flush behavior before trusting custom configurations."],"exampleFix":"// before (no docs -> no segments)\nlet fixture = FtsFixture::create(0).await?;\nlet stats = fixture.index_stats(None)?;\n// after\nlet fixture = FtsFixture::create(1000).await?;\nlet stats = fixture.index_stats(None)?;","handlingStrategy":"validation","validationCode":"// before calling index_stats, confirm the index exists and has rows\nlet fixture = FtsFixture::create(documents).await?; // create() runs check_index for all cases\nassert!(documents > 0, \"index_stats needs a populated FTS index\");\nlet stats = fixture.index_stats(None)?;","typeGuard":"fn has_segments(rows: &[(String, u64, usize, u64)]) -> bool {\n    let segs = rows.iter().filter(|(p, ..)| p.starts_with(\"fts2/seg/\")).count();\n    let sized = rows.iter().filter(|(p, ..)| p.starts_with(\"fts2/chunk/\")).count();\n    segs > 0 && sized >= segs\n}","tryCatchPattern":"match fixture.index_stats(cache_pages) {\n    Ok(stats) => stats,\n    Err(e) if e.to_string().contains(\"missing FTS segment data\") => {\n        eprintln!(\"FTS index was not flushed to segments; recreate the fixture with documents > 0\");\n        return Err(e);\n    }\n    Err(e) => return Err(e),\n}","preventionTips":["Always build the fixture via FtsFixture::create with a positive document count before calling index_stats.","After any engine change to FTS internals, diff the dumped `fts2/...` paths against the prefixes expected in index_stats.","Open the same fts.db the fixture wrote to, with the index-method feature enabled, so the dumper sees the index rows."],"tags":["fts","benchmark","internal-invariant"],"backgroundTag":"empty-result-set","analyzedSha":"492c4a71cd7c2649e7df83da1471b74f4b1c7aa9","analyzedAt":"2026-09-13T18:13:59.796Z","contentChangedAt":"2026-09-13T18:13:59.796Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}