tursodatabase/turso · error

missing FTS segment data

Error message

missing FTS segment data

What it means

After dumping the FTS index backing rows with FtsBackingRowDumper, index_stats inspects the dumped paths: segment files under `fts2/seg/` count the segments, and chunk files under `fts2/chunk/<segment>/...` accumulate per-segment byte sizes. The ensure! asserts that at least one segment exists AND every counted segment has at least one chunk contributing bytes. If either fails, the dump produced inconsistent or empty FTS segment metadata and memory stats cannot be computed, so it throws "missing FTS segment data".

Source

Thrown at perf/memory/src/fts.rs:418

                turso_core::IOResult::Done(()) => break,
                turso_core::IOResult::IO(completions) => {
                    while !completions.finished() {
                        io.step()?;
                    }
                }
            }
        }
        let mut sizes = std::collections::BTreeMap::<&str, usize>::new();
        let mut segments = 0;
        for (path, _, bytes, _) in &dumper.rows {
            if let Some(suffix) = path.strip_prefix("fts2/chunk/") {
                let (segment, _) = suffix.split_once('/').expect("segment chunk path");
                *sizes.entry(segment).or_default() += bytes;
            } else if path.starts_with("fts2/seg/") {
                segments += 1;
            }
        }
        ensure!(
            segments > 0 && sizes.len() == segments,
            "missing FTS segment data"
        );
        let mut page_size = 0;
        conn.query("PRAGMA page_size")?
            .expect("page size query")
            .run_with_row_callback(|row| {
                page_size = row.get::<i64>(0)? as usize;
                Ok(())
            })?;
        let stats = IndexStats {
            segment_bytes: sizes.values().sum(),
            segments,
            largest_segment_bytes: *sizes.values().max().unwrap(),
            page_size,
            configured_cache_pages: cache_pages,
        };
        drop(dumper);

View on GitHub (pinned to 492c4a71cd)

Solutions

1. Verify the fixture was created with documents > 0 and the FTS index (docs_fts) actually exists and was built (FtsFixture::create already checks queries use the index).
2. Print/inspect dumper.rows paths to see what the index actually contains; if paths no longer match `fts2/seg/` and `fts2/chunk/<segment>/...`, update index_stats to the new internal layout after an engine change.
3. Make sure index_stats opens the same fts.db directory the fixture wrote to (self.directory.path().join("fts.db")) and with_index_method(true) is enabled.
4. Reproduce with the known-good corpus from the test (1000 or 1001 documents) to confirm the segment flush behavior before trusting custom configurations.

Example fix

// before (no docs -> no segments)
let fixture = FtsFixture::create(0).await?;
let stats = fixture.index_stats(None)?;
// after
let fixture = FtsFixture::create(1000).await?;
let stats = fixture.index_stats(None)?;

// before calling index_stats, confirm the index exists and has rows
let fixture = FtsFixture::create(documents).await?; // create() runs check_index for all cases
assert!(documents > 0, "index_stats needs a populated FTS index");
let stats = fixture.index_stats(None)?;

fn has_segments(rows: &[(String, u64, usize, u64)]) -> bool {
    let segs = rows.iter().filter(|(p, ..)| p.starts_with("fts2/seg/")).count();
    let sized = rows.iter().filter(|(p, ..)| p.starts_with("fts2/chunk/")).count();
    segs > 0 && sized >= segs
}

match fixture.index_stats(cache_pages) {
    Ok(stats) => stats,
    Err(e) if e.to_string().contains("missing FTS segment data") => {
        eprintln!("FTS index was not flushed to segments; recreate the fixture with documents > 0");
        return Err(e);
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling FtsFixture::index_stats on a database where the FTS index `docs_fts` has no segments (index never populated, rows flushed differently than expected, or `fts2/seg/` rows absent from the dumper), or where segment chunk paths under `fts2/chunk/` don't line up 1:1 with counted segments (e.g. sizes.len() != segments because a segment has no chunk data or an unexpected path layout produced a duplicate/extra segment key).

Common situations: Running the memory benchmark against a fixture created with a changed/empty corpus so the index was never flushed to segment files; a Turso engine change renaming or reshaping the internal `fts2/seg/` / `fts2/chunk/` path layout; opening the wrong database file (e.g. path changed) so the dumper sees no index rows; running index_stats before any documents are committed.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

Related errors

• index contains {} segment bytes, below requested minimum {};
• transactions must overlap before queries start
• documents must be positive
• connections must be positive
• FTS index not used for {case:?}: {details:?}

AI-assisted analysis of tursodatabase/turso@492c4a71cd (2026-09-13). Data as JSON: /api/errors/a19500183964e4af. Report an issue: GitHub.