RyanCodrai/turbovec · critical · ValueError

persisted store is inconsistent with its index: a {what} in

Error message

persisted store is inconsistent with its index: a {what} in the side-car has no vector in the index (internal record id {h}). The .tvim index and its JSON side-car are out of sync.

What it means

Every handle recorded in the JSON side-car must correspond to a vector actually present in the .tvim index. If any side-car handle is absent from the index, the pair is out of sync and reloading would produce records pointing at missing vectors, so check_persisted_handles raises ValueError including the offending internal record id.

Source

Thrown at turbovec-python/python/turbovec/_persist.py:465

            than the index, a handle the index doesn't contain, or a
            watermark below the largest handle in use.
    """
    handle_list = [int(h) for h in handles]
    n_index = len(index)

    if len(set(handle_list)) != len(handle_list):
        raise ValueError(
            f"persisted store is corrupt: duplicate {what} handles in the side-car"
        )
    if len(handle_list) != n_index:
        raise ValueError(
            f"persisted store is inconsistent with its index: side-car has "
            f"{len(handle_list)} {what} handle(s) but the index holds {n_index}. "
            f"The .tvim index and its JSON side-car are out of sync."
        )
    for h in handle_list:
        if not index.contains(h):
            raise ValueError(
                f"persisted store is inconsistent with its index: a {what} in "
                f"the side-car has no vector in the index (internal record id "
                f"{h}). The .tvim index and its JSON side-car are out of sync."
            )
    if next_u64 is not None and handle_list and int(next_u64) < max(handle_list):
        raise ValueError(
            f"persisted store is corrupt: the handle watermark next_u64="
            f"{int(next_u64)} is below the largest {what} handle in use "
            f"({max(handle_list)}). Loading it would reissue live handles "
            f"on the next write."
        )


def check_sidecar_keysets(
    mapping_keys: Iterable,
    sidecar_keys: Iterable,
    *,
    what: str = "entry",

View on GitHub (pinned to ccab9f325e)

Solutions

Restore both .tvim and JSON files from the same consistent backup.
Rebuild the store from source data (re-add all documents) if no backup exists.
If the extra side-car entries are known-dead, remove them and re-run the handle checks (counts, containment, watermark).
Before load, validate containment yourself: `all(index.contains(h) for h in handles)` and fail fast with a clear message.

Example fix

// before
# side-car lists handle 7, index contains only 0..5
store = turbovec.load('store.tvim')
// after
missing = [h for h in handles if not index.contains(h)]
if missing:
    raise RuntimeError(f'restore consistent backup; orphan handles: {missing}')
store = turbovec.load('store.tvim')

Defensive patterns

Strategy: validation

Validate before calling

def handles_covered(index, handles) -> bool:
    return all(index.contains(int(h)) for h in handles)
# verify before turbovec.load

Try / catch

try:
    store = turbovec.load(path)
except ValueError as e:
    if 'no vector in the index' in str(e):
        restore_both_files_from_backup()
    else:
        raise

Prevention

Never delete/compact index vectors except through the library API.
Keep index and side-car from the same save; restore them as a pair.
Run containment validation (index.contains per handle) before loading externally-touched stores.
Back up the pair together with a manifest so partial restores are detectable.

When it happens

Trigger: load / load_from_disk / from_persist_path on a pair where the index was rebuilt, compacted, or partially deleted (dropping vectors) while the side-car still lists their handles; mixed files from two different saves.

Common situations: Deleting vectors from the index file directly or with a tool unaware of the side-car; restoring only the side-car from backup; a compaction/GC step that removed vectors without rewriting the JSON.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/ff48e42356dc92ea. Report an issue: GitHub.