RyanCodrai/turbovec · critical · ValueError

persisted store is inconsistent with its index: side-car has

Error message

persisted store is inconsistent with its index: side-car has {len(handle_list)} {what} handle(s) but the index holds {n_index}. The .tvim index and its JSON side-car are out of sync.

What it means

The .tvim index and the JSON side-car are two halves of one persisted store; the side-car must hold exactly one handle per vector in the index. When the counts differ, the two files are out of sync (one was written, edited, truncated, or replaced without the other), so loading is aborted with ValueError.

Source

Thrown at turbovec-python/python/turbovec/_persist.py:458

        next_u64: the side-car's handle watermark, if the caller has it.
            Handles are issued by pre-incrementing it, so it must be at
            least the largest handle in use; a smaller value reissues live
            handles on the next write (issue #321).

    Raises:
        ValueError: if the side-car has duplicate handles, a different count
            than the index, a handle the index doesn't contain, or a
            watermark below the largest handle in use.
    """
    handle_list = [int(h) for h in handles]
    n_index = len(index)

    if len(set(handle_list)) != len(handle_list):
        raise ValueError(
            f"persisted store is corrupt: duplicate {what} handles in the side-car"
        )
    if len(handle_list) != n_index:
        raise ValueError(
            f"persisted store is inconsistent with its index: side-car has "
            f"{len(handle_list)} {what} handle(s) but the index holds {n_index}. "
            f"The .tvim index and its JSON side-car are out of sync."
        )
    for h in handle_list:
        if not index.contains(h):
            raise ValueError(
                f"persisted store is inconsistent with its index: a {what} in "
                f"the side-car has no vector in the index (internal record id "
                f"{h}). The .tvim index and its JSON side-car are out of sync."
            )
    if next_u64 is not None and handle_list and int(next_u64) < max(handle_list):
        raise ValueError(
            f"persisted store is corrupt: the handle watermark next_u64="
            f"{int(next_u64)} is below the largest {what} handle in use "
            f"({max(handle_list)}). Loading it would reissue live handles "
            f"on the next write."
        )

View on GitHub (pinned to ccab9f325e)

Solutions

  1. Treat the pair as atomic: restore both files from the same backup/save point.
  2. Rebuild the store from source data if no consistent backup exists.
  3. If you know which file is stale, regenerate it: re-export the side-car from the index or vice versa with the writing turbovec version.
  4. Before loading, sanity-check: `len(handles) == index_size`; if not, abort and surface a user-facing corruption message.

Example fix

// before
# index holds 10 vectors, side-car has 8 handles -> load
store = turbovec.load('store.tvim')
// after
# restore matching pair first, or guard:
if len(sidecar['handles']) != index_size:
    raise RuntimeError('store pair out of sync; restore both files')
store = turbovec.load('store.tvim')
Defensive patterns

Strategy: validation

Validate before calling

def pair_in_sync(index_size: int, handles) -> bool:
    return len(list(handles)) == index_size
# compare with the .tvim index record count before loading

Try / catch

try:
    store = turbovec.load(path)
except ValueError as e:
    if 'out of sync' in str(e):
        restore_both_files_from_backup()
    else:
        raise

Prevention

When it happens

Trigger: load / load_from_disk / from_persist_path on a pair where the side-car handle array length != number of records in the index: e.g. index rebuilt/saved without updating the side-car, side-car truncated by a crashed write, or a mismatched pair of files from different saves.

Common situations: Copying only one of the two files to another machine; an interrupted atomic_save from an older version; manually pruning index vectors without touching the side-car; restoring one file from an older backup.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/1098b79c0d4ac7ac. Report an issue: GitHub.