{"record":{"id":"ff48e42356dc92ea","repo":"RyanCodrai/turbovec","slug":"persisted-store-is-inconsistent-with-its-index-a","errorCode":null,"errorMessage":"persisted store is inconsistent with its index: a {what} in the side-car has no vector in the index (internal record id {h}). The .tvim index and its JSON side-car are out of sync.","messagePattern":"persisted store is inconsistent with its index: a (.+?) in the side-car has no vector in the index \\(internal record id (.+?)\\)\\. The \\.tvim index and its JSON side-car are out of sync\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"turbovec-python/python/turbovec/_persist.py","lineNumber":465,"sourceCode":"            than the index, a handle the index doesn't contain, or a\n            watermark below the largest handle in use.\n    \"\"\"\n    handle_list = [int(h) for h in handles]\n    n_index = len(index)\n\n    if len(set(handle_list)) != len(handle_list):\n        raise ValueError(\n            f\"persisted store is corrupt: duplicate {what} handles in the side-car\"\n        )\n    if len(handle_list) != n_index:\n        raise ValueError(\n            f\"persisted store is inconsistent with its index: side-car has \"\n            f\"{len(handle_list)} {what} handle(s) but the index holds {n_index}. \"\n            f\"The .tvim index and its JSON side-car are out of sync.\"\n        )\n    for h in handle_list:\n        if not index.contains(h):\n            raise ValueError(\n                f\"persisted store is inconsistent with its index: a {what} in \"\n                f\"the side-car has no vector in the index (internal record id \"\n                f\"{h}). The .tvim index and its JSON side-car are out of sync.\"\n            )\n    if next_u64 is not None and handle_list and int(next_u64) < max(handle_list):\n        raise ValueError(\n            f\"persisted store is corrupt: the handle watermark next_u64=\"\n            f\"{int(next_u64)} is below the largest {what} handle in use \"\n            f\"({max(handle_list)}). Loading it would reissue live handles \"\n            f\"on the next write.\"\n        )\n\n\ndef check_sidecar_keysets(\n    mapping_keys: Iterable,\n    sidecar_keys: Iterable,\n    *,\n    what: str = \"entry\",","sourceCodeStart":447,"sourceCodeEnd":483,"githubUrl":"https://github.com/RyanCodrai/turbovec/blob/ccab9f325e6ce2a270a87daf01ae4e443bcf2d49/turbovec-python/python/turbovec/_persist.py#L447-L483","documentation":"Every handle recorded in the JSON side-car must correspond to a vector actually present in the .tvim index. If any side-car handle is absent from the index, the pair is out of sync and reloading would produce records pointing at missing vectors, so check_persisted_handles raises ValueError including the offending internal record id.","triggerScenarios":"load / load_from_disk / from_persist_path on a pair where the index was rebuilt, compacted, or partially deleted (dropping vectors) while the side-car still lists their handles; mixed files from two different saves.","commonSituations":"Deleting vectors from the index file directly or with a tool unaware of the side-car; restoring only the side-car from backup; a compaction/GC step that removed vectors without rewriting the JSON.","solutions":["Restore both .tvim and JSON files from the same consistent backup.","Rebuild the store from source data (re-add all documents) if no backup exists.","If the extra side-car entries are known-dead, remove them and re-run the handle checks (counts, containment, watermark).","Before load, validate containment yourself: `all(index.contains(h) for h in handles)` and fail fast with a clear message."],"exampleFix":"// before\n# side-car lists handle 7, index contains only 0..5\nstore = turbovec.load('store.tvim')\n// after\nmissing = [h for h in handles if not index.contains(h)]\nif missing:\n    raise RuntimeError(f'restore consistent backup; orphan handles: {missing}')\nstore = turbovec.load('store.tvim')","handlingStrategy":"validation","validationCode":"def handles_covered(index, handles) -> bool:\n    return all(index.contains(int(h)) for h in handles)\n# verify before turbovec.load","typeGuard":null,"tryCatchPattern":"try:\n    store = turbovec.load(path)\nexcept ValueError as e:\n    if 'no vector in the index' in str(e):\n        restore_both_files_from_backup()\n    else:\n        raise","preventionTips":["Never delete/compact index vectors except through the library API.","Keep index and side-car from the same save; restore them as a pair.","Run containment validation (index.contains per handle) before loading externally-touched stores.","Back up the pair together with a manifest so partial restores are detectable."],"tags":["python","persistence","corruption","consistency"],"backgroundTag":"checksum-mismatch","analyzedSha":"ccab9f325e6ce2a270a87daf01ae4e443bcf2d49","analyzedAt":"2026-09-06T08:39:18.516Z","contentChangedAt":"2026-09-06T08:39:18.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}