RyanCodrai/turbovec · error · io::Error

duplicate ids in v7 file

Error message

duplicate ids in v7 file

What it means

from_v7_load validates ids read from a version-7 persisted file: after TurboQuantIndex::from_v7 loads the index, the id vector is sorted and checked for adjacent duplicates. Duplicates mean the file's id table is corrupt or was written by buggy code, so loading fails with an InvalidData io::Error rather than creating an ambiguous map.

Source

Thrown at turbovec/src/id_map.rs:934

        }
        Ok(Self {
            inner,
            slot_to_id,
            id_to_slot: std::sync::OnceLock::new(),
            sorted_ids: std::sync::Mutex::new(sorted),
            deferred_added: std::sync::Mutex::new(Default::default()),
        })
    }

    /// Shared tail of the path and byte v7 loaders. `path` is `None` for
    /// a byte image, which is not a sync destination.
    fn from_v7_load(mut l: crate::io_v7::V7Load, path: Option<&Path>) -> std::io::Result<Self> {
        let slot_to_id = std::mem::take(&mut l.ids);
        let inner = TurboQuantIndex::from_v7(l, path)?;
        let mut sorted = slot_to_id.clone();
        sorted.sort_unstable();
        if sorted.windows(2).any(|w| w[0] == w[1]) {
            return Err(std::io::Error::new(
                std::io::ErrorKind::InvalidData,
                "duplicate ids in v7 file",
            ));
        }
        Ok(Self {
            inner,
            slot_to_id,
            id_to_slot: std::sync::OnceLock::new(),
            sorted_ids: std::sync::Mutex::new(sorted),
            deferred_added: std::sync::Mutex::new(Default::default()),
        })
    }

    /// Serialize the index in the `.tvim` byte format to any
    /// [`std::io::Write`] sink. Emits exactly the bytes [`Self::write`]
    /// would put in the file.
    ///
    /// Unlike [`Self::write`] there is no atomic-replace behaviour: the

View on GitHub (pinned to ccab9f325e)

Solutions

1. Re-export the store in a current version from source data so the id table is written correctly.
2. Locate the duplicate ids (sort + compare adjacent) to identify the corrupted artifact, then restore from backup.
3. If migrating from an old version, run a dedup/repair pass on the ids before load and rebuild the file with the current writer.

Example fix

// before
let map = IdMap::from_v7_load(v7_load, Some(path))?; // Err: duplicate ids in v7 file
// after
let mut sorted = v7_load.ids.clone(); sorted.sort_unstable();
if sorted.windows(2).any(|w| w[0] == w[1]) { /* repair or restore file first */ }
let map = IdMap::from_v7_load(v7_load, Some(path))?;

let mut s = l.ids.clone(); s.sort_unstable();
if s.windows(2).any(|w| w[0] == w[1]) {
    return Err(io::Error::new(io::ErrorKind::InvalidData, "v7 ids not unique"));
}

let map = IdMap::from_v7_load(l, Some(path)).map_err(|e| {
    if e.kind() == io::ErrorKind::InvalidData { LoadError::CorruptV7 } else { LoadError::Io(e) }
})?;

When it happens

Trigger: Loading a v7 file whose stored ids vector contains the same id twice — produced by a writer bug, corrupted/truncated-then-hand-edited file, or an ids artifact from a different build mixed in.

Common situations: Reading v7 files produced by an older/buggy turbovec version; files transferred or edited manually; mixing index and id side-files from different saves.

Related errors

• duplicate ids
• persisted store is corrupt: duplicate {what} handles in the
• persisted store is inconsistent with its index: side-car has
• persisted store is inconsistent with its index: a {what} in
• persisted store is corrupt: the handle watermark next_u64={i

AI-assisted analysis of RyanCodrai/turbovec@ccab9f325e (2026-09-06). Data as JSON: /api/errors/b60e1544a26cdb67. Report an issue: GitHub.