{"record":{"id":"5f7795ddfe12ffd4","repo":"RyanCodrai/turbovec","slug":"file-too-large-for-this-platform","errorCode":null,"errorMessage":"file too large for this platform","messagePattern":"file too large for this platform","errorType":"validation","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"turbovec/src/io_v7.rs","lineNumber":1222,"sourceCode":"\n/// Load a v7 file. Blocks carry no checksums — the commit's delta\n/// digest covers every unit at the sync that wrote it, which is all\n/// the crash protocol needs; detecting later external damage is out of\n/// scope, as it is for v6.\npub(crate) fn load(path: &Path, expect_calib_gen: u64, expect_kind: u8) -> io::Result<V7Load> {\n    // Allocate for what the file DECLARES, capped by what it actually\n    // holds — never for its apparent length. An image's size is fixed by\n    // its geometry and row count, so a padded or sparse file is bytes\n    // nobody asked for, and reading it wholesale made memory track the\n    // file rather than the index. That is the v6 defect #487 fixed, and\n    // v7 inherited it when it became the only format. Trailing bytes are\n    // ignored, as they were before.\n    let f = File::open(path)?;\n    let on_disk = f.metadata()?.len();\n    let want = declared_len(&f).unwrap_or(on_disk).min(on_disk);\n    let mut raw = vec![\n        0u8;\n        usize::try_from(want).map_err(|_| io::Error::new(\n            io::ErrorKind::InvalidData,\n            \"file too large for this platform\"\n        ))?\n    ];\n    crate::io::read_exact_at(&f, &mut raw, 0)?;\n    load_image(raw, expect_calib_gen, expect_kind, &path.display().to_string())\n}\n\n/// Bytes a complete image of this file's geometry and row count occupies,\n/// read from the superblock and the two header slots.\n///\n/// `None` whenever the prefix is not self-consistent: the parser then\n/// sees the whole file and produces its own, better message rather than\n/// a truncation artefact of a guess made here.\nfn declared_len(f: &File) -> Option<u64> {\n    let mut sb = [0u8; 64];\n    crate::io::read_exact_at(f, &mut sb, 0).ok()?;\n    if &sb[0..4] != V7_MAGIC || sb[4] != V7_VERSION {","sourceCodeStart":1204,"sourceCodeEnd":1240,"githubUrl":"https://github.com/RyanCodrai/turbovec/blob/ccab9f325e6ce2a270a87daf01ae4e443bcf2d49/turbovec/src/io_v7.rs#L1204-L1240","documentation":"When opening a v7 file the loader sizes a Vec<u8> from the declared length clamped to the on-disk size; if that byte count exceeds usize::MAX on the current platform the conversion fails and this InvalidData error is thrown. It means the index file cannot be addressed in a single in-memory buffer on this architecture.","triggerScenarios":"usize::try_from(want) failing because declared_len (or the on-disk size) exceeds the platform's addressable range — practically only on 32-bit targets or absurdly large files.","commonSituations":"Loading a multi-gigabyte index on a 32-bit build; a corrupt header declaring a bogus huge length.","solutions":["Use a 64-bit build of turbovec for very large indexes","Split the index into smaller shards that fit the platform limit","Verify declared_len in the header is sane; regenerate the file if corrupt","Stream/load in chunks if the library offers a streaming loader"],"exampleFix":"// before\n// 32-bit target loading a 5 GB index -> 'file too large for this platform'\nlet idx = Index::load_v7(\"huge.tv\")?;\n// after\n// build for 64-bit: cargo build --target x86_64-unknown-linux-gnu\nlet idx = Index::load_v7(\"huge.tv\")?;","handlingStrategy":"validation","validationCode":"let len = std::fs::metadata(path)?.len();\nif len > usize::MAX as u64 {\n    return Err(anyhow!(\"index too large for this (32-bit?) platform\"));\n}","typeGuard":"fn fits_platform(path: &Path) -> bool {\n    std::fs::metadata(path)\n        .map(|m| u64::from(usize::try_from(m.len()).unwrap_or(usize::MAX)) == m.len())\n        .unwrap_or(false)\n}","tryCatchPattern":"match Index::load_v7(path) {\n    Err(e) if e.to_string().contains(\"file too large for this platform\") => {\n        eprintln!(\"switch to a 64-bit build or shard the index\");\n        Err(e)\n    }\n    other => other,\n}","preventionTips":["Use 64-bit builds for production index loading","Shard very large indexes into per-shard files","Validate declared lengths in headers as a corruption check"],"tags":["io","platform-limit","file-size"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"ccab9f325e6ce2a270a87daf01ae4e443bcf2d49","analyzedAt":"2026-09-06T08:39:18.516Z","contentChangedAt":"2026-09-06T08:39:18.516Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}