{"record":{"id":"4b31d2e6f1e4b2ad","repo":"tracel-ai/burn","slug":"invaliddata","errorCode":"InvalidData","errorMessage":"Storage boundaries not available for key '{}'. Cannot perform lazy loading.","messagePattern":"Storage boundaries not available for key '(.+?)'\\. Cannot perform lazy loading\\.","errorType":"error_code","errorClass":"std::io::Error","httpStatus":null,"severity":"error","filePath":"crates/burn-store/src/pytorch/lazy_data.rs","lineNumber":267,"sourceCode":"            .storage_map\n            .read()\n            .unwrap_or_else(|poisoned| poisoned.into_inner());\n\n        if let Some(ref map) = *storage_map\n            && let Some(&(offset, size)) = map.get(storage_key)\n        {\n            // Load only this specific storage\n            let mut file = File::open(&self.path)?;\n            file.seek(std::io::SeekFrom::Start(self.data_offset + offset))?;\n\n            let mut buffer = vec![0u8; size as usize];\n            file.read_exact(&mut buffer)?;\n            return Ok(buffer);\n        }\n\n        // NO FALLBACK! If we don't have storage boundaries, we cannot load data lazily\n        // The storage map MUST be built from tensor metadata for lazy loading to work\n        Err(std::io::Error::new(\n            std::io::ErrorKind::InvalidData,\n            format!(\n                \"Storage boundaries not available for key '{}'. Cannot perform lazy loading.\",\n                storage_key\n            ),\n        ))\n    }\n}\n\nimpl TarSource {\n    /// Create a new TAR source by parsing storages data.\n    ///\n    /// # Arguments\n    /// * `storages_data` - Raw storages blob with structure:\n    ///   - Count pickle (number of storages)\n    ///   - For each storage: metadata pickle + u64 num_elements + raw binary data\n    pub fn new(storages_data: Vec<u8>) -> std::io::Result<Self> {\n        use super::pickle_reader::{read_pickle, storage_type_to_element_size};","sourceCodeStart":249,"sourceCodeEnd":285,"githubUrl":"https://github.com/tracel-ai/burn/blob/d16f7ba2ed0d41408189384044cc886fb4c8f957/crates/burn-store/src/pytorch/lazy_data.rs#L249-L285","documentation":"Lazy loading in burn's PyTorch record reader requires a precomputed map of storage boundaries (byte offset and size per storage key) built from tensor metadata. `LazyData::read` deliberately has no fallback: if the key is absent from that map, it cannot know where the tensor bytes live in the file, so it raises InvalidData rather than reading the whole file eagerly.","triggerScenarios":"Calling `LazyData::read(key)` (via lazy record loading of a .pt/.pth file) with a storage key that was not present when the storage map was built — e.g. the metadata pickle lacked an entry for that tensor, the map was never initialized, or the key's path suffix (`data/N`) does not match any recorded storage.","commonSituations":"Loading a PyTorch checkpoint saved by an unusual producer (older/newer torch, quantized or sparse tensors) whose storage metadata doesn't line up; requesting a tensor key that doesn't exist in the file; using lazy loading on a file where zip metadata parsing partially failed.","solutions":["Verify the key exists in the file's tensor metadata before lazy-reading it (list keys from the record/reader first)","Ensure the map builder ran successfully — re-open the file so storage boundaries are built from metadata","Fall back to eager (non-lazy) loading of the record, which reads storages without needing the boundary map","Check the file was produced by a supported torch.save format and is not truncated"],"exampleFix":"// before\nlet data = lazy_data.read(\"data/42\")?; // panics with InvalidData if unmapped\n// after\nif lazy_data.has_storage(\"data/42\") {\n    let data = lazy_data.read(\"data/42\")?;\n} else {\n    let record = full_loader.load(&path)?; // eager fallback\n}","handlingStrategy":"try-catch","validationCode":"fn can_lazy_read(src: &LazyData, key: &str) -> bool {\n    let storage_key = key.split('/').next_back().unwrap_or(key);\n    src.has_storage(storage_key)\n}","typeGuard":null,"tryCatchPattern":"match lazy_data.read(key) {\n    Ok(bytes) => use_bytes(bytes),\n    Err(e) if e.kind() == std::io::ErrorKind::InvalidData\n        && e.to_string().contains(\"Storage boundaries not available\") =>\n    {\n        let record = eager_loader.load(path)?; // non-lazy fallback\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Validate keys against the record's tensor metadata before lazy reads","Re-open files cleanly so the storage map is rebuilt from metadata","Prefer eager loading for checkpoints from unusual torch versions","Never reuse a LazyData handle after the underlying file changed on disk"],"tags":["io","pytorch","lazy-loading","invalid-data"],"backgroundTag":"storage-boundaries-unavailable","analyzedSha":"d16f7ba2ed0d41408189384044cc886fb4c8f957","analyzedAt":"2026-09-05T13:19:14.260Z","contentChangedAt":"2026-09-05T13:19:14.260Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}