{"record":{"id":"2efdacc126a7959b","repo":"quickwit-oss/tantivy","slug":"error-when-reading-block-in-doc-store","errorCode":null,"errorMessage":"error when reading block in doc store","messagePattern":"error when reading block in doc store","errorType":"exception","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"src/store/reader.rs","lineNumber":345,"sourceCode":"                    .unwrap_or(true);\n                let res = if alive {\n                    Some((curr_block.clone(), doc_pos))\n                } else {\n                    None\n                };\n                doc_pos += 1;\n                res\n            })\n            .map(move |(block, doc_pos)| {\n                let block = block\n                    .ok_or_else(|| {\n                        DataCorruption::comment_only(\n                            \"the current checkpoint in the doc store iterator is none, this \\\n                             should never happen\",\n                        )\n                    })?\n                    .map_err(|error_kind| {\n                        std::io::Error::new(error_kind, \"error when reading block in doc store\")\n                    })?;\n\n                let range = block_read_index(&block, doc_pos)?;\n                Ok(block.slice(range))\n            })\n    }\n\n    /// Summarize total space usage of this store reader.\n    pub fn space_usage(&self) -> StoreSpaceUsage {\n        self.space_usage.clone()\n    }\n}\n\nfn block_read_index(block: &[u8], doc_pos: u32) -> crate::Result<Range<usize>> {\n    let doc_pos = doc_pos as usize;\n    let size_of_u32 = std::mem::size_of::<u32>();\n\n    let index_len_pos = block.len() - size_of_u32;","sourceCodeStart":327,"sourceCodeEnd":363,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/src/store/reader.rs#L327-L363","documentation":"Thrown in iter_raw() when reading a doc store block fails; the underlying io::ErrorKind from the block read is preserved but the message is replaced with the generic \"error when reading block in doc store\". It is reached through iter() while streaming raw blocks, so any block-read failure (missing block, IO error, corruption surfaced as an error kind) ends up here.","triggerScenarios":"Iterating doc store blocks via iter()/iter_raw() when the underlying block lookup returns an Err(io::ErrorKind), e.g. reading past the last block, a checkpoint pointing at a nonexistent block, or IO failures on the underlying file.","commonSituations":"Iterating a doc store whose checkpoint/block index is inconsistent (e.g. after a failed commit or partially deleted segment); disk/permission errors while streaming; consuming an iterator beyond the valid range of a corrupted segment.","solutions":["Log the io::ErrorKind (preserved in error.kind()) to identify the root cause before the message is flattened","Restore or rebuild the corrupted segment; verify checkpoints match existing blocks","Check file permissions and disk health on the index directory","Re-open the index fresh to rule out stale in-memory checkpoint state"],"exampleFix":"// before\nlet block = checkpoint.block_addr.map_err(|error_kind| std::io::Error::new(error_kind, \"error when reading block in doc store\"))?;\n// after\nlet block = checkpoint.block_addr.map_err(|error_kind| std::io::Error::new(error_kind.clone(), format!(\"error when reading block in doc store (kind={error_kind:?})\")))?;","handlingStrategy":"try-catch","validationCode":"fn checkpoint_is_resolvable(checkpoint: &Checkpoint, store: &DocStoreReader) -> bool {\n    match &checkpoint.block_addr {\n        Some(addr) => addr.block_id < store.num_blocks(),\n        None => false, // none-checkpoint would trip the internal comment-only corruption\n    }\n}","typeGuard":"fn has_valid_checkpoint<T>(checkpoint: &Option<T>) -> bool {\n    checkpoint.is_some()\n}","tryCatchPattern":"match doc_store_iter.next() {\n    Err(e) => {\n        // the original io::ErrorKind is preserved\n        eprintln!(\"doc store block read failed with kind={:?}: {e}\", e.kind());\n        if e.kind() == std::io::ErrorKind::InvalidData {\n            // corruption: rebuild or restore segment\n        } else {\n            // transient IO: safe to retry with backoff\n        }\n    }\n    Ok(item) => process(item),\n}","preventionTips":["Inspect error.kind() — the message is generic but the kind is preserved","Verify checkpoints reference existing blocks after failed/interrupted commits","Check disk health and permissions on the index directory before long iterations","Re-open the index after writer crashes to reload consistent checkpoint state"],"tags":["io","iteration","data-corruption","rust"],"backgroundTag":"block-read-failure","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}