{"record":{"id":"19dfc6acabdb6c04","repo":"quickwit-oss/tantivy","slug":"invaliddata-19dfc6","errorCode":"InvalidData","errorMessage":"Could not convert value to compact_space. This is a bug.","messagePattern":"Could not convert value to compact_space\\. This is a bug\\.","errorType":"error_code","errorClass":"io::Error","httpStatus":null,"severity":"error","filePath":"columnar/src/column_values/u128_based/compact_space/mod.rs","lineNumber":241,"sourceCode":"        let footer_len = writer.written_bytes() as u32;\n        footer_len.serialize(writer)?;\n\n        Ok(())\n    }\n\n    pub fn compress_into(\n        self,\n        vals: impl Iterator<Item = u128>,\n        write: &mut impl Write,\n    ) -> io::Result<()> {\n        let mut bitpacker = BitPacker::default();\n        for val in vals {\n            let compact = self\n                .params\n                .compact_space\n                .u128_to_compact(val)\n                .map_err(|_| {\n                    io::Error::new(\n                        io::ErrorKind::InvalidData,\n                        \"Could not convert value to compact_space. This is a bug.\",\n                    )\n                })?;\n            bitpacker.write(compact as u64, self.params.num_bits, write)?;\n        }\n        bitpacker.close(write)?;\n        self.write_footer(write)?;\n        Ok(())\n    }\n}\n\n#[derive(Debug, Clone)]\npub struct CompactSpaceDecompressor {\n    data: OwnedBytes,\n    params: IPCodecParams,\n}\n","sourceCodeStart":223,"sourceCodeEnd":259,"githubUrl":"https://github.com/quickwit-oss/tantivy/blob/b5d8deb80c26924e6b007a5b1a7630f35ca64de4/columnar/src/column_values/u128_based/compact_space/mod.rs#L223-L259","documentation":"During u128 fast-field compression, each value is mapped into a compact space defined by the codec params; u128_to_compact is total over the space the params were computed for. If any value falls outside it, this io::Error (InvalidData) is returned because it indicates an internal invariant violation — hence the message \"This is a bug\". Users normally see it only via corrupted metadata or a genuine library bug.","triggerScenarios":"Calling compress_into (directly or via serialize_column_values_u128/compress) where params' compact space was computed from different values than those being compressed — e.g. mismatched stats, corrupted header describing value range, or memory corruption.","commonSituations":"Segment files whose serialized codec params don't match the values (truncation/partial rewrite); concurrency bugs during serialization; tantivy internal bugs (report upstream).","solutions":["Report to tantivy with a reproducer — the message explicitly indicates a library bug.","Re-index the affected segment so params and values are recomputed together.","Verify no external tool rewrote or truncated columnar files.","Upgrade tantivy in case the bug is already fixed."],"exampleFix":"// before\nlet compact = params.compact_space.u128_to_compact(val)?; // may fail\n// after\n// ensure params are derived from the exact same column values:\nlet params = CompactSpaceParams::from_values(vals)?;\nlet compact = params.compact_space.u128_to_compact(val)?; // total by construction","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"let compact = match params.compact_space.u128_to_compact(val) {\n    Ok(c) => c,\n    Err(_) => {\n        // \"This is a bug\": log value + params, rebuild segment, report upstream\n        unreachable_rebuild_segment();\n    }\n};","preventionTips":["Report occurrences to tantivy — this path asserts an internal invariant.","Never modify or hand-write columnar codec params on disk.","Always recompute params from the same value set being serialized.","Keep atomic segment writes so headers can't mismatch data."],"tags":["io","codec","u128","internal-bug"],"backgroundTag":"compact-space-conversion-bug","analyzedSha":"b5d8deb80c26924e6b007a5b1a7630f35ca64de4","analyzedAt":"2026-09-05T13:20:51.521Z","contentChangedAt":"2026-09-05T13:20:51.521Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}