{"record":{"id":"30dde56959aa31e5","repo":"databendlabs/databend","slug":"error-parsing-dictionary","errorCode":null,"errorMessage":"Error parsing dictionary","messagePattern":"Error parsing dictionary","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"src/query/service/src/servers/flight/v1/exchange/serde/exchange_deserializer.rs","lineNumber":113,"sourceCode":"}\n\npub fn deserialize_block(\n    dict: Vec<DataPacket>,\n    fragment_data: FragmentData,\n    schema: &DataSchema,\n    arrow_schema: Arc<ArrowSchema>,\n) -> Result<DataBlock> {\n    let mut dictionaries_by_id = HashMap::new();\n    for dict_packet in dict {\n        if let DataPacket::Dictionary(data) = dict_packet {\n            let message =\n                root_as_message(&data.data_header[..]).expect(\"Error parsing first message\");\n            let buffer = Buffer::from(data.data_body);\n            arrow_ipc::reader::read_dictionary(\n                &buffer,\n                message\n                    .header_as_dictionary_batch()\n                    .expect(\"Error parsing dictionary\"),\n                &arrow_schema,\n                &mut dictionaries_by_id,\n                &message.version(),\n            )\n            .expect(\"Error reading dictionary\");\n        }\n    }\n\n    let batch = flight_data_to_arrow_batch(&fragment_data.data, arrow_schema, &dictionaries_by_id)?;\n    let data_block = DataBlock::from_record_batch(schema, &batch)?;\n    Ok(data_block)\n}\n\nimpl BlockMetaTransform<ExchangeDeserializeMeta> for TransformExchangeDeserializer {\n    const UNKNOWN_MODE: UnknownMode = UnknownMode::Pass;\n    const NAME: &'static str = \"TransformExchangeDeserializer\";\n\n    fn transform(&mut self, mut meta: ExchangeDeserializeMeta) -> Result<Vec<DataBlock>> {","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/service/src/servers/flight/v1/exchange/serde/exchange_deserializer.rs#L95-L131","documentation":"After the Arrow IPC message header verifies, the code asks for the message's `header_as_dictionary_batch()` and unwraps it with `.expect(\"Error parsing dictionary\")`. A `None` here means the message parsed but its header type is not a `DictionaryBatch` — the packet labeled as a dictionary contained a different batch type (e.g., a RecordBatch) or an unsupported/empty header. This is a wire-format invariant violation on the exchange path and panics the deserializer.","triggerScenarios":"`deserialize_block` receiving a `DataPacket::Dictionary` whose IPC message header is actually a `RecordBatch` (sender/receiver packet-type mismatch), an IPC message with no header, or data produced by an Arrow writer emitting non-standard dictionary encoding.","commonSituations":"Version skew between cluster nodes after an Arrow or Databend upgrade changing how dictionaries are serialized; custom/forked serialization code writing the wrong header type into `Dictionary` packets; corrupted spill files whose bytes were shifted so the header decodes as the wrong type.","solutions":["Ensure all nodes run matching versions of Databend/Arrow so dictionary packet framing agrees on both ends.","Inspect the sender side: confirm `DataPacket::Dictionary` packets are only created from genuine dictionary-batch IPC messages.","If data came from a spill file, verify the file was not partially written or truncated, then re-run the query.","Return a typed error instead of panicking so the affected query fails without killing the deserialization task."],"exampleFix":"// before\nmessage\n    .header_as_dictionary_batch()\n    .expect(\"Error parsing dictionary\"),\n\n// after\nmessage.header_as_dictionary_batch().ok_or_else(|| {\n    ErrorCode::Internal(\n        \"IPC message is not a dictionary batch in DataPacket::Dictionary\",\n    )\n})?,","handlingStrategy":"validation","validationCode":"// Check header type before read_dictionary\nlet message = root_as_message(&data.data_header[..]).map_err(|e| ErrorCode::Internal(format!(\"bad IPC message: {e}\")))?;\nif message.header_as_dictionary_batch().is_none() {\n    return Err(ErrorCode::Internal(\"DataPacket::Dictionary did not contain a dictionary batch\"));\n}","typeGuard":"fn is_dictionary_batch(msg: &arrow_ipc::Message) -> bool {\n    msg.header_as_dictionary_batch().is_some()\n}","tryCatchPattern":"std::panic::catch_unwind(|| deserialize_block(&dict, &fragment_data, schema.clone()))\n    .unwrap_or_else(|_| Err(ErrorCode::Internal(\"invalid dictionary batch header in exchange packet\")))","preventionTips":["Align serialization format versions across the cluster before rolling upgrades.","Validate spill files for truncation before re-reading them.","Add unit tests asserting DataPacket::Dictionary always wraps a DictionaryBatch header."],"tags":["arrow","ipc","serialization","panic","distributed-query"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}