{"record":{"id":"38a4573e5c22b34c","repo":"databendlabs/databend","slug":"error-parsing-first-message","errorCode":null,"errorMessage":"Error parsing first message","messagePattern":"Error parsing first message","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"src/query/service/src/servers/flight/v1/exchange/serde/exchange_deserializer.rs","lineNumber":107,"sourceCode":"        let data_block = deserialize_block(dict, fragment_data, &schema, arrow_schema)?;\n        if data_block.num_columns() == 0 {\n            return Ok(DataBlock::new_with_meta(vec![], row_count as usize, meta));\n        }\n        data_block.add_meta(meta)\n    }\n}\n\npub fn deserialize_block(\n    dict: Vec<DataPacket>,\n    fragment_data: FragmentData,\n    schema: &DataSchema,\n    arrow_schema: Arc<ArrowSchema>,\n) -> Result<DataBlock> {\n    let mut dictionaries_by_id = HashMap::new();\n    for dict_packet in dict {\n        if let DataPacket::Dictionary(data) = dict_packet {\n            let message =\n                root_as_message(&data.data_header[..]).expect(\"Error parsing first message\");\n            let buffer = Buffer::from(data.data_body);\n            arrow_ipc::reader::read_dictionary(\n                &buffer,\n                message\n                    .header_as_dictionary_batch()\n                    .expect(\"Error parsing dictionary\"),\n                &arrow_schema,\n                &mut dictionaries_by_id,\n                &message.version(),\n            )\n            .expect(\"Error reading dictionary\");\n        }\n    }\n\n    let batch = flight_data_to_arrow_batch(&fragment_data.data, arrow_schema, &dictionaries_by_id)?;\n    let data_block = DataBlock::from_record_batch(schema, &batch)?;\n    Ok(data_block)\n}","sourceCodeStart":89,"sourceCodeEnd":125,"githubUrl":"https://github.com/databendlabs/databend/blob/288d84d76e20a2f8f7173bda9691eb6ece301aa9/src/query/service/src/servers/flight/v1/exchange/serde/exchange_deserializer.rs#L89-L125","documentation":"During block deserialization on the data-exchange path, dictionary `DataPacket`s are parsed as Arrow IPC messages via `root_as_message(&data.data_header[..]).expect(\"Error parsing first message\")`. FlatBuffers verification failed, meaning the header bytes are not a valid Arrow IPC message. This panic indicates corrupted, truncated, or non-dictionary IPC payload exchanged between query nodes (or a schema/serialization version mismatch), and it crashes the deserialization task instead of returning a `Result`.","triggerScenarios":"`deserialize_block(dict, fragment_data, arrow_schema)` receiving a `DataPacket::Dictionary` whose `data_header` is empty, truncated by network send/recv buffering bugs, or produced by an incompatible Arrow IPC writer version; also caused by byte-offset errors in the packet framing upstream in `recv_data` / `read` / spilled-file readers.","commonSituations":"Mixed Databend versions in a cluster where the exchange serde format changed; corrupted spill files being read back via `read_unmanage_spilled_file`; network issues or buggy proxies mangling flight data packets.","solutions":["Verify all cluster nodes run the same Databend version; rolling upgrades that cross exchange-format changes corrupt packets.","Check the source of the dictionary packet: if from a spill file, validate the file is not truncated/corrupted (checksum/size) and re-run the query.","Confirm the packet framing logic in `recv_data`/`read` slices `data_header` correctly (no off-by-one or partial reads).","Replace the expects with proper error returns (`flatbuffers::root_as_message` error mapped to `ErrorCode::Internal`) so bad packets fail the query cleanly."],"exampleFix":"// before\nlet message =\n    root_as_message(&data.data_header[..]).expect(\"Error parsing first message\");\n\n// after\nlet message = root_as_message(&data.data_header[..]).map_err(|e| {\n    ErrorCode::Internal(format!(\n        \"Error parsing dictionary IPC message: {}\",\n        e\n    ))\n})?;","handlingStrategy":"validation","validationCode":"// Validate dictionary packet before deserialize\nif data.data_header.is_empty() {\n    return Err(ErrorCode::Internal(\"empty dictionary data_header in exchange packet\"));\n}","typeGuard":"fn is_valid_dict_packet(p: &DataPacket) -> bool {\n    matches!(p, DataPacket::Dictionary(d) if !d.data_header.is_empty())\n}","tryCatchPattern":"match std::panic::catch_unwind(|| deserialize_block(&dict, &fragment_data, schema.clone())) {\n    Ok(res) => res,\n    Err(_) => Err(ErrorCode::Internal(\"corrupt arrow IPC dictionary packet received\")),\n}","preventionTips":["Keep all cluster nodes on the same Databend/Arrow version.","Enable checksum/integrity checks on spill files and validate file sizes before reading.","Watch for network middleboxes/proxies that may mangle flight data streams."],"tags":["arrow","ipc","serialization","panic","distributed-query"],"backgroundTag":"protobuf-unmarshal-failed","analyzedSha":"288d84d76e20a2f8f7173bda9691eb6ece301aa9","analyzedAt":"2026-09-11T11:29:36.208Z","contentChangedAt":"2026-09-11T11:29:36.208Z","schemaVersion":2},"datasetVersion":"2026-09-16T04:17:20.429Z"}