{"record":{"id":"181c531e124a3d66","repo":"risingwavelabs/risingwave","slug":"failed-to-match-the-magic-byte-0","errorCode":null,"errorMessage":"failed to match the magic byte 0","messagePattern":"failed to match the magic byte 0","errorType":"exception","errorClass":"WireFormatError::NoMagic","httpStatus":null,"severity":"error","filePath":"src/connector/src/schema/schema_registry/util.rs","lineNumber":48,"sourceCode":"        match ele.parse::<Url>() {\n            Ok(url) => urls.push(url),\n            Err(e) => errs.push(e),\n        }\n    }\n    if urls.is_empty() {\n        bail_invalid_option_error!(\"no valid url provided, errs: {errs:?}\");\n    }\n    tracing::debug!(\n        \"schema registry client will use url {:?} to connect, the rest failed because: {:?}\",\n        urls,\n        errs\n    );\n    Ok(urls)\n}\n\n#[derive(Debug, thiserror::Error)]\npub enum WireFormatError {\n    #[error(\"failed to match the magic byte 0\")]\n    NoMagic,\n    #[error(\"failed to read the 4-byte schema ID\")]\n    NoSchemaId,\n    #[error(\"failed to parse message indexes\")]\n    ParseMessageIndexes,\n}\n\n/// Returns `(schema_id, payload)`\n///\n/// Refer to [Confluent schema registry wire format](https://docs.confluent.io/platform/7.6/schema-registry/fundamentals/serdes-develop/index.html#wire-format)\n///\n/// | Bytes | Area        | Description                                                                                        |\n/// |-------|-------------|----------------------------------------------------------------------------------------------------|\n/// | 0     | Magic Byte  | Confluent serialization format version number; currently always `0`.                               |\n/// | 1-4   | Schema ID   | 4-byte schema ID as returned by Schema Registry.                                                   |\n/// | 5-... | Data        | Serialized data for the specified schema format (for example, binary encoding for Avro or Protobuf.|\npub(crate) fn extract_schema_id(payload: &[u8]) -> Result<(i32, &[u8]), WireFormatError> {\n    use byteorder::{BigEndian, ReadBytesExt as _};","sourceCodeStart":30,"sourceCodeEnd":66,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/schema/schema_registry/util.rs#L30-L66","documentation":"WireFormatError::NoMagic is thrown when decoding a Confluent wire-format message payload: the first (magic) byte of the message is not 0, which is the only version the decoder supports. This means the payload does not follow the Confluent Schema Registry envelope (magic byte + 4-byte schema ID + payload).","triggerScenarios":"Parsing a Kafka message value that does not begin with byte 0 — e.g. messages produced without the schema registry serializer, plain JSON/Avro without the envelope, or tombstones/empty messages being fed to the decoder.","commonSituations":"Producing to the topic with a non-registry serializer (e.g. plain JsonProducer), messages from a producer using a newer wire format, or misconfigured topic where mixed message formats exist.","solutions":["Verify the producer uses a Confluent-compatible serializer (KafkaAvroSerializer / KafkaJsonSchemaSerializer / Protobuf with registry) and points at the same registry.","Inspect a raw message's first byte (hex) to confirm the envelope; if absent, re-produce data with the registry serializer.","If tombstones are expected, filter out null/empty payloads before decoding.","Check that no producer upgrades changed the wire format for this topic."],"exampleFix":"// before: decoding every message\nlet (id, payload) = parse_wire_format(bytes)?;\n// after: skip non-envelope payloads\nif bytes.first() == Some(&0) {\n    let (id, payload) = parse_wire_format(bytes)?;\n}","handlingStrategy":"type-guard","validationCode":"fn has_confluent_envelope(bytes: &[u8]) -> bool {\n    bytes.first() == Some(&0)\n}","typeGuard":"fn is_wire_format_v0(b: &[u8]) -> bool { b.first() == Some(&0) && b.len() >= 5 }","tryCatchPattern":"match parse_wire_format(bytes) {\n    Ok((id, payload)) => decode(id, payload),\n    Err(WireFormatError::NoMagic) => {\n        tracing::warn!(\"non-registry payload; skipping\");\n        skip();\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Ensure all producers use Confluent registry serializers.","Filter tombstones/null payloads before decoding.","Verify topic producers have not changed serialization format.","Spot-check first payload byte when onboarding new topics."],"tags":["kafka","wire-format","schema-registry"],"backgroundTag":"unexpected-response-shape","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}