{"record":{"id":"1d323a8555ad28f8","repo":"risingwavelabs/risingwave","slug":"payload-shorter-than-18-byte-glue-header","errorCode":null,"errorMessage":"payload shorter than 18-byte glue header","messagePattern":"payload shorter than 18-byte glue header","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/connector/src/parser/avro/parser.rs","lineNumber":129,"sourceCode":"                )?))\n            }\n            WriterSchemaCache::File => {\n                // FIXME: we should not use `Reader` (file header) here. See comment above and https://github.com/risingwavelabs/risingwave/issues/12871\n                let mut reader = Reader::with_schema(&self.schema.original_schema, payload)?;\n                match reader.next() {\n                    Some(Ok(v)) => Ok(Some(v)),\n                    Some(Err(e)) => Err(e)?,\n                    None => bail!(\"avro parse unexpected eof\"),\n                }\n            }\n            WriterSchemaCache::Glue(resolver) => {\n                // <https://github.com/awslabs/aws-glue-schema-registry/blob/v1.1.20/common/src/main/java/com/amazonaws/services/schemaregistry/utils/AWSSchemaRegistryConstants.java#L59-L61>\n                // byte 0:      header version = 3\n                // byte 1:      compression: 0 = no compression; 5 = zlib (unsupported)\n                // byte 2..=17: 16-byte UUID as schema version id\n                // byte 18..:   raw avro payload\n                if payload.len() < 18 {\n                    bail!(\"payload shorter than 18-byte glue header\");\n                }\n                if payload[0] != 3 {\n                    bail!(\n                        \"Only support glue header version 3 but found {}\",\n                        payload[0]\n                    );\n                }\n                if payload[1] != 0 {\n                    bail!(\"Non-zero compression {} not supported\", payload[1]);\n                }\n                let schema_version_id = uuid::Uuid::from_slice(&payload[2..18]).unwrap();\n                let writer_schema = resolver.get_by_id(schema_version_id).await?;\n                let mut raw_payload = &payload[18..];\n                Ok(Some(from_avro_datum(\n                    writer_schema.as_ref(),\n                    &mut raw_payload,\n                    Some(&self.schema.original_schema),\n                )?))","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/parser/avro/parser.rs#L111-L147","documentation":"AWS Glue Schema Registry frames Avro payloads with an 18-byte header: 1 byte version (3), 1 byte compression, and 16 bytes of schema-version UUID. parse_avro_value checks payload.len() < 18 before parsing the header and bails when the buffer is too short, meaning the message is not Glue-framed Avro.","triggerScenarios":"Avro source configured with Glue Schema Registry receiving a payload shorter than 18 bytes — e.g. plain unframed Avro, an empty message, or the wrong serializer on the producer.","commonSituations":"Producer not using the Glue Schema Registry serializer while the RW source expects Glue framing; test messages sent manually; wrong topic wired up with non-Glue producers.","solutions":["Configure the producer to use the AWS Glue Schema Registry Avro serializer","Verify the topic actually carries Glue-framed messages (first byte should be 0x03)","Send a valid test record through the producer and re-check","Switch the RW source config to the matching schema registry type (e.g. Confluent) if Glue is not actually used"],"exampleFix":"// producer before: AvroEncoder without Glue framing\n// producer after: AWSKafkaAvroSerializer with schemaAutoRegistration, region and registryName set","handlingStrategy":"validation","validationCode":"function hasGlueHeader(buf) {\n  return buf && buf.length >= 18;\n}\n// producer side: assert frames emitted by Glue serializer are >= 18 bytes of header + body","typeGuard":"function isGlueFramed(buf) {\n  return buf instanceof Uint8Array && buf.length >= 18 && buf[0] === 3 && buf[1] === 0;\n}","tryCatchPattern":"match parse_result {\n  Err(e) if e.to_string().contains(\"18-byte glue header\") => {\n    error!(\"producer is not using Glue Schema Registry serializer\");\n  }\n  other => other?,\n}","preventionTips":["Use the AWS Glue Schema Registry Avro serializer on all producers","Confirm topic payload framing manually before wiring the RW source","Don't send raw/unframed Avro bytes to a Glue-configured source"],"tags":["avro","glue","schema-registry","header"],"backgroundTag":"unexpected-response-shape","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}