What causes "failed to parse message indexes" in risingwavelabs/risingwave?

Verify the producer's Confluent serializer version and that it writes indexes per the wire format spec.

risingwavelabs/risingwave · error · WireFormatError::ParseMessageIndexes

failed to parse message indexes

Error message

failed to parse message indexes

What it means

WireFormatError::ParseMessageIndexes is thrown when decoding a Confluent wire format v1 (multi-message/array) payload where the trailing array of message indexes cannot be parsed. Indexes are used to point at a nested position in the schema; failure means the variable-length index encoding is malformed.

Source

Thrown at src/connector/src/schema/schema_registry/util.rs:52

    }
    if urls.is_empty() {
        bail_invalid_option_error!("no valid url provided, errs: {errs:?}");
    }
    tracing::debug!(
        "schema registry client will use url {:?} to connect, the rest failed because: {:?}",
        urls,
        errs
    );
    Ok(urls)
}

#[derive(Debug, thiserror::Error)]
pub enum WireFormatError {
    #[error("failed to match the magic byte 0")]
    NoMagic,
    #[error("failed to read the 4-byte schema ID")]
    NoSchemaId,
    #[error("failed to parse message indexes")]
    ParseMessageIndexes,
}

/// Returns `(schema_id, payload)`
///
/// Refer to [Confluent schema registry wire format](https://docs.confluent.io/platform/7.6/schema-registry/fundamentals/serdes-develop/index.html#wire-format)
///
/// | Bytes | Area        | Description                                                                                        |
/// |-------|-------------|----------------------------------------------------------------------------------------------------|
/// | 0     | Magic Byte  | Confluent serialization format version number; currently always `0`.                               |
/// | 1-4   | Schema ID   | 4-byte schema ID as returned by Schema Registry.                                                   |
/// | 5-... | Data        | Serialized data for the specified schema format (for example, binary encoding for Avro or Protobuf.|
pub(crate) fn extract_schema_id(payload: &[u8]) -> Result<(i32, &[u8]), WireFormatError> {
    use byteorder::{BigEndian, ReadBytesExt as _};

    let mut cursor = payload;
    if !cursor.read_u8().is_ok_and(|magic| magic == 0) {
        return Err(WireFormatError::NoMagic);

View on GitHub (pinned to 6469eb736d)

Solutions

Verify the producer's Confluent serializer version and that it writes indexes per the wire format spec.
Dump raw payload bytes and validate the trailing zigzag varint index array manually.
Fall back to producing wire-format v0 messages (single top-level record) if nested indexing is not needed.
Re-serialize affected records with the official Confluent serializer.

Example fix

// before: assuming v0 payload
let (id, payload) = parse(bytes)?; // indexes never parsed
// after: handle v1 with valid index array
if bytes.first() == Some(&1) {
    let indexes = parse_message_indexes(&mut cursor)?; // zigzag varints
}

Defensive patterns

Strategy: try-catch

Type guard

fn is_wire_format_v1(b: &[u8]) -> bool { b.first() == Some(&1) }

Try / catch

match parse_wire_format(bytes) {
    Ok((id, payload)) => decode(id, payload),
    Err(WireFormatError::ParseMessageIndexes) => {
        tracing::warn!("bad v1 message indexes; falling back to raw decode");
        fallback_decode(bytes);
    }
    Err(e) => return Err(e.into()),
}

Prevention

Use official Confluent serializers that emit spec-compliant v1 indexes.
Prefer v0 wire format when nested schemas are not needed.
Validate index encoding with unit tests on sample payloads.
Guard against truncated records before index parsing.

When it happens

Trigger: Parsing a wire-format payload whose trailing bytes (after magic + schema ID) should be a zigzag-varint-encoded index array but are missing, empty, or corrupt — e.g. truncated messages or non-conformant producers.

Common situations: Custom producers writing v1 wire format incorrectly, truncated Kafka records, or payloads hand-assembled in tests without proper index encoding.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/801d7f1a1a1a53fb. Report an issue: GitHub.