risingwavelabs/risingwave · error · AccessError

Fail to convert protobuf Any into jsonb

Error message

Fail to convert protobuf Any into jsonb: {0}

What it means

AccessError::ProtobufAnyToJson in src/connector/codec/src/decoder/mod.rs:46. Thrown when converting a google.protobuf.Any value from a Protobuf payload into a jsonb: the underlying serde_json (de)serialization of the packed Any fails, and the serde_json::Error is attached as source.

Solutions

  1. Avoid google.protobuf.Any in schemas consumed by RisingWave; use concrete message types or a JSON/string field
  2. Fix the producer so Any payloads always contain a valid, descriptor-registered message
  3. Check the source serde_json::Error in the chain to see which part of the Any JSON conversion failed
  4. Pre-convert Any fields to plain JSON strings upstream

Example fix

// before
message Wrapper { google.protobuf.Any payload = 1; }
// after
message Wrapper { string payload_json = 1; } // producer serializes inner msg to JSON text
Defensive patterns

Strategy: try-catch

Validate before calling

// Producer-side guard: ensure Any fields are always packed with a known type URL
fn any_is_valid(any: &prost_types::Any) -> bool {
    !any.type_url.is_empty() && any.value.len() > 0
}

Try / catch

match decode_protobuf(bytes, desc) {
    Ok(v) => v,
    Err(e) if e.to_string().starts_with("Fail to convert protobuf Any into jsonb") => {
        // degrade: decode remaining fields, log offending Any
        tracing::warn!(%e, "skipping record with undecodable Any");
        default_record()
    }
    Err(e) => return Err(e.into()),
}

Prevention

When it happens

Trigger: Protobuf decoder encounters a field of type google.protobuf.Any and attempts to render it (including type URL and unpacked value) as JSON via serde_json; conversion fails because the packed message cannot be serialized to the expected JSON shape (e.g. malformed Any payload, unsupported inner message).

Common situations: Producers writing Any fields containing message types RisingWave cannot serialize to JSON; corrupted or hand-crafted binary payloads whose Any bytes are not a valid embedded message; missing descriptors for the packed inner type.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/5dfa4c0d9fbaac11. Report an issue: GitHub.

Appendix: source

Thrown at src/connector/codec/src/decoder/mod.rs:46

    #[error("Undefined field `{name}` at `{path}`")]
    Undefined { name: String, path: String },
    #[error("Cannot parse value `{value}` with type `{got}` into expected type `{expected}`")]
    TypeError {
        expected: String,
        got: String,
        value: String,
    },
    #[error("Unsupported data type `{ty}`")]
    UnsupportedType { ty: String },

    /// CDC auto schema change specific error that may include table context
    #[error("CDC auto schema change error: unsupported data type `{ty}` in table `{table_name}`")]
    CdcAutoSchemaChangeError { ty: String, table_name: String },

    #[error("Unsupported additional column `{name}`")]
    UnsupportedAdditionalColumn { name: String },

    #[error("Fail to convert protobuf Any into jsonb: {0}")]
    ProtobufAnyToJson(#[source] serde_json::Error),

    /// Parquet parser specific errors
    #[error("Parquet parser error: {message}")]
    ParquetParser { message: String },

    /// Errors that are not categorized into variants above.
    #[error("{message}")]
    Uncategorized { message: String },

    #[error(transparent)]
    NotImplemented(#[from] NotImplemented),
    // NOTE: We intentionally don't embed `anyhow::Error` in `AccessError` since it happens
    // in record-level and it might be too heavy to capture the backtrace
    // when creating a new `anyhow::Error`.
}

pub type AccessResult<T = Datum> = std::result::Result<T, AccessError>;

View on GitHub (pinned to 6469eb736d)