risingwavelabs/risingwave · error

failed to convert JSON schema to Avro schema: {}

Error message

failed to convert JSON schema to Avro schema: {}

What it means

Raised in JsonSchema::json_schema_to_columns (src/connector/codec/src/decoder/json/mod.rs:208-214). After $refs are resolved, the JSON schema is converted to Avro via the `jst` (jsonschema-transpiler) crate inside rw_catch_unwind; if that conversion panics, the panic message is wrapped into `failed to convert JSON schema to Avro schema: {msg}`. This indicates the JSON schema uses constructs the transpiler cannot map to Avro (uncommon keywords, unsupported combinators, or malformed schema shapes).

Source

Thrown at src/connector/codec/src/decoder/json/mod.rs:210

impl crate::JsonSchema {
    /// ## Notes on type conversion
    /// Map will be used when an object doesn't have `properties` but has `additionalProperties`.
    /// When an object has `properties` and `additionalProperties`, the latter will be ignored.
    /// <https://github.com/mozilla/jsonschema-transpiler/blob/fb715c7147ebd52427e0aea09b2bba2d539850b1/src/jsonschema.rs#L228-L280>
    ///
    /// TODO: examine other stuff like `oneOf`, `patternProperties`, etc.
    pub async fn json_schema_to_columns(
        &mut self,
        retrieval_url: Url,
    ) -> anyhow::Result<Vec<Field>> {
        JsonRef::new()
            .deref_value(&mut self.0, &retrieval_url)
            .await?;
        let avro_schema =
            rw_catch_unwind(|| jst::convert_avro(&self.0, jst::Context::default()).to_string())
                .map_err(|payload| {
                    anyhow!(
                        "failed to convert JSON schema to Avro schema: {}",
                        panic_message::panic_message(&payload)
                    )
                })?;
        let schema =
            apache_avro::Schema::parse_str(&avro_schema).context("failed to parse avro schema")?;
        avro_schema_to_fields(&schema, Some(MapHandling::Jsonb))
    }
}

View on GitHub (pinned to 6469eb736d)

Solutions

1. Simplify the JSON schema to plain object/properties types with scalar leaf types the transpiler supports
2. Move complex constructs (`oneOf`, `anyOf`, `patternProperties`) into explicitly typed `properties`
3. Read the panic message in the error to identify the unsupported construct and rewrite it
4. Pre-convert the schema to Avro yourself and use AVRO encoding instead of JSON schema conversion

Example fix

// before
{"oneOf": [{"type": "string"}, {"type": "object", "properties": {"a": {"type": "integer"}}}]}
// after
{"type": "object", "properties": {"a": {"type": "integer"}}}

// Pre-validate the schema converts cleanly with jsonschema-transpiler before use:
// cargo run -- schema.json  (or convert with jst in a build step)
fn is_plain_object_schema(v: &serde_json::Value) -> bool {
    v.get("type").and_then(|t| t.as_str()) == Some("object")
        && v.get("properties").map(|p| p.is_object()).unwrap_or(false)
}

match json_schema_to_columns(schema, url).await {
    Ok(cols) => cols,
    Err(e) if e.to_string().contains("failed to convert JSON schema to Avro schema") => {
        // fall back: simplify schema or switch encoding to AVRO
        Vec::new()
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Calling json_schema_to_columns on a schema whose dereferenced form triggers a panic in `jst::convert_avro` — e.g. JSON Schema keywords with no Avro equivalent, deeply unusual `type` shapes, or null/non-object roots the transpiler does not handle.

Common situations: Schemas authored for JSON Schema validation (draft-07 with `oneOf`/`patternProperties`/conditional keywords) being reused as RisingWave protobuf/JSON encoded sources; schemas produced by generators emitting non-standard extensions; empty or `true`/`false` boolean schemas.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• schema invalid, record type required at top level of the sch
• circular reference detected in Avro schema: {} -> {}
• Root schema of debezium shall be a record but got: {root:?}
• schema compilation error: {0}
• Encode error: {0}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/6657da959e22c3d4. Report an issue: GitHub.