What causes "circular reference detected in Avro schema: {} -> {}" in risingwavelabs/risingwave?

Restructure the schema to remove the recursive record reference (flatten or inline one level).

circular reference detected in Avro schema: {} -> {}

Error message

circular reference detected in Avro schema: {} -> {}

What it means

avro_type_mapping tracks visited record full names in ancestor_records while recursively resolving Avro types. If a record's unique name is already in the ancestor chain, recursion would loop forever, so the function bails with this message naming the chain and the re-entered record.

Source

Thrown at src/connector/codec/src/decoder/avro/schema.rs:148

        Schema::LocalTimestampMillis => DataType::Timestamp,
        Schema::LocalTimestampMicros => DataType::Timestamp,
        Schema::TimestampMillis => DataType::Timestamptz,
        Schema::TimestampMicros => DataType::Timestamptz,
        Schema::Duration => DataType::Interval,
        Schema::Bytes => DataType::Bytea,
        Schema::Enum { .. } => DataType::Varchar,
        Schema::TimeMillis => DataType::Time,
        Schema::TimeMicros => DataType::Time,
        Schema::Record(RecordSchema { fields, name, .. }) => {
            if name.name == DBZ_VARIABLE_SCALE_DECIMAL_NAME
                && name.namespace == Some(DBZ_VARIABLE_SCALE_DECIMAL_NAMESPACE.into())
            {
                return Ok(DataType::Decimal);
            }

            let unique_name = name.fullname(None);
            if ancestor_records.contains(&unique_name) {
                bail!(
                    "circular reference detected in Avro schema: {} -> {}",
                    ancestor_records.join(" -> "),
                    unique_name
                );
            }

            ancestor_records.push(unique_name);
            let ty = StructType::new(
                fields
                    .iter()
                    .map(|f| {
                        Ok((
                            &f.name,
                            avro_type_mapping(&f.schema, ancestor_records, refs, map_handling)?,
                        ))
                    })
                    .collect::<anyhow::Result<Vec<_>>>()?,
            )

View on GitHub (pinned to 6469eb736d)

Solutions

Restructure the schema to remove the recursive record reference (flatten or inline one level).
Break the cycle by making the back-reference part of a union with "null" if the decoder supports optional recursion.
Pre-generate the recursive types or use a schema version without recursion for this connector.

Example fix

// before
{"type":"record","name":"Node","fields":[{"name":"next","type":"Node"}]}
// after
{"type":"record","name":"Node","fields":[{"name":"next","type":["null","Node"],"default":null}]}

Defensive patterns

Strategy: validation

Validate before calling

function hasDirectSelfRef(s, seen = new Set()) {
  const name = s?.name;
  if (name && seen.has(name)) return true;
  if (name) seen.add(name);
  for (const f of s?.fields ?? []) {
    const t = f.type;
    const inner = Array.isArray(t) ? t.find(x => typeof x === 'object') : t;
    if (inner && hasDirectSelfRef(inner, seen)) return true;
  }
  seen.delete(name);
  return false;
}

Prevention

Detect recursive records in schemas during schema review/registration.
Prefer flattening one level of recursion for streaming decode pipelines.
If recursion is required, use a null-union back-reference and verify decoder support first.

When it happens

Trigger: Calling avro_schema_to_fields on an Avro schema in which a record references itself (directly or via a chain of nested records) without an eligible union/null indirection that the resolver treats as breaking the cycle.

Common situations: Recursive Avro schemas produced by generators (e.g. a linked-list or tree type); hand-edited schemas with self references; schema registry evolution accidentally introducing recursion.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/5ac193f5cead8adb. Report an issue: GitHub.