risingwavelabs/risingwave · error · SinkError::BigQuery

Can't find {}

Error message

Can't find {}

What it means

For non-append-only (upsert/mutable) BigQuery sinks, RisingWave adds a synthetic protobuf field named CHANGE_TYPE ('_rw_change_type') to the generated message descriptor to carry row change type. This error means that field could not be found in the built message descriptor even though it should have just been added - an internal inconsistency indicating the field was not appended or its name changed.

Source

Thrown at src/connector/src/sink/big_query.rs:698

                ..Default::default()
            };
            descriptor_proto.field.push(field);
        }

        let descriptor_pool = build_protobuf_descriptor_pool(&descriptor_proto)?;
        let message_descriptor = descriptor_pool
            .get_message_by_name(&config.common.table)
            .ok_or_else(|| {
                SinkError::BigQuery(anyhow::anyhow!(
                    "Can't find message proto {}",
                    config.common.table
                ))
            })?;
        let proto_field = if !is_append_only {
            let proto_field = message_descriptor
                .get_field_by_name(CHANGE_TYPE)
                .ok_or_else(|| {
                    SinkError::BigQuery(anyhow::anyhow!("Can't find {}", CHANGE_TYPE))
                })?;
            Some(proto_field)
        } else {
            None
        };
        let row_encoder = ProtoEncoder::new(
            schema.clone(),
            None,
            message_descriptor.clone(),
            ProtoHeader::None,
        )?;
        Ok((
            Self {
                write_stream: format!(
                    "projects/{}/datasets/{}/tables/{}/streams/_default",
                    config.common.project, config.common.dataset, config.common.table
                ),
                config,

View on GitHub (pinned to 6469eb736d)

Solutions

1. If you only append data, use an append-only sink so the CHANGE_TYPE field is never required.
2. Check that the code pushes the CHANGE_TYPE FieldDescriptorProto (name exactly CHANGE_TYPE) into descriptor_proto.field before building the pool.
3. Rebuild the workspace to clear stale protobuf codegen artifacts and retry.
4. Inspect message_descriptor field names at runtime to see which fields actually exist.

Example fix

// before
let field = message_descriptor.get_field_by_name(CHANGE_TYPE).ok_or_else(|| SinkError::BigQuery(anyhow::anyhow!("Can't find {}", CHANGE_TYPE)))?;
// after
assert!(descriptor_proto.field.iter().any(|f| f.name() == CHANGE_TYPE), "CHANGE_TYPE field missing from descriptor");

-- Only use upsert mode when you truly need mutations;
-- append-only sinks skip the CHANGE_TYPE field entirely.
-- Check your sink_type before CREATE SINK: append-only avoids this path.

match sink_creation {
    Err(SinkError::BigQuery(e)) if e.to_string().contains("Can't find _rw_change_type") => {
        // rebuild/upgrade, or switch to an append-only sink, then retry
    }
    other => other?,
}

When it happens

Trigger: BigQuerySinkWriter::new with is_append_only=false, after build_protobuf_descriptor_pool, when message_descriptor.get_field_by_name(CHANGE_TYPE) returns None - e.g. the CHANGE_TYPE field was not pushed to descriptor_proto.field before pool construction, or the registered name differs.

Common situations: Running an upsert (debezium/mutable) sink where the pool dropped or renamed the added field; protobuf codegen version mismatch; a code path where descriptor_proto.field was not mutated before building the pool.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Can't find message proto {}
• query_epoch not set in distributed lookup join
• no chunk in IngestDmlPayloadRequest
• Failed to decode prost: field not found `{}`
• Invalid datetime: seconds: {secs}, nanoseconds: {nsecs}

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/12ff973daea3e119. Report an issue: GitHub.