risingwavelabs/risingwave · error · AccessError

{message}

Error message

{message}

What it means

A wrapper variant of the record decoder's AccessError used for Parquet-format parse failures, plus an Uncategorized variant for errors that fit no other variant. RisingWave throws it when a Parquet file's data cannot be decoded or mapped into the expected schema at record level. Backtraces are intentionally not captured to keep per-record error overhead low.

Source

Thrown at src/connector/codec/src/decoder/mod.rs:54

    #[error("Unsupported data type `{ty}`")]
    UnsupportedType { ty: String },

    /// CDC auto schema change specific error that may include table context
    #[error("CDC auto schema change error: unsupported data type `{ty}` in table `{table_name}`")]
    CdcAutoSchemaChangeError { ty: String, table_name: String },

    #[error("Unsupported additional column `{name}`")]
    UnsupportedAdditionalColumn { name: String },

    #[error("Fail to convert protobuf Any into jsonb: {0}")]
    ProtobufAnyToJson(#[source] serde_json::Error),

    /// Parquet parser specific errors
    #[error("Parquet parser error: {message}")]
    ParquetParser { message: String },

    /// Errors that are not categorized into variants above.
    #[error("{message}")]
    Uncategorized { message: String },

    #[error(transparent)]
    NotImplemented(#[from] NotImplemented),
    // NOTE: We intentionally don't embed `anyhow::Error` in `AccessError` since it happens
    // in record-level and it might be too heavy to capture the backtrace
    // when creating a new `anyhow::Error`.
}

pub type AccessResult<T = Datum> = std::result::Result<T, AccessError>;

/// Access to a field in the data structure. Created by `AccessBuilder`.
///
/// It's the `ENCODE ...` part in `FORMAT ... ENCODE ...`
pub trait Access {
    /// Accesses `path` in the data structure (*parsed* Avro/JSON/Protobuf data),
    /// and then converts it to RisingWave `Datum`.
    ///

View on GitHub (pinned to 6469eb736d)

Solutions

1. Compare the Parquet file schema with the source's declared columns (USE iste what the reader expects) and align column names/types
2. Add or fix the row schema so missing fields have defaults or are nullable
3. Re-ingest a non-corrupt version of the file / check producer write completion
4. If it is genuinely a new error class, check whether upstream risingwave has a dedicated variant; otherwise report with the message

Example fix

// before: source columns declare fs.pay_time as TIMESTAMP but Parquet stores INT96 string
CREATE SOURCE t (...) FORMAT PLAIN ENCODE PARQUET;
// after: align types with file schema
CREATE SOURCE t (pay_time TIMESTAMP) ... ENCODE PARQUET;

// Before ingesting, verify Parquet schema matches declared columns
use parquet::file::reader::{FileReader, SerializedFileReader};
let reader = SerializedFileReader::new(file)?;
let schema = reader.metadata().file_metadata().schema();
assert_eq!(schema.get_fields().len(), expected_cols.len(), "Parquet schema drift");

match res {
    Err(e) if e.to_string().contains("Parquet parser error") => {
        log::warn!("skipping bad record: {e}");
        // route to dead-letter / retry ingest with corrected schema
    }
    other => other?,
}

When it happens

Trigger: Decoding a Parquet record in a source whose format is Parquet and the underlying parquet crate returns an error: mismatched column types, missing fields with no default, corrupt row groups, or schema drift between the file and the declared RW schema.

Common situations: Files written by a producer with a newer/older schema version than the table's columns; Parquet logical types (e.g. decimal, timestamp) that don't match the SQL column types; truncated or partially uploaded files in object storage.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

• Unsupported data type `{ty}`
• pk index {pk_index} is out of range for parquet schema with
• Join key types are not aligned: LHS: {outer_type:?}, RHS: {i
• Join key types are not aligned: LHS: {outer_type:?}, RHS: {i
• schema invalid, record type required at top level of the sch

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/17eefc982117510d. Report an issue: GitHub.