risingwavelabs/risingwave · error · AccessError

Parquet parser error: {message}

Error message

Parquet parser error: {message}

What it means

AccessError::ParquetParser in src/connector/codec/src/decoder/mod.rs:50. A catch-all variant for errors specific to the Parquet parser, carrying a free-form `message`. It wraps any decoding problem encountered while reading Parquet-encoded payloads (schema mismatch, unsupported column encodings, corrupted data, projection errors).

Source

Thrown at src/connector/codec/src/decoder/mod.rs:50

        expected: String,
        got: String,
        value: String,
    },
    #[error("Unsupported data type `{ty}`")]
    UnsupportedType { ty: String },

    /// CDC auto schema change specific error that may include table context
    #[error("CDC auto schema change error: unsupported data type `{ty}` in table `{table_name}`")]
    CdcAutoSchemaChangeError { ty: String, table_name: String },

    #[error("Unsupported additional column `{name}`")]
    UnsupportedAdditionalColumn { name: String },

    #[error("Fail to convert protobuf Any into jsonb: {0}")]
    ProtobufAnyToJson(#[source] serde_json::Error),

    /// Parquet parser specific errors
    #[error("Parquet parser error: {message}")]
    ParquetParser { message: String },

    /// Errors that are not categorized into variants above.
    #[error("{message}")]
    Uncategorized { message: String },

    #[error(transparent)]
    NotImplemented(#[from] NotImplemented),
    // NOTE: We intentionally don't embed `anyhow::Error` in `AccessError` since it happens
    // in record-level and it might be too heavy to capture the backtrace
    // when creating a new `anyhow::Error`.
}

pub type AccessResult<T = Datum> = std::result::Result<T, AccessError>;

/// Access to a field in the data structure. Created by `AccessBuilder`.
///
/// It's the `ENCODE ...` part in `FORMAT ... ENCODE ...`

View on GitHub (pinned to 6469eb736d)

Solutions

1. Read `message` for the root cause and fix accordingly (most specific failure detail lives there)
2. Rewrite the Parquet files with standard encodings/compressions supported by the parser (e.g. snappy, plain encoding)
3. Validate the files with `parquet-tools`/`pyarrow` to detect corruption before ingestion
4. Check the table schema against the file schema for drifted/renamed columns

Example fix

# before: producer uses zstd + exotic dictionary encoding
pq.write_table(table, 'data.parquet', compression='zstd')
# after
pq.write_table(table, 'data.parquet', compression='snappy', use_dictionary=True)

// Validate Parquet files before ingestion (Python)
import pyarrow.parquet as pq
f = pq.ParquetFile('data.parquet')
assert f.metadata.num_rows > 0
assert all(c is not None for c in f.schema_arrow), 'corrupt schema'

match parquet_decoder.decode(batch) {
    Ok(rows) => rows,
    Err(e) if e.to_string().starts_with("Parquet parser error") => {
        tracing::warn!(%e, "parquet decode failed; quarantining file");
        quarantine(batch)
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Decoding a Parquet payload (e.g. Iceberg or Parquet-format source) fails inside the Parquet parser path; the parser reports the underlying reason via `message` — e.g. column type not supported, invalid page data, or requested field not found.

Common situations: Files written by producers with Parquet features unsupported by the parser (new compression codecs, nested maps/lists in unexpected form); schema evolution in the table the file belongs to; corrupted/truncated Parquet files.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Pb decode error: {0}
• Invalid vector item: {0} {1}
• {message}
• {0}
• avro parse unexpected eof

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/a4fa6a5523f76229. Report an issue: GitHub.