{"record":{"id":"a4fa6a5523f76229","repo":"risingwavelabs/risingwave","slug":"parquet-parser-error-message","errorCode":null,"errorMessage":"Parquet parser error: {message}","messagePattern":"Parquet parser error: (.+?)","errorType":"exception","errorClass":"AccessError","httpStatus":null,"severity":"error","filePath":"src/connector/codec/src/decoder/mod.rs","lineNumber":50,"sourceCode":"        expected: String,\n        got: String,\n        value: String,\n    },\n    #[error(\"Unsupported data type `{ty}`\")]\n    UnsupportedType { ty: String },\n\n    /// CDC auto schema change specific error that may include table context\n    #[error(\"CDC auto schema change error: unsupported data type `{ty}` in table `{table_name}`\")]\n    CdcAutoSchemaChangeError { ty: String, table_name: String },\n\n    #[error(\"Unsupported additional column `{name}`\")]\n    UnsupportedAdditionalColumn { name: String },\n\n    #[error(\"Fail to convert protobuf Any into jsonb: {0}\")]\n    ProtobufAnyToJson(#[source] serde_json::Error),\n\n    /// Parquet parser specific errors\n    #[error(\"Parquet parser error: {message}\")]\n    ParquetParser { message: String },\n\n    /// Errors that are not categorized into variants above.\n    #[error(\"{message}\")]\n    Uncategorized { message: String },\n\n    #[error(transparent)]\n    NotImplemented(#[from] NotImplemented),\n    // NOTE: We intentionally don't embed `anyhow::Error` in `AccessError` since it happens\n    // in record-level and it might be too heavy to capture the backtrace\n    // when creating a new `anyhow::Error`.\n}\n\npub type AccessResult<T = Datum> = std::result::Result<T, AccessError>;\n\n/// Access to a field in the data structure. Created by `AccessBuilder`.\n///\n/// It's the `ENCODE ...` part in `FORMAT ... ENCODE ...`","sourceCodeStart":32,"sourceCodeEnd":68,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/codec/src/decoder/mod.rs#L32-L68","documentation":"AccessError::ParquetParser in src/connector/codec/src/decoder/mod.rs:50. A catch-all variant for errors specific to the Parquet parser, carrying a free-form `message`. It wraps any decoding problem encountered while reading Parquet-encoded payloads (schema mismatch, unsupported column encodings, corrupted data, projection errors).","triggerScenarios":"Decoding a Parquet payload (e.g. Iceberg or Parquet-format source) fails inside the Parquet parser path; the parser reports the underlying reason via `message` — e.g. column type not supported, invalid page data, or requested field not found.","commonSituations":"Files written by producers with Parquet features unsupported by the parser (new compression codecs, nested maps/lists in unexpected form); schema evolution in the table the file belongs to; corrupted/truncated Parquet files.","solutions":["Read `message` for the root cause and fix accordingly (most specific failure detail lives there)","Rewrite the Parquet files with standard encodings/compressions supported by the parser (e.g. snappy, plain encoding)","Validate the files with `parquet-tools`/`pyarrow` to detect corruption before ingestion","Check the table schema against the file schema for drifted/renamed columns"],"exampleFix":"# before: producer uses zstd + exotic dictionary encoding\npq.write_table(table, 'data.parquet', compression='zstd')\n# after\npq.write_table(table, 'data.parquet', compression='snappy', use_dictionary=True)","handlingStrategy":"try-catch","validationCode":"// Validate Parquet files before ingestion (Python)\nimport pyarrow.parquet as pq\nf = pq.ParquetFile('data.parquet')\nassert f.metadata.num_rows > 0\nassert all(c is not None for c in f.schema_arrow), 'corrupt schema'","typeGuard":null,"tryCatchPattern":"match parquet_decoder.decode(batch) {\n    Ok(rows) => rows,\n    Err(e) if e.to_string().starts_with(\"Parquet parser error\") => {\n        tracing::warn!(%e, \"parquet decode failed; quarantining file\");\n        quarantine(batch)\n    }\n    Err(e) => return Err(e.into()),\n}","preventionTips":["Write Parquet with widely supported codecs (snappy) and plain encodings","Checksum and size-verify files before ingestion to catch truncation","Contract-test file schemas against the RW table schema per producer release"],"tags":["parquet","parser","decoding"],"backgroundTag":"file-read-failed","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}