{"record":{"id":"604e117e688995c0","repo":"risingwavelabs/risingwave","slug":"position-delete-file-has-unsupported-format","errorCode":null,"errorMessage":"position-delete file {} has unsupported format {:?}; expected Puffin or Parquet","messagePattern":"position-delete file (.+?) has unsupported format (.+?); expected Puffin or Parquet","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/connector/src/sink/iceberg/position_delete.rs","lineNumber":142,"sourceCode":"            })?;\n            delete_vector.insert(pos as u64);\n        }\n    }\n\n    Ok(delete_vector)\n}\n\n/// Reads the deleted positions of a single position-delete `DataFile` regardless of on-disk format,\npub async fn read_position_deletes_from_file(\n    file_io: &FileIO,\n    delete_file: &DataFile,\n) -> Result<DeleteVector> {\n    match delete_file.file_format() {\n        DataFileFormat::Puffin => read_dv_positions_from_data_file(file_io, delete_file).await,\n        DataFileFormat::Parquet => {\n            read_parquet_position_deletes_from_file(file_io, delete_file).await\n        }\n        other => bail!(\n            \"position-delete file {} has unsupported format {:?}; expected Puffin or Parquet\",\n            delete_file.file_path(),\n            other\n        ),\n    }\n}\n\n/// Writes `delete_vector` as a single Puffin deletion-vector blob referencing `data_file_path`,\n/// and returns its [`DataFile`] metadata (content `PositionDeletes`, format `Puffin`) with\n/// `referenced_data_file` set.\npub async fn write_dv_puffin_file(\n    table: &Table,\n    location_generator: &DefaultLocationGenerator,\n    file_name_generator: &DefaultFileNameGenerator,\n    data_file_path: String,\n    delete_vector: &DeleteVector,\n    partition_key: Option<&PartitionKey>,\n) -> Result<DataFile> {","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/connector/src/sink/iceberg/position_delete.rs#L124-L160","documentation":"When reading Iceberg position-delete files to build delete vectors, only Puffin (deletion vector) and Parquet formats are supported. If the delete file's `DataFileFormat` is anything else (e.g. Avat/Orc-style formats), the reader bails because it cannot interpret the file's delete records.","triggerScenarios":"Encountering a position-delete entry in the Iceberg table metadata whose format is neither `puffin` nor `parquet` — typically from tables written by another engine with different delete-file formats — while the sink collects input DVs or flushes.","commonSituations":"Consuming an Iceberg table produced by third-party writers (Spark/Flink with ORC deletes); tables where a writer stored position deletes in an unsupported format; stale/mismatched metadata after a writer upgrade.","solutions":["Rewrite the source table's delete files to Puffin/Parquet (e.g. run a compaction/rewrite with a Spark/Flink job) so RisingWave can read them.","Re-create or rewrite the table avoiding non-Puffin/Parquet position deletes (use copy-on-write or DVs).","Check which engine/format produced the delete files and whether upgrading RisingWave adds support for that format.","File/verify support for the specific `DataFileFormat` you hit, since the match already covers Puffin and Parquet."],"exampleFix":"// before: engine wrote ORC position-deletes, unreadable here\n// after: compact/rewrite deletes to Puffin or Parquet, e.g. Spark:\n//   CALL catalog.system.rewrite_deletes(table => 'db.tbl', options => map('format','parquet'))","handlingStrategy":"validation","validationCode":"fn check_delete_file_format(fmt: &str) -> Result<(), String> {\n    match fmt {\n        \"puffin\" | \"parquet\" => Ok(()),\n        other => Err(format!(\"unsupported position-delete format {other}; rewrite deletes as puffin/parquet\")),\n    }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Inspect table metadata for delete file formats before pointing a RisingWave iceberg sink at an external table.","Configure upstream writers to use Puffin DVs or Parquet position deletes.","Run periodic compaction on the source table to rewrite legacy delete formats."],"tags":["iceberg","delete-files","puffin","parquet","unsupported-format"],"backgroundTag":"unsupported-operation","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}