position-delete writer produced invalid file count for {data

Error message

position-delete writer produced invalid file count for {data_file_path}

What it means

This error is thrown when the iceberg-rs position-delete `DataFileWriter::close()` returns a number of DataFile builders other than exactly one. The code uses a slice pattern `let [mut builder] = data_files.try_into()` to destructurally assert the invariant that closing a position-delete writer yields exactly one file; any other count (0 or >1) breaks the writer contract and aborts the sink with this anyhow error.

Source

Thrown at src/connector/src/sink/iceberg/position_delete.rs:262

        if positions.len() == POSITION_DELETE_WRITE_CHUNK_SIZE {
            write_position_delete_chunk(
                &mut writer,
                &arrow_schema,
                &data_file_path,
                std::mem::take(&mut positions),
            )
            .await?;
            positions.reserve(POSITION_DELETE_WRITE_CHUNK_SIZE);
        }
    }
    if !positions.is_empty() {
        write_position_delete_chunk(&mut writer, &arrow_schema, &data_file_path, positions).await?;
    }

    let data_files = writer.close().await?;
    // `close` will yield exactly one builder here.
    let [mut builder] = data_files.try_into().map_err(|_| {
        anyhow!("position-delete writer produced invalid file count for {data_file_path}")
    })?;

    // `ParquetWriter` builds the file as `DataContentType::Data` with an empty partition; override
    // those for a file-scoped V2 position-delete file and attach `referenced_data_file`.
    builder
        .content(DataContentType::PositionDeletes)
        .referenced_data_file(Some(data_file_path));
    if let Some(partition_key) = partition_key {
        builder
            .partition(partition_key.data().clone())
            .partition_spec_id(partition_key.spec().spec_id());
    }
    builder
        .build()
        .context("Failed to build position-delete file metadata")
}

/// Writes one chunk of `positions` as a `(file_path, pos)` batch into `writer`. Every row shares

View on GitHub (pinned to 6469eb736d)

Solutions

Check that the input `positions` collection is non-empty before calling write_parquet_position_delete_file, or short-circuit returning None for empty input
Verify target_file_size_mb for the sink is large enough that a single position-delete file never rolls into multiple files
Inspect the iceberg-rs version pinned in Cargo.toml for changes to DataFileWriter::close() semantics and pin to a version where close yields exactly one builder
If the invariant is intentionally relaxable, replace the slice pattern with a match on data_files.as_slice() handling 0 and n>1 cases explicitly

Example fix

// before
let [mut builder] = data_files.try_into().map_err(|_| {
    anyhow!("position-delete writer produced invalid file count for {data_file_path}")
})?;
// after
let mut builders = data_files;
if builders.is_empty() {
    return Ok(None); // nothing written
}
anyhow::ensure!(builders.len() == 1, "expected 1 position-delete file, got {}", builders.len());
let builder = builders.pop().unwrap();

Defensive patterns

Strategy: validation

Validate before calling

if positions.is_empty() {
    return Ok(None); // nothing to write, skip writer close entirely
}

Type guard

fn single_builder(files: Vec<iceberg::spec::DataFileBuilder>) -> Option<iceberg::spec::DataFileBuilder> {
    let mut it = files.into_iter();
    match (it.next(), it.next()) {
        (Some(b), None) => Some(b),
        _ => None,
    }
}

Try / catch

match data_files.try_into() {
    Ok([builder]) => builder,
    _ => return Err(anyhow!("position-delete writer produced invalid file count")),
}

Prevention

Skip file creation entirely when the positions list is empty
Keep target_file_size_mb large enough that one position-delete batch never rolls into multiple files
Pin the iceberg-rs version and re-read close() semantics on every upgrade
Add a unit test asserting close() yields exactly one builder for a typical chunk

When it happens

Trigger: Calling write_parquet_position_delete_file when the underlying iceberg `DataFileWriter` (via RollingFileWriterBuilder) closes into zero files (empty write, IO failure swallowed) or multiple files because the rolling writer split output across the target file size, violating the code's one-builder assumption.

Common situations: A data file with zero delete positions being flushed; very large position-delete payloads exceeding target_file_size_mb causing the rolling writer to emit several parquet files; upgrading the iceberg-rs crate and its close() semantics changing.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of risingwavelabs/risingwave@6469eb736d (2026-09-11). Data as JSON: /api/errors/c3ff52b3e1fc995d. Report an issue: GitHub.