apache/iceberg · error · IllegalStateException

V2 positional delete file %s attached to main data file %s;

Error message

V2 positional delete file %s attached to main data file %s; the converter expects a V3 target with deletion vectors only.

What it means

loadExistingDVs collects the deletion vectors attached to each main data file in the scan task. If a V2 positional delete file is attached to a data file on the target, the converter cannot represent those deletes as DVs and throws IllegalStateException naming both files. The target table is expected to be format v3 with positional deletes already expressed as deletion vectors.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertReader.java:230

  @Override
  public void close() throws Exception {
    super.close();
    tableLoader.close();
  }

  private Schema appendRowPosition(Schema schema) {
    List<Types.NestedField> columns = Lists.newArrayList(schema.columns());
    columns.add(MetadataColumns.ROW_POSITION);
    return new Schema(columns);
  }

  private PositionDeleteIndex loadExistingDVs(FileScanTask task, String dataFilePath) {
    List<DeleteFile> dvs = Lists.newArrayList();
    for (DeleteFile deleteFile : task.deletes()) {
      if (ContentFileUtil.isDV(deleteFile)) {
        dvs.add(deleteFile);
      } else if (deleteFile.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException(
            String.format(
                "V2 positional delete file %s attached to main data file %s; "
                    + "the converter expects a V3 target with deletion vectors only.",
                deleteFile.location(), dataFilePath));
      } else if (deleteFile.content() == FileContent.EQUALITY_DELETES && !stagingOnTargetBranch) {
        // When stagingBranch == targetBranch the target carries unconverted equality deletes; they
        // are indexed as rows here and converted via the planner's RESOLVE_DELETE commands. On a
        // separate target branch an attached equality delete means an unconverted delete leaked
        // onto the target, which the converter cannot reason about.
        throw new IllegalStateException(
            String.format(
                "Equality delete file %s attached to main data file %s; the converter expects "
                    + "equality deletes only on the staging branch, converted to DVs on the target.",
                deleteFile.location(), dataFilePath));
      }
    }

    if (dvs.isEmpty()) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

Stop all v2 writers from committing to the target branch; upgrade them to v3 DV writers.
Rewrite/convert existing positional deletes to DVs (e.g., via rewrite delete files / compaction with DV support) before running the converter.
Verify the table's format version is 3 and all engines writing to it support DVs.
Run the converter only after a clean cycle shows no POSITION_DELETES content on the target branch.

Example fix

// before
// legacy writer on same table
spark.sql("ALTER TABLE t SET TBLPROPERTIES ('format-version'='3')");
convertEqualityDeletes(table); // fails: v2 writers still add position deletes
// after
// 1) stop legacy writers, 2) convert existing position deletes to DVs,
// 3) then convert
spark.sql("CALL catalog.system.rewrite_delete_files(t => 't', options => map('format','v3'))");
convertEqualityDeletes(table);

Defensive patterns

Strategy: validation

Validate before calling

// confirm target branch has no V2 positional deletes before converting
for (FileScanTask t : table.newScan().planFiles()) {
  for (DeleteFile d : t.deletes()) {
    if (d.content() == FileContent.POSITION_DELETES) {
      throw new IllegalStateException("Position deletes remain on target: " + d.location());
    }
  }
}

Prevention

Upgrade all writers to format v3 DV output before enabling conversion.
Rewrite existing position deletes to DVs first (rewrite delete files action).
Verify table.formatVersion() == 3 and writer compatibility.
Pre-scan target branch deletes as a gate before each conversion cycle.

When it happens

Trigger: existingDeletes -> loadExistingDVs encounters task.deletes() containing a file with content POSITION_DELETES while scanning main data files.

Common situations: The table was upgraded toward v3 but older writers (Spark/Flink v2 writers) still commit positional delete files; a merge-on-read v2 table being converted before legacy position deletes were rewritten to DVs; sharing the target branch with a legacy writer.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/67313ad83c2fec47. Report an issue: GitHub.