apache/iceberg · error · IllegalStateException

V2 positional delete file %s attached to main data file %s;

Error message

V2 positional delete file %s attached to main data file %s; the converter expects a V3 target with deletion vectors only.

What it means

Thrown by EqualityConvertReader.loadExistingDVs when a scan task for a main data file has an attached V2 positional delete file. The converter requires the V3 target branch to carry only deletion vectors for deletes; a positional delete on the target means legacy delete state that the converter cannot consume, so it aborts.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertReader.java:230

  @Override
  public void close() throws Exception {
    super.close();
    tableLoader.close();
  }

  private Schema appendRowPosition(Schema schema) {
    List<Types.NestedField> columns = Lists.newArrayList(schema.columns());
    columns.add(MetadataColumns.ROW_POSITION);
    return new Schema(columns);
  }

  private PositionDeleteIndex loadExistingDVs(FileScanTask task, String dataFilePath) {
    List<DeleteFile> dvs = Lists.newArrayList();
    for (DeleteFile deleteFile : task.deletes()) {
      if (ContentFileUtil.isDV(deleteFile)) {
        dvs.add(deleteFile);
      } else if (deleteFile.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException(
            String.format(
                "V2 positional delete file %s attached to main data file %s; "
                    + "the converter expects a V3 target with deletion vectors only.",
                deleteFile.location(), dataFilePath));
      } else if (deleteFile.content() == FileContent.EQUALITY_DELETES && !stagingOnTargetBranch) {
        // When stagingBranch == targetBranch the target carries unconverted equality deletes; they
        // are indexed as rows here and converted via the planner's RESOLVE_DELETE commands. On a
        // separate target branch an attached equality delete means an unconverted delete leaked
        // onto the target, which the converter cannot reason about.
        throw new IllegalStateException(
            String.format(
                "Equality delete file %s attached to main data file %s; the converter expects "
                    + "equality deletes only on the staging branch, converted to DVs on the target.",
                deleteFile.location(), dataFilePath));
      }
    }

    if (dvs.isEmpty()) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Rewrite/compact affected data files on the target branch so positional deletes are applied and removed (CALL rewrite_data_files / RewriteDataFilesAction).
  2. Stop or upgrade any remaining V2-style writers (old Spark/Flink jobs) writing positional deletes to the target branch.
  3. Verify table format-version is 3 and that no old snapshots with positional deletes were restored onto the branch.

Example fix

// before
spark.sql("ALTER TABLE db.t SET TBLPROPERTIES ('format-version'='3')");
runEqualityConvert(table); // old positional deletes still attached
// after
spark.sql("ALTER TABLE db.t SET TBLPROPERTIES ('format-version'='3')");
spark.sql("CALL cat.system.rewrite_data_files(table => 'db.t', strategy => 'sort')"); // absorb positional deletes
runEqualityConvert(table);
Defensive patterns

Strategy: validation

Validate before calling

// check for positional deletes on the target before conversion
table.scan().useSnapshot(table.snapshotForBranch(targetBranch).snapshotId())
    .planFiles().forEach(t -> t.deletes().forEach(d -> {
      if (d.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException("Target has V2 positional deletes: " + d.location());
      }
    }));

Try / catch

try {
  runEqualityConvertJob(table, cfg);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("positional delete file")) {
    LOG.error("Target branch carries V2 positional deletes; run rewrite_data_files to absorb them", e);
  } else throw e;
}

Prevention

When it happens

Trigger: Main-branch data files still referenced by positional delete files written before the table was upgraded to V3, or non-Flink/older writers committing positional deletes to the target branch, and the reader then loads existing deletes for such a file via existingDeletes.

Common situations: Upgrading format-version to 3 without rewriting/compacting old positional deletes away; Spark or Flink 1.x jobs with old Iceberg versions still writing to the target branch; restore of a pre-V3 snapshot onto the target branch.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/e054819b8db125a3. Report an issue: GitHub.