apache/iceberg · error · IllegalStateException

V2 positional delete file %s attached to main data file %s;

Error message

V2 positional delete file %s attached to main data file %s; the converter expects a V3 target with deletion vectors only.

What it means

loadExistingDVs collects the deletion vectors attached to a main data file and hard-fails if it finds a V2 positional delete file instead. The equality-delete converter assumes the target table is format V3 with deletion vectors only; a positional delete attached to a main data file breaks that assumption and is thrown as IllegalStateException naming both files.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertReader.java:230

  @Override
  public void close() throws Exception {
    super.close();
    tableLoader.close();
  }

  private Schema appendRowPosition(Schema schema) {
    List<Types.NestedField> columns = Lists.newArrayList(schema.columns());
    columns.add(MetadataColumns.ROW_POSITION);
    return new Schema(columns);
  }

  private PositionDeleteIndex loadExistingDVs(FileScanTask task, String dataFilePath) {
    List<DeleteFile> dvs = Lists.newArrayList();
    for (DeleteFile deleteFile : task.deletes()) {
      if (ContentFileUtil.isDV(deleteFile)) {
        dvs.add(deleteFile);
      } else if (deleteFile.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException(
            String.format(
                "V2 positional delete file %s attached to main data file %s; "
                    + "the converter expects a V3 target with deletion vectors only.",
                deleteFile.location(), dataFilePath));
      } else if (deleteFile.content() == FileContent.EQUALITY_DELETES && !stagingOnTargetBranch) {
        // When stagingBranch == targetBranch the target carries unconverted equality deletes; they
        // are indexed as rows here and converted via the planner's RESOLVE_DELETE commands. On a
        // separate target branch an attached equality delete means an unconverted delete leaked
        // onto the target, which the converter cannot reason about.
        throw new IllegalStateException(
            String.format(
                "Equality delete file %s attached to main data file %s; the converter expects "
                    + "equality deletes only on the staging branch, converted to DVs on the target.",
                deleteFile.location(), dataFilePath));
      }
    }

    if (dvs.isEmpty()) {

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Run a rewrite/delete-file compaction that removes legacy positional deletes (converts them to DVs) before enabling the equality converter.
  2. Verify table format version is 3 and that no writer with format-version 2 is appending to the target branch during maintenance.
  3. Point the converter at a branch that only receives V3/DV writes, or first fully rewrite affected data files.

Example fix

// before
// table.formatVersion()==2, positional deletes attached
TableScan scan = table.newScan();

// after
Table upgraded = ...; // migrate to format version 3 first
Preconditions.checkArgument(upgraded.ops().current().formatVersion() >= 3,
    "Equality converter requires format version >= 3");
Defensive patterns

Strategy: validation

Validate before calling

Preconditions.checkArgument(
    table.ops().current().formatVersion() >= 3,
    "Equality converter requires format version 3+ with deletion vectors");
// optionally scan once and assert no POSITION_DELETES deletes are attached

Try / catch

try {
  loadExistingDVs(task, dataFilePath);
} catch (IllegalStateException e) {
  LOG.error("Legacy V2 positional deletes present: {}", e.getMessage());
  // run compaction that converts positional deletes to DVs first
}

Prevention

When it happens

Trigger: A data file scan task on the target/main branch has task.deletes() containing a DeleteFile with content == FileContent.POSITION_DELETES, i.e. the table still has V2-style .pos deletes when the converter runs.

Common situations: Table was upgraded to format version 3 but legacy positional deletes from V2 writes are still attached; another writer (older engine version) keeps producing positional deletes on the same branch while maintenance runs.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/d7c0f09bd2cdab74. Report an issue: GitHub.