apache/iceberg · error · IllegalStateException

Equality delete file %s attached to main data file %s; the c

Error message

Equality delete file %s attached to main data file %s; the converter expects equality deletes only on the staging branch, converted to DVs on the target.

What it means

The loader found an equality delete file attached to a main-branch data file while stagingBranch != targetBranch. Equality deletes are only expected on the staging branch (where the converter converts them to DVs); on a separate target branch their presence means an unconverted delete leaked onto the target and the converter cannot reason about it, so it fails fast.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertReader.java:240

  }

  private PositionDeleteIndex loadExistingDVs(FileScanTask task, String dataFilePath) {
    List<DeleteFile> dvs = Lists.newArrayList();
    for (DeleteFile deleteFile : task.deletes()) {
      if (ContentFileUtil.isDV(deleteFile)) {
        dvs.add(deleteFile);
      } else if (deleteFile.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException(
            String.format(
                "V2 positional delete file %s attached to main data file %s; "
                    + "the converter expects a V3 target with deletion vectors only.",
                deleteFile.location(), dataFilePath));
      } else if (deleteFile.content() == FileContent.EQUALITY_DELETES && !stagingOnTargetBranch) {
        // When stagingBranch == targetBranch the target carries unconverted equality deletes; they
        // are indexed as rows here and converted via the planner's RESOLVE_DELETE commands. On a
        // separate target branch an attached equality delete means an unconverted delete leaked
        // onto the target, which the converter cannot reason about.
        throw new IllegalStateException(
            String.format(
                "Equality delete file %s attached to main data file %s; the converter expects "
                    + "equality deletes only on the staging branch, converted to DVs on the target.",
                deleteFile.location(), dataFilePath));
      }
    }

    if (dvs.isEmpty()) {
      return null;
    }

    return deleteLoader.loadPositionDeletes(dvs, dataFilePath);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

Redirect all equality-delete-writing jobs to the staging branch (or the same branch used for conversion).
Re-branch the target from a snapshot without unconverted equality deletes, then run the conversion.
Set stagingBranch == targetBranch if your pipeline intentionally keeps unconverted equality deletes on the target (they will be indexed and resolved instead).
Audit branch configuration of sink and maintenance jobs so deletes never land on the target branch directly.

Example fix

// before
FlinkSink.forRowData(...).branch("main")... // equality deletes land on target
// after
FlinkSink.forRowData(...).branch("staging")... // conversion converts them to DVs on target

Defensive patterns

Strategy: validation

Validate before calling

// pre-flight: equality deletes must be absent on a distinct target branch
if (!stagingOnTargetBranch) {
  table.snapshot(targetBranch).deleteFiles(table.io()).forEach(f ->
    checkState(f.content() != FileContent.EQUALITY_DELETES,
        "eq delete leaked onto target: " + f.location()));
}

Type guard

boolean targetFreeOfEqualityDeletes(Table table, String branch) {
  return table.snapshot(branch).deleteFiles(table.io()).stream()
      .noneMatch(f -> f.content() == FileContent.EQUALITY_DELETES);
}

Try / catch

try {
  convert(table);
} catch (IllegalStateException e) {
  if (e.getMessage().startsWith("Equality delete file")) {
    rerouteWriterToStagingBranch(); rerun();
  } else throw e;
}

Prevention

Route all equality-delete-producing sinks (upsert mode, MERGE) to the staging branch.
Document branch ownership: target receives only DVs and appends.
Audit the writing job's branch config before enabling conversion.
Run a pre-flight manifest audit for equality deletes on the target branch.

When it happens

Trigger: EqualityConvertReader.loadExistingDVs() encounters DeleteFile content EQUALITY_DELETES on the target branch while stagingOnTargetBranch is false (staging and target are different branches).

Common situations: A writer configured with equality deletes (e.g. upsert-mode Flink sink or Spark MERGE) committed directly to the target branch instead of the staging branch; misconfigured branch routing of the writing job.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/ff06a6baf5fe3d65. Report an issue: GitHub.