apache/iceberg · error · IllegalStateException

Equality delete file %s attached to main data file %s; the c

Error message

Equality delete file %s attached to main data file %s; the converter expects equality deletes only on the staging branch, converted to DVs on the target.

What it means

loadExistingDVs rejects equality delete files attached directly to main data files when staging and target branches differ. The converter expects equality deletes only on the staging branch, where they are read as rows and converted to deletion vectors on the target; an equality delete attached on the target branch is an unconverted leak the converter cannot reason about, so it throws IllegalStateException naming the delete and data files.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertReader.java:240

  }

  private PositionDeleteIndex loadExistingDVs(FileScanTask task, String dataFilePath) {
    List<DeleteFile> dvs = Lists.newArrayList();
    for (DeleteFile deleteFile : task.deletes()) {
      if (ContentFileUtil.isDV(deleteFile)) {
        dvs.add(deleteFile);
      } else if (deleteFile.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException(
            String.format(
                "V2 positional delete file %s attached to main data file %s; "
                    + "the converter expects a V3 target with deletion vectors only.",
                deleteFile.location(), dataFilePath));
      } else if (deleteFile.content() == FileContent.EQUALITY_DELETES && !stagingOnTargetBranch) {
        // When stagingBranch == targetBranch the target carries unconverted equality deletes; they
        // are indexed as rows here and converted via the planner's RESOLVE_DELETE commands. On a
        // separate target branch an attached equality delete means an unconverted delete leaked
        // onto the target, which the converter cannot reason about.
        throw new IllegalStateException(
            String.format(
                "Equality delete file %s attached to main data file %s; the converter expects "
                    + "equality deletes only on the staging branch, converted to DVs on the target.",
                deleteFile.location(), dataFilePath));
      }
    }

    if (dvs.isEmpty()) {
      return null;
    }

    return deleteLoader.loadPositionDeletes(dvs, dataFilePath);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Configure writers to send equality deletes to the staging branch (or enable staging on the target branch) while the converter is active.
2. Trigger an immediate conversion cycle that consumes the leaked equality delete, then resume normal maintenance.
3. Double-check the maintenance table-branch configuration so the target branch only holds converted DV output.
4. Temporarily disable writers to the target branch (or quiesce commits via the trigger lock) until the backlog of equality deletes is converted.

Example fix

// before
// staging and target differ, but an eq delete landed on target
// after
// either run staging on the same branch:
builder.stagingBranch(targetBranch);
// or pause merge-on-read writes to targetBranch until conversion catches up

boolean eqDeletesOnTarget = table.newScan().useBranch(targetBranch)
    .planFiles().stream()
    .flatMap(t -> t.deletes().stream())
    .anyMatch(d -> d.content() == FileContent.EQUALITY_DELETES);
Preconditions.checkState(!eqDeletesOnTarget, "Unconverted equality deletes on target branch");

try {
  loadExistingDVs(task, dataFilePath);
} catch (IllegalStateException e) {
  LOG.error("Unconverted equality deletes leaked to target: {}", e.getMessage());
  // trigger an immediate conversion cycle or pause writers
}

When it happens

Trigger: task.deletes() on a main-branch data file contains a DeleteFile with content == FileContent.EQUALITY_DELETES while stagingOnTargetBranch is false (separate staging branch configured).

Common situations: A normal Flink/Spark writer with merge-on-read equality deletes committed to the target branch while the converter maintenance job is enabled with a separate staging branch; misconfigured branch names so 'target' receives raw equality deletes.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• Equality delete file %s attached to main data file %s; the c
• Equality delete file %s attached to main data file %s; the c
• Cannot create or replace branch on non-Iceberg table: $table
• %s does not implement deleteFile
• %s does not implement addFile

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/70f4e30ba571e147. Report an issue: GitHub.