apache/iceberg · error · IllegalStateException

Equality delete file %s attached to main data file %s; the c

Error message

Equality delete file %s attached to main data file %s; the converter expects equality deletes only on the staging branch, converted to DVs on the target.

What it means

loadExistingDVs treats equality delete files attached to main data files as an error unless stagingBranch equals targetBranch. When staging is on a separate branch, any equality delete found on the target means an unconverted delete leaked onto the target branch, which the converter cannot reason about — it throws IllegalStateException naming the delete file and data file.

Source

Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertReader.java:240

  }

  private PositionDeleteIndex loadExistingDVs(FileScanTask task, String dataFilePath) {
    List<DeleteFile> dvs = Lists.newArrayList();
    for (DeleteFile deleteFile : task.deletes()) {
      if (ContentFileUtil.isDV(deleteFile)) {
        dvs.add(deleteFile);
      } else if (deleteFile.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException(
            String.format(
                "V2 positional delete file %s attached to main data file %s; "
                    + "the converter expects a V3 target with deletion vectors only.",
                deleteFile.location(), dataFilePath));
      } else if (deleteFile.content() == FileContent.EQUALITY_DELETES && !stagingOnTargetBranch) {
        // When stagingBranch == targetBranch the target carries unconverted equality deletes; they
        // are indexed as rows here and converted via the planner's RESOLVE_DELETE commands. On a
        // separate target branch an attached equality delete means an unconverted delete leaked
        // onto the target, which the converter cannot reason about.
        throw new IllegalStateException(
            String.format(
                "Equality delete file %s attached to main data file %s; the converter expects "
                    + "equality deletes only on the staging branch, converted to DVs on the target.",
                deleteFile.location(), dataFilePath));
      }
    }

    if (dvs.isEmpty()) {
      return null;
    }

    return deleteLoader.loadPositionDeletes(dvs, dataFilePath);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Redirect all writers that produce equality deletes to the staging branch (or upgrade them to v3 DV writers on main).
  2. Pause legacy equality-delete writers while the conversion runs, then rerun the cycle.
  3. Verify the branch routing of every writer job; only the converter's staging flow may add equality deletes.
  4. If the layout should allow it, set stagingBranch == targetBranch so unconverted deletes are indexed and converted via RESOLVE_DELETE commands instead of erroring.

Example fix

// before
EqualityDeleteConversionConfig.builder()
    .stagingBranch("__iceberg_edc_staging")
    .targetBranch("main") // legacy v2 writer still commits eq-deletes to main
    .build();
// after
// stop the legacy writer, or align branches so unconverted deletes are handled:
EqualityDeleteConversionConfig.builder()
    .stagingBranch("main")
    .targetBranch("main")
    .build();
Defensive patterns

Strategy: validation

Validate before calling

// confirm no unconverted equality deletes exist on the target when staging != target
if (!stagingBranch.equals(targetBranch)) {
  for (FileScanTask t : table.newScan().useBranch(targetBranch).planFiles()) {
    for (DeleteFile d : t.deletes()) {
      if (d.content() == FileContent.EQUALITY_DELETES) {
        throw new IllegalStateException("Unconverted eq-delete on target: " + d.location());
      }
    }
  }
}

Prevention

When it happens

Trigger: existingDeletes -> loadExistingDVs finds a FileContent.EQUALITY_DELETES delete attached to a main data file while stagingBranch != targetBranch, i.e., a writer committed equality deletes directly to the target branch outside the conversion flow.

Common situations: A legacy v2 writer (Spark or Flink with equality deletes) writing to the target branch concurrently with the converter; misconfigured writer targeting the wrong branch; branch routing mistakes in multi-job setups.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/368609637050344a. Report an issue: GitHub.