apache/iceberg · error · IllegalStateException

Equality delete file %s attached to main data file %s; the c

Error message

Equality delete file %s attached to main data file %s; the converter expects equality deletes only on the staging branch, converted to DVs on the target.

What it means

Thrown by EqualityConvertReader.loadExistingDVs when an equality delete file is attached to a main data file while the staging branch is separate from the target branch (stagingOnTargetBranch is false). The converter expects equality deletes only on the staging branch, already converted to DVs on the target; an attached equality delete on the target means an unconverted delete leaked there and cannot be handled.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/maintenance/operator/EqualityConvertReader.java:240

  }

  private PositionDeleteIndex loadExistingDVs(FileScanTask task, String dataFilePath) {
    List<DeleteFile> dvs = Lists.newArrayList();
    for (DeleteFile deleteFile : task.deletes()) {
      if (ContentFileUtil.isDV(deleteFile)) {
        dvs.add(deleteFile);
      } else if (deleteFile.content() == FileContent.POSITION_DELETES) {
        throw new IllegalStateException(
            String.format(
                "V2 positional delete file %s attached to main data file %s; "
                    + "the converter expects a V3 target with deletion vectors only.",
                deleteFile.location(), dataFilePath));
      } else if (deleteFile.content() == FileContent.EQUALITY_DELETES && !stagingOnTargetBranch) {
        // When stagingBranch == targetBranch the target carries unconverted equality deletes; they
        // are indexed as rows here and converted via the planner's RESOLVE_DELETE commands. On a
        // separate target branch an attached equality delete means an unconverted delete leaked
        // onto the target, which the converter cannot reason about.
        throw new IllegalStateException(
            String.format(
                "Equality delete file %s attached to main data file %s; the converter expects "
                    + "equality deletes only on the staging branch, converted to DVs on the target.",
                deleteFile.location(), dataFilePath));
      }
    }

    if (dvs.isEmpty()) {
      return null;
    }

    return deleteLoader.loadPositionDeletes(dvs, dataFilePath);
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

Ensure all delete-producing writers commit to the configured staging branch, not the target branch; check the maintenance job's branch configuration.
Run the conversion job so the leaked equality deletes on the target are converted (or use stagingOnTargetBranch mode if that matches your topology).
Compact/rewrite the affected target data files to eliminate the attached equality deletes, then rerun conversion.

Example fix

// before: writers on target branch
writeJob(table).branch(targetBranch).execute(); // equality deletes land on target
runEqualityConvert(table);
// after
writeJob(table).branch(stagingBranch).execute(); // deletes staged, converted to DVs
runEqualityConvert(table);

Defensive patterns

Strategy: validation

Validate before calling

// confirm delete-producing writers commit to the staging branch
if (!stagingBranch.equals(deleteWriterBranch)) {
  throw new IllegalArgumentException("Writers must commit equality deletes to staging branch " + stagingBranch);
}

Try / catch

try {
  runEqualityConvertJob(table, cfg);
} catch (IllegalStateException e) {
  if (e.getMessage().startsWith("Equality delete file")) {
    LOG.error("Unconverted equality deletes on target; fix writer branch routing or run conversion", e);
  } else throw e;
}

Prevention

Route all delete-producing jobs to the staging branch, never the target
Double-check staging/target branch properties in the maintenance config
Run conversion promptly after deletes are staged so nothing leaks
Use stagingOnTargetBranch mode only when your topology actually commits to the same branch

When it happens

Trigger: Unconverted equality deletes committed directly to the target branch (e.g. by a job writing to targetBranch instead of stagingBranch, or stagingBranch == wrong branch configuration), while the converter runs in separate-branch mode and reads deletes for a main data file via existingDeletes.

Common situations: Misconfigured maintenance table properties pointing staging and target branches incorrectly; a normal Flink/Spark write job committing equality deletes straight to the target branch; recovery/rollback placing pre-conversion snapshots on the target.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/296343c5a41e6dbd. Report an issue: GitHub.