apache/beam · error · IllegalStateException

Unexpected content type for DeleteFile

Error message

Unexpected content type for DeleteFile: {}

What it means

SerializableDeleteFile.createDeleteFile() rebuilds a DeleteFile from its serialized form and switches on the content type (POSITION_DELETES, EQUALITY_DELETES, etc.). An unknown content type falls into the default branch and throws IllegalStateException. This indicates corrupted or future-version serialization data the current connector cannot reconstruct.

Solutions

1. Pin a single version of the Beam IO iceberg connector across all pipeline stages and workers.
2. Inspect the serialized contentType value to identify where the unexpected value originated.
3. Re-serialize affected state with the current library version.
4. If a new content type is legitimate, add a builder branch for it in createDeleteFile().

Example fix

// before
// running pipeline with beam-sdks-java-io-iceberg 2.61 workers after writing state with 2.63
// after
// align versions everywhere
// implementation 'org.apache.beam:beam-sdks-java-io-iceberg:2.63.0' // same version on writer and reader

// check serialization compatibility before deserializing
if (!KNOWN_CONTENT_TYPES.contains(serializableDeleteFile.getContentType())) {
  throw new IllegalStateException("Unknown DeleteFile content type: " + serializableDeleteFile.getContentType());
}

try {
  DeleteFile df = SerializableDeleteFile.reconstruct(serialized);
} catch (IllegalStateException e) {
  throw new IllegalStateException("Incompatible serialized delete file (version mismatch?): " + e.getMessage(), e);
}

When it happens

Trigger: Deserializing a SerializableDeleteFile whose getContentType() is not a recognized DeleteFile content type, e.g. data serialized by a newer Iceberg/Beam version adding a new content type, or hand-edited/corrupted serialized state.

Common situations: Mixing Beam connector versions across pipeline stages (old writer, new reader) or vice versa; restoring a checkpoint/Fusion state produced by a different library version; manual modification of serialized JSON fields.

Related errors

• BATCH scan not supported
• Cannot access state in non-window observing context.
• Cannot read format
• Encountered an error when creating a RecordWriter for table
• Encountered an error when parsing filter

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/77d167088394f9a8. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/SerializableDeleteFile.java:327

          for (int i = 0; i < fieldIds.size(); i++) {
            equalityFieldIds[i] = fieldIds.get(i);
          }
        }
        SortOrder sortOrder = SortOrder.unsorted();
        if (sortOrders != null) {
          sortOrder =
              checkStateNotNull(
                  sortOrders.get(getSortOrderId()),
                  "This DeleteFile was originally created with sort order id '%s', "
                      + "but table only has sort order ids: %s.",
                  getSortOrderId(),
                  sortOrders.keySet());
        }
        deleteFileBuilder =
            deleteFileBuilder.ofEqualityDeletes(equalityFieldIds).withSortOrder(sortOrder);
        break;
      default:
        throw new IllegalStateException(
            "Unexpected content type for DeleteFile: " + getContentType());
    }

    // contentOffset / contentSizeInBytes really are Puffin-only: build() rejects a non-null value
    // for either on any other format, and requires both (plus referencedDataFile) on Puffin.
    if (getFileFormat().equalsIgnoreCase(FileFormat.PUFFIN.name())) {
      deleteFileBuilder =
          deleteFileBuilder
              .withContentOffset(checkStateNotNull(getContentOffset()))
              .withContentSizeInBytes(checkStateNotNull(getContentSizeInBytes()));
    }
    return deleteFileBuilder.build();
  }

  private StructLike partition(PartitionSpec spec) {
    return (StructLike)
        SingleValueParser.fromJson(spec.partitionType(), checkStateNotNull(getJsonPartition()));
  }

View on GitHub (pinned to 12126d8942)