apache/iceberg · error · IllegalArgumentException

Unsupported data statistics type: " + statisticsType

Error message

Unsupported data statistics type: " + statisticsType

What it means

DataStatisticsSerializer (type serializer for DataStatistics) supports a fixed set of statistics types (e.g. sketch-based reservoir statistics). Deserializing bytes whose statisticsType byte is unrecognized throws IllegalArgumentException because it cannot reconstruct the DataStatistics instance.

Solutions

1. Restore with the same or newer Iceberg version that wrote the state
2. Rewrite the savepoint with the current version before restoring
3. Check state backend integrity if the type byte appears corrupted

// restore only state written by a known-compatible version
if (!runningVersion.equals(stateWrittenVersion)) {
  fail("DataStatistics state version mismatch");
}

try {
  stats = serializer.deserialize(version, bytes);
} catch (IllegalArgumentException e) {
  throw new StateMigrationException("Unsupported statistics type in state", e);
}

When it happens

Trigger: Restoring checkpoints/savepoints written by a newer Iceberg version that serialized a statistics type the current runtime does not know, or corrupt state bytes flipping the type field.

Common situations: Version skew between writer and restorer jobs using the sink-shuffle data statistics feature; corrupted state backend data.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• Could not deserialize the WriteResult object
• Failed to deserialize IcebergSourceSplit. Encountered…
• Failed to deserialize IcebergSourceSplit. Encountered…
• Failed to deserialize the split.
• Unknown serialize version:

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/79edadfd60632ade. Report an issue: GitHub.

Appendix: source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/DataStatisticsSerializer.java:91

  public DataStatistics copy(DataStatistics obj) {
    StatisticsType statisticsType = obj.type();
    if (statisticsType == StatisticsType.Map) {
      MapDataStatistics from = (MapDataStatistics) obj;
      Map<SortKey, Long> fromStats = (Map<SortKey, Long>) from.result();
      Map<SortKey, Long> toStats = Maps.newHashMap(fromStats);
      return new MapDataStatistics(toStats);
    } else if (statisticsType == StatisticsType.Sketch) {
      // because ReservoirItemsSketch doesn't expose enough public methods for cloning,
      // this implementation adopted the less efficient serialization and deserialization.
      SketchDataStatistics from = (SketchDataStatistics) obj;
      ReservoirItemsSketch<SortKey> fromStats = (ReservoirItemsSketch<SortKey>) from.result();
      byte[] bytes = fromStats.toByteArray(sketchSerializer);
      Memory memory = Memory.wrap(bytes);
      ReservoirItemsSketch<SortKey> toStats =
          ReservoirItemsSketch.heapify(memory, sketchSerializer);
      return new SketchDataStatistics(toStats);
    } else {
      throw new IllegalArgumentException("Unsupported data statistics type: " + statisticsType);
    }
  }

  @Override
  public DataStatistics copy(DataStatistics from, DataStatistics reuse) {
    // not much benefit to reuse
    return copy(from);
  }

  @Override
  public int getLength() {
    return -1;
  }

  @SuppressWarnings("unchecked")
  @Override
  public void serialize(DataStatistics obj, DataOutputView target) throws IOException {
    StatisticsType statisticsType = obj.type();

View on GitHub (pinned to 86d9c8fc54)