apache/iceberg · error · IllegalArgumentException

Invalid statistics type: ${type}. Should be Map or Sketch

Error message

Invalid statistics type: ${type}. Should be Map or Sketch

What it means

RangePartitioner.delegatePartitioner selects a concrete partitioner based on the GlobalStatistics type: Map uses MapRangePartitioner, Sketch uses SketchRangePartitioner. Any other type is invalid because range partitioning requires one of these statistics forms. This is a defensive dispatch guard.

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/RangePartitioner.java:79

    }
  }

  private AtomicLong roundRobinCounter(int numPartitions) {
    if (roundRobinCounter == null) {
      // randomize the starting point to avoid synchronization across subtasks
      this.roundRobinCounter = new AtomicLong(new Random().nextInt(numPartitions));
    }

    return roundRobinCounter;
  }

  private Partitioner<RowData> delegatePartitioner(GlobalStatistics statistics) {
    if (statistics.type() == StatisticsType.Map) {
      return new MapRangePartitioner(schema, sortOrder, statistics.mapAssignment());
    } else if (statistics.type() == StatisticsType.Sketch) {
      return new SketchRangePartitioner(schema, sortOrder, statistics.rangeBounds());
    } else {
      throw new IllegalArgumentException(
          String.format("Invalid statistics type: %s. Should be Map or Sketch", statistics.type()));
    }
  }

  /**
   * Util method that handles rescale (write parallelism / numPartitions change).
   *
   * @param partition partition calculated based on the existing statistics
   * @param numPartitionsStatsCalculation number of partitions when the assignment was calculated
   *     based on
   * @param numPartitions current number of partitions
   * @return adjusted partition if necessary.
   */
  static int adjustPartitionWithRescale(
      int partition, int numPartitionsStatsCalculation, int numPartitions) {
    if (numPartitionsStatsCalculation <= numPartitions) {
      // no rescale or scale-up case.
      // new subtasks are ignored and not assigned any keys, which is sub-optimal and only

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Use matching connector versions for job submission and all TaskManagers.
2. Restart from clean state instead of restoring an incompatible checkpoint.
3. Confirm the GlobalStatistics instance was produced by this library's coordinator.

StatisticsType t = statistics.type();
if (t != StatisticsType.Map && t != StatisticsType.Sketch) {
  throw new IllegalArgumentException("Range partitioning requires Map or Sketch statistics, got " + t);
}

boolean hasUsableStatistics(GlobalStatistics s) {
  return s.type() == StatisticsType.Map || s.type() == StatisticsType.Sketch;
}

try {
  partitioner = delegatePartitioner(globalStatistics);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Invalid statistics type")) {
    partitioner = fallbackRoundRobinPartitioner;
  } else throw e;
}

When it happens

Trigger: Feeding a GlobalStatistics with an unrecognized StatisticsType into partition(), typically from deserialized state written by an incompatible connector version or a corrupt statistics payload.

Common situations: Checkpoint/savepoint restore across connector versions, or mixed jar versions among TaskManagers during a shuffle job.

Related errors

• Altering partition keys is not supported yet.
• Failed to encode partition
• Failed to decode partition
• Altering partition keys is not supported yet.
• Invalid operator event type: ${eventType}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/bbc0fb65ad1b1ed4. Report an issue: GitHub.