apache/iceberg · error · IllegalArgumentException

Invalid statistics type: %s. Should be Map or Sketch

Error message

Invalid statistics type: %s. Should be Map or Sketch

What it means

RangePartitioner.delegatePartitioner selects the actual partitioner based on the GlobalStatistics type: Map yields MapRangePartitioner, Sketch yields SketchRangePartitioner. Any other type throws IllegalArgumentException('Invalid statistics type: %s. Should be Map or Sketch'), because the partitioner cannot interpret other statistics payloads.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/sink/shuffle/RangePartitioner.java:79

    }
  }

  private AtomicLong roundRobinCounter(int numPartitions) {
    if (roundRobinCounter == null) {
      // randomize the starting point to avoid synchronization across subtasks
      this.roundRobinCounter = new AtomicLong(new Random().nextInt(numPartitions));
    }

    return roundRobinCounter;
  }

  private Partitioner<RowData> delegatePartitioner(GlobalStatistics statistics) {
    if (statistics.type() == StatisticsType.Map) {
      return new MapRangePartitioner(schema, sortOrder, statistics.mapAssignment());
    } else if (statistics.type() == StatisticsType.Sketch) {
      return new SketchRangePartitioner(schema, sortOrder, statistics.rangeBounds());
    } else {
      throw new IllegalArgumentException(
          String.format("Invalid statistics type: %s. Should be Map or Sketch", statistics.type()));
    }
  }

  /**
   * Util method that handles rescale (write parallelism / numPartitions change).
   *
   * @param partition partition calculated based on the existing statistics
   * @param numPartitionsStatsCalculation number of partitions when the assignment was calculated
   *     based on
   * @param numPartitions current number of partitions
   * @return adjusted partition if necessary.
   */
  static int adjustPartitionWithRescale(
      int partition, int numPartitionsStatsCalculation, int numPartitions) {
    if (numPartitionsStatsCalculation <= numPartitions) {
      // no rescale or scale-up case.
      // new subtasks are ignored and not assigned any keys, which is sub-optimal and only

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Ensure the coordinator has completed at least one checkpoint so real global statistics (Map or Sketch) are distributed to writers
2. Align Iceberg connector versions across the job, especially when restoring from a savepoint
3. Log/inspect the type value printed in the message and trace where the GlobalStatistics instance was constructed
4. If the type is a valid new type from a newer release, upgrade the partitioner side

if (statistics != null && statistics.type() != StatisticsType.Map && statistics.type() != StatisticsType.Sketch) {
  throw new IllegalArgumentException("GlobalStatistics type not partitioner-compatible: " + statistics.type());
}

static boolean partitionerUsable(GlobalStatistics gs) {
  return gs != null && (gs.type() == StatisticsType.Map || gs.type() == StatisticsType.Sketch);
}

try {
  Partitioner<RowData> p = partitioner.partition(statistics);
} catch (IllegalArgumentException e) {
  // fall back to default (round-robin/hash) partitioning until valid statistics arrive
}

When it happens

Trigger: partition() receives a GlobalStatistics whose type() is neither Map nor Sketch — e.g. an uninitialized/placeholder GlobalStatistics, or statistics deserialized from a different connector version with an extra type.

Common situations: GlobalStatistics propagated to writers before the coordinator produced real statistics; version skew between writer tasks and coordinator; savepoint restore across incompatible connector versions.

Related errors

• Unsupported data statistics type: ${statisticsType}
• Unknown type for int field. Type name: ${className}
• Unknown type for long field. Type name: ${className}
• Unknown type for timestamp_ns: ${class}
• Unknown type for binary field. Type name: ${className}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/8f0f24d762566c35. Report an issue: GitHub.