apache/iceberg · info
Fallback to use 'none' distribution mode, because there are
Error message
Fallback to use 'none' distribution mode, because there are no equality fields set and table is unpartitioned
What it means
In the new IcebergSink builder's hash-distribution path, when no equality fields are configured and the table is unpartitioned there is no key to hash on, so the sink logs this warning and falls back to 'none' distribution (returns the input unchanged).
Source
Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java:1001
throw new RuntimeException("Unrecognized " + WRITE_DISTRIBUTION_MODE + ": " + mode);
}
}
private DataStream<RowData> distributeDataStreamByNoneDistributionMode(
DataStream<RowData> input, Schema iSchema) {
if (equalityFieldIds.isEmpty()) {
return input;
} else {
LOG.info("Distribute rows by equality fields, because there are equality fields set");
return input.keyBy(new EqualityFieldKeySelector(iSchema, flinkRowType, equalityFieldIds));
}
}
private DataStream<RowData> distributeDataStreamByHashDistributionMode(
DataStream<RowData> input, Schema iSchema, PartitionSpec partitionSpec) {
if (equalityFieldIds.isEmpty()) {
if (partitionSpec.isUnpartitioned()) {
LOG.warn(
"Fallback to use 'none' distribution mode, because there are no equality fields set "
+ "and table is unpartitioned");
return input;
} else {
return input.keyBy(new PartitionKeySelector(partitionSpec, iSchema, flinkRowType));
}
} else {
if (partitionSpec.isUnpartitioned()) {
LOG.info(
"Distribute rows by equality fields, because there are equality fields set "
+ "and table is unpartitioned");
return input.keyBy(new EqualityFieldKeySelector(iSchema, flinkRowType, equalityFieldIds));
} else {
for (PartitionField partitionField : partitionSpec.fields()) {
Preconditions.checkState(
equalityFieldIds.contains(partitionField.sourceId()),
"In 'hash' distribution mode with equality fields set, source column '%s' of partition field '%s' "
+ "should be included in equality fields: '%s'",View on GitHub (pinned to 86d9c8fc54)
Solutions
- Use DistributionMode.NONE for unpartitioned append-only streams
- Add partitions to the table so HASH can key by partition
- Configure equality fields/primary key for upsert workloads
- Drop the explicit hash distribution setting
Example fix
// before IcebergSink.forRowData(input).distributionMode(DistributionMode.HASH)... // after IcebergSink.forRowData(input).distributionMode(DistributionMode.NONE)...
Defensive patterns
Strategy: validation
Validate before calling
if (mode == DistributionMode.HASH && equalityFieldIds.isEmpty() && table.spec().isUnpartitioned()) {
mode = DistributionMode.NONE; // avoid pointless hash request
} Prevention
- Match distribution mode to table shape (partitioned/PK vs append-only)
- Validate sink configuration in a pre-flight check before env.execute
- Avoid copying distribution settings between different tables
When it happens
Trigger: IcebergSink constructor calls distributeDataStreamByHashDistributionMode with empty equalityFieldIds and partitionSpec.isUnpartitioned() (e.g. WRITE_DISTRIBUTION_MODE=hash or DistributionMode.HASH requested).
Common situations: Append-only writes to an unpartitioned table with hash mode requested; users copying config from a partitioned table's job; defaulting to hash in streaming without a primary key.
Understand the failure class
Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.
Related errors
- Fallback to use 'none' distribution mode, because there are
- Class %s does not implement DynamicRecordGeneratorSQL
- Failed to instantiate DynamicRecordGeneratorSQL %s
- Fallback to use 'none' distribution mode, because there are
- Hash distribute rows by equality fields, even though {}=rang
AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12).
Data as JSON: /api/errors/5f7e0208f190cb39.
Report an issue: GitHub.