apache/iceberg · warning
Fallback to use 'none' distribution mode, because there are…
Error message
Fallback to use 'none' distribution mode, because there are no equality fields set and table is unpartitioned
What it means
In the newer IcebergSink builder, requesting hash distribution mode with no equality fields on an unpartitioned table gives nothing to hash by, so the sink falls back to no shuffle ('none' mode) and logs this warning. Write parallelism will not be clustered by any key.
Solutions
- Set distribution mode to 'none' (or drop the hash setting) for this table.
- Add partitions to the table spec so hash keying by partition applies.
- Set .equalityFieldColumns(...) on the IcebergSink builder if a logical key exists.
Example fix
// before IcebergSink.builder().distributionMode(DistributionMode.HASH)... // unpartitioned, no keys // after IcebergSink.builder().distributionMode(DistributionMode.NONE)...
Defensive patterns
Strategy: validation
Validate before calling
if (mode == DistributionMode.HASH && equalityFieldIds.isEmpty() && table.spec().isUnpartitioned()) {
mode = DistributionMode.NONE;
} Prevention
- Validate distribution mode against table spec and equality fields before building the sink
- Avoid blanket write.distribution-mode=hash across heterogeneous tables
- Prefer NONE for unpartitioned key-less tables
When it happens
Trigger: IcebergSink.builder() with DistributionMode.HASH (or table property write.distribution-mode=hash), equalityFieldIds empty and partitionSpec unpartitioned, evaluated in distributeDataStreamByHashDistributionMode.
Common situations: Setting distribution-mode=hash on an unpartitioned key-less table; template configs applied across many tables regardless of partitioning; expecting hash mode to mitigate small-file issues without keys.
Understand the failure class
Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.
Related errors
- Fallback to use 'none' distribution mode, because there are…
- : Fallback to use 'none' distribution mode, because there…
- : Fallback to use 'none' distribution mode, because there…
- Hash distribute rows by equality fields, even though
- Hash distribute rows by equality fields, even though
AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12).
Data as JSON: /api/errors/7f97eb92cb1263cb.
Report an issue: GitHub.
Appendix: source
Thrown at flink/v2.2/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java:1001
throw new RuntimeException("Unrecognized " + WRITE_DISTRIBUTION_MODE + ": " + mode);
}
}
private DataStream<RowData> distributeDataStreamByNoneDistributionMode(
DataStream<RowData> input, Schema iSchema) {
if (equalityFieldIds.isEmpty()) {
return input;
} else {
LOG.info("Distribute rows by equality fields, because there are equality fields set");
return input.keyBy(new EqualityFieldKeySelector(iSchema, flinkRowType, equalityFieldIds));
}
}
private DataStream<RowData> distributeDataStreamByHashDistributionMode(
DataStream<RowData> input, Schema iSchema, PartitionSpec partitionSpec) {
if (equalityFieldIds.isEmpty()) {
if (partitionSpec.isUnpartitioned()) {
LOG.warn(
"Fallback to use 'none' distribution mode, because there are no equality fields set "
+ "and table is unpartitioned");
return input;
} else {
return input.keyBy(new PartitionKeySelector(partitionSpec, iSchema, flinkRowType));
}
} else {
if (partitionSpec.isUnpartitioned()) {
LOG.info(
"Distribute rows by equality fields, because there are equality fields set "
+ "and table is unpartitioned");
return input.keyBy(new EqualityFieldKeySelector(iSchema, flinkRowType, equalityFieldIds));
} else {
for (PartitionField partitionField : partitionSpec.fields()) {
Preconditions.checkState(
equalityFieldIds.contains(partitionField.sourceId()),
"In 'hash' distribution mode with equality fields set, source column '%s' of partition field '%s' "
+ "should be included in equality fields: '%s'",View on GitHub (pinned to 86d9c8fc54)