apache/iceberg · info

Fallback to use 'none' distribution mode, because there are

Error message

Fallback to use 'none' distribution mode, because there are no equality fields set and table is unpartitioned

What it means

In the new IcebergSink builder's hash-distribution path, when no equality fields are configured and the table is unpartitioned there is no key to hash on, so the sink logs this warning and falls back to 'none' distribution (returns the input unchanged).

Source

Thrown at flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java:1001

        throw new RuntimeException("Unrecognized " + WRITE_DISTRIBUTION_MODE + ": " + mode);
    }
  }

  private DataStream<RowData> distributeDataStreamByNoneDistributionMode(
      DataStream<RowData> input, Schema iSchema) {
    if (equalityFieldIds.isEmpty()) {
      return input;
    } else {
      LOG.info("Distribute rows by equality fields, because there are equality fields set");
      return input.keyBy(new EqualityFieldKeySelector(iSchema, flinkRowType, equalityFieldIds));
    }
  }

  private DataStream<RowData> distributeDataStreamByHashDistributionMode(
      DataStream<RowData> input, Schema iSchema, PartitionSpec partitionSpec) {
    if (equalityFieldIds.isEmpty()) {
      if (partitionSpec.isUnpartitioned()) {
        LOG.warn(
            "Fallback to use 'none' distribution mode, because there are no equality fields set "
                + "and table is unpartitioned");
        return input;
      } else {
        return input.keyBy(new PartitionKeySelector(partitionSpec, iSchema, flinkRowType));
      }
    } else {
      if (partitionSpec.isUnpartitioned()) {
        LOG.info(
            "Distribute rows by equality fields, because there are equality fields set "
                + "and table is unpartitioned");
        return input.keyBy(new EqualityFieldKeySelector(iSchema, flinkRowType, equalityFieldIds));
      } else {
        for (PartitionField partitionField : partitionSpec.fields()) {
          Preconditions.checkState(
              equalityFieldIds.contains(partitionField.sourceId()),
              "In 'hash' distribution mode with equality fields set, source column '%s' of partition field '%s' "
                  + "should be included in equality fields: '%s'",

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Use DistributionMode.NONE for unpartitioned append-only streams
  2. Add partitions to the table so HASH can key by partition
  3. Configure equality fields/primary key for upsert workloads
  4. Drop the explicit hash distribution setting

Example fix

// before
IcebergSink.forRowData(input).distributionMode(DistributionMode.HASH)...
// after
IcebergSink.forRowData(input).distributionMode(DistributionMode.NONE)...
Defensive patterns

Strategy: validation

Validate before calling

if (mode == DistributionMode.HASH && equalityFieldIds.isEmpty() && table.spec().isUnpartitioned()) {
  mode = DistributionMode.NONE; // avoid pointless hash request
}

Prevention

When it happens

Trigger: IcebergSink constructor calls distributeDataStreamByHashDistributionMode with empty equalityFieldIds and partitionSpec.isUnpartitioned() (e.g. WRITE_DISTRIBUTION_MODE=hash or DistributionMode.HASH requested).

Common situations: Append-only writes to an unpartitioned table with hash mode requested; users copying config from a partitioned table's job; defaulting to hash in streaming without a primary key.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors


AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/5f7e0208f190cb39. Report an issue: GitHub.