What causes "Could not find a partition transform for '{}'." in apache/beam?

Check Beam PartitionUtils source for the exact accepted regex patterns and match your strings, e.g. 'year(col)', 'month(col)', 'day(col)', 'hour(col)', 'bucket[N](col)', 'truncate[L](col)'

apache/beam · error · IllegalArgumentException

Could not find a partition transform for '{}'.

Error message

Could not find a partition transform for '{}'.

What it means

PartitionUtils.toPartitionSpec translates Beam's partitioning DSL strings (e.g. year(ts), bucket[16](id), truncate[10](name)) into an Iceberg PartitionSpec. Each field string must match one of the registered transform regexes; if no regex matches, IllegalArgumentException is thrown because the transform cannot be represented.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/PartitionUtils.java:107

  static PartitionSpec toPartitionSpec(@Nullable List<String> fields, Schema schema) {
    if (fields == null) {
      return PartitionSpec.unpartitioned();
    }
    PartitionSpec.Builder builder = PartitionSpec.builderFor(schema);

    for (String field : fields) {
      boolean matched = false;
      for (Map.Entry<Pattern, BiFunction<PartitionSpec.Builder, Matcher, PartitionSpec.Builder>>
          entry : TRANSFORMATIONS.entrySet()) {
        Matcher matcher = entry.getKey().matcher(field);
        if (matcher.find()) {
          builder = entry.getValue().apply(builder, matcher);
          matched = true;
          break;
        }
      }
      if (!matched) {
        throw new IllegalArgumentException(
            "Could not find a partition transform for '" + field + "'.");
      }
    }

    return builder.build();
  }

  private static final Map<Pattern, Function<Matcher, Term>> TERMS =
      ImmutableMap.of(
          HOUR,
          matcher -> Expressions.hour(checkStateNotNull(matcher.group(1))),
          DAY,
          matcher -> Expressions.day(checkStateNotNull(matcher.group(1))),
          MONTH,
          matcher -> Expressions.month(checkStateNotNull(matcher.group(1))),
          YEAR,
          matcher -> Expressions.year(checkStateNotNull(matcher.group(1))),
          TRUNCATE,

View on GitHub (pinned to 12126d8942)

Solutions

Check Beam PartitionUtils source for the exact accepted regex patterns and match your strings, e.g. 'year(col)', 'month(col)', 'day(col)', 'hour(col)', 'bucket[N](col)', 'truncate[L](col)'
Use identity by passing the bare column name if you just want untransformed partitioning
Fix bracket syntax: use square brackets for parameters, e.g. bucket[8](user_id) not bucket(8)(user_id)
Only use transforms that exist in the table's actual PartitionSpec; inspect table.spec() to copy field names

Example fix

// before
"bucket(16)(user_id)"
// after
"bucket[16](user_id)"

Defensive patterns

Strategy: validation

Validate before calling

java.util.regex.Pattern p = java.util.regex.Pattern.compile("(year|month|day|hour)\\((\\w+)\\)|(bucket)\\[(\\d+)\\]\\((\\w+)\\)|(truncate)\\[(\\d+)\\]\\((\\w+)\\)|^(\\w+)$");
if (!p.matcher(partitionField).matches()) throw new IllegalArgumentException("Invalid partition transform: " + partitionField);

Type guard

null

Try / catch

try {
  PartitionSpec spec = PartitionUtils.toPartitionSpec(schema, partitionFields);
} catch (IllegalArgumentException e) {
  LOG.error("Partition transform syntax rejected: {}", e.getMessage());
  // fix string and retry, or fall back to identity partitioning
}

Prevention

Use exactly the documented syntax: day(col), bucket[8](col), truncate[10](col)
Copy transform names from table.spec() rather than hand-writing
Add a startup validation that parses all partition strings before launching the pipeline

When it happens

Trigger: Calling Read.from(table).withPartitioning/toPartitionSpec with a partition field string not matching supported patterns — misspelled transform (years vs year), wrong bracket syntax (bucket(16) instead of bucket[16]), unsupported transform (identity on a type without support), or a transform name entirely absent from the matcher list.

Common situations: Copy-pasting Iceberg DDL transform syntax (e.g. 'month(ts)') that uses a different bracket convention than Beam's matcher expects; using transforms like 'void' or bucket sizes not covered; typos in column names inside the transform string.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/522c71166574b252. Report an issue: GitHub.