What causes "Could not find a partition transform for '{}'." in apache/beam?

Check Beam PartitionUtils source for the exact accepted regex patterns and match your strings, e.g. 'year(col)', 'month(col)', 'day(col)', 'hour(col)', 'bucket[N](col)', 'truncate[L](col)'

apache/beam · error · IllegalArgumentException

Could not find a partition transform for

Error message

Could not find a partition transform for '{}'.

What it means

PartitionUtils.toPartitionSpec translates Beam's partitioning DSL strings (e.g. year(ts), bucket[16](id), truncate[10](name)) into an Iceberg PartitionSpec. Each field string must match one of the registered transform regexes; if no regex matches, IllegalArgumentException is thrown because the transform cannot be represented.

Solutions

Check Beam PartitionUtils source for the exact accepted regex patterns and match your strings, e.g. 'year(col)', 'month(col)', 'day(col)', 'hour(col)', 'bucket[N](col)', 'truncate[L](col)'
Use identity by passing the bare column name if you just want untransformed partitioning
Fix bracket syntax: use square brackets for parameters, e.g. bucket[8](user_id) not bucket(8)(user_id)
Only use transforms that exist in the table's actual PartitionSpec; inspect table.spec() to copy field names

Example fix

// before
"bucket(16)(user_id)"
// after
"bucket[16](user_id)"

Defensive patterns

Strategy: validation

Validate before calling

java.util.regex.Pattern p = java.util.regex.Pattern.compile("(year|month|day|hour)\\((\\w+)\\)|(bucket)\\[(\\d+)\\]\\((\\w+)\\)|(truncate)\\[(\\d+)\\]\\((\\w+)\\)|^(\\w+)$");
if (!p.matcher(partitionField).matches()) throw new IllegalArgumentException("Invalid partition transform: " + partitionField);

Type guard

null

Try / catch

try {
  PartitionSpec spec = PartitionUtils.toPartitionSpec(schema, partitionFields);
} catch (IllegalArgumentException e) {
  LOG.error("Partition transform syntax rejected: {}", e.getMessage());
  // fix string and retry, or fall back to identity partitioning
}

Prevention

Use exactly the documented syntax: day(col), bucket[8](col), truncate[10](col)
Copy transform names from table.spec() rather than hand-writing
Add a startup validation that parses all partition strings before launching the pipeline

When it happens

Trigger: Calling Read.from(table).withPartitioning/toPartitionSpec with a partition field string not matching supported patterns — misspelled transform (years vs year), wrong bracket syntax (bucket(16) instead of bucket[16]), unsupported transform (identity on a type without support), or a transform name entirely absent from the matcher list.

Common situations: Copy-pasting Iceberg DDL transform syntax (e.g. 'month(ts)') that uses a different bracket convention than Beam's matcher expects; using transforms like 'void' or bucket sizes not covered; typos in column names inside the transform string.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/522c71166574b252. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/PartitionUtils.java:107

  static PartitionSpec toPartitionSpec(@Nullable List<String> fields, Schema schema) {
    if (fields == null) {
      return PartitionSpec.unpartitioned();
    }
    PartitionSpec.Builder builder = PartitionSpec.builderFor(schema);

    for (String field : fields) {
      boolean matched = false;
      for (Map.Entry<Pattern, BiFunction<PartitionSpec.Builder, Matcher, PartitionSpec.Builder>>
          entry : TRANSFORMATIONS.entrySet()) {
        Matcher matcher = entry.getKey().matcher(field);
        if (matcher.find()) {
          builder = entry.getValue().apply(builder, matcher);
          matched = true;
          break;
        }
      }
      if (!matched) {
        throw new IllegalArgumentException(
            "Could not find a partition transform for '" + field + "'.");
      }
    }

    return builder.build();
  }

  private static final Map<Pattern, Function<Matcher, Term>> TERMS =
      ImmutableMap.of(
          HOUR,
          matcher -> Expressions.hour(checkStateNotNull(matcher.group(1))),
          DAY,
          matcher -> Expressions.day(checkStateNotNull(matcher.group(1))),
          MONTH,
          matcher -> Expressions.month(checkStateNotNull(matcher.group(1))),
          YEAR,
          matcher -> Expressions.year(checkStateNotNull(matcher.group(1))),
          TRUNCATE,

View on GitHub (pinned to 12126d8942)