apache/beam · error · RuntimeException

Encountered an error when parsing filter: '{filter}'

Error message

Encountered an error when parsing filter: '{filter}'

What it means

FilterUtils.getReferencedFieldNames parses a SQL filter expression with Calcite's SqlParser to collect the field names it references. Any parse failure is wrapped in a RuntimeException with the offending filter text, chaining the original exception as the cause.

Source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/FilterUtils.java:106

  public static final Set<SqlKind> SUPPORTED_OPS = FILTERS.keySet();

  /**
   * Parses a SQL filter expression string and returns a set of all field names referenced within
   * it.
   */
  static Set<String> getReferencedFieldNames(@Nullable String filter) {
    if (filter == null || filter.trim().isEmpty()) {
      return new HashSet<>();
    }

    SqlParser parser = SqlParser.create(filter);
    try {
      SqlNode expression = parser.parseExpression();
      Set<String> fieldNames = new HashSet<>();
      extractFieldNames(expression, fieldNames);
      return fieldNames;
    } catch (Exception exception) {
      throw new RuntimeException(
          String.format("Encountered an error when parsing filter: '%s'", filter), exception);
    }
  }

  private static void extractFieldNames(SqlNode node, Set<String> fieldNames) {
    if (node instanceof SqlIdentifier) {
      fieldNames.add(getFieldName((SqlIdentifier) node));
    } else if (node instanceof SqlBasicCall) {
      // recursively check operands
      SqlBasicCall call = (SqlBasicCall) node;
      for (SqlNode operand : call.getOperandList()) {
        extractFieldNames(operand, fieldNames);
      }
    } else if (node instanceof SqlNodeList) {
      // For IN clauses, the right-hand side is a SqlNodeList, so iterate through its elements
      SqlNodeList nodeList = (SqlNodeList) node;
      for (SqlNode element : nodeList.getList()) {
        if (element != null) {

View on GitHub (pinned to 12126d8942)

Solutions

Inspect the chained cause (exception.getCause()) to see the exact SqlParseException and fix the filter string accordingly.
Rewrite the filter to a Calcite-parseable SQL expression (balanced parens, quoted literals).
Prefer passing Expression objects or validated filter strings into the pipeline rather than hand-written SQL strings.

Example fix

// before
getReferencedFieldNames("ts >= 2024-01-01 AND id IN (1,2", schema);
// after
getReferencedFieldNames("ts >= '2024-01-01' AND id IN (1, 2)", schema);

Defensive patterns

Strategy: try-catch

Validate before calling

try { SqlParser.create(filter).parseExpression(); } catch (Exception e) { throw new IllegalArgumentException("Invalid filter: " + filter, e); }

Try / catch

try {
  Set<String> fields = FilterUtils.getReferencedFieldNames(filter, schema);
} catch (RuntimeException e) {
  LOG.error("Cannot parse filter '{}': {}", filter, e.getCause());
  // fall back to unfiltered read or abort pipeline
}

Prevention

Validate filter strings with a test parse before submitting pipelines.
Use single quotes for SQL string literals and balance parentheses.
Store filters as Iceberg Expressions instead of raw strings when possible.

When it happens

Trigger: Calling FilterUtils.getReferencedFieldNames(filter, ...) with a string that is not a valid SQL expression — e.g., a filter in Iceberg's string syntax that Calcite cannot parse, or a truncated/malformed predicate string.

Common situations: Filters built for a different SQL dialect; string filters pasted from Iceberg/Spark syntax that differ from Calcite's grammar; typos like missing quotes around literals or unbalanced parentheses.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/77c899997af21829. Report an issue: GitHub.