apache/beam · error · RuntimeException

Encountered an error when parsing filter

Error message

Encountered an error when parsing filter: '{filter}'

What it means

FilterUtils.getReferencedFieldNames parses a SQL filter expression with Calcite's SqlParser to collect the field names it references. Any parse failure is wrapped in a RuntimeException with the offending filter text, chaining the original exception as the cause.

Solutions

Inspect the chained cause (exception.getCause()) to see the exact SqlParseException and fix the filter string accordingly.
Rewrite the filter to a Calcite-parseable SQL expression (balanced parens, quoted literals).
Prefer passing Expression objects or validated filter strings into the pipeline rather than hand-written SQL strings.

Example fix

// before
getReferencedFieldNames("ts >= 2024-01-01 AND id IN (1,2", schema);
// after
getReferencedFieldNames("ts >= '2024-01-01' AND id IN (1, 2)", schema);

Defensive patterns

Strategy: try-catch

Validate before calling

try { SqlParser.create(filter).parseExpression(); } catch (Exception e) { throw new IllegalArgumentException("Invalid filter: " + filter, e); }

Try / catch

try {
  Set<String> fields = FilterUtils.getReferencedFieldNames(filter, schema);
} catch (RuntimeException e) {
  LOG.error("Cannot parse filter '{}': {}", filter, e.getCause());
  // fall back to unfiltered read or abort pipeline
}

Prevention

Validate filter strings with a test parse before submitting pipelines.
Use single quotes for SQL string literals and balance parentheses.
Store filters as Iceberg Expressions instead of raw strings when possible.

When it happens

Trigger: Calling FilterUtils.getReferencedFieldNames(filter, ...) with a string that is not a valid SQL expression — e.g., a filter in Iceberg's string syntax that Calcite cannot parse, or a truncated/malformed predicate string.

Common situations: Filters built for a different SQL dialect; string filters pasted from Iceberg/Spark syntax that differ from Calcite's grammar; typos like missing quotes around literals or unbalanced parentheses.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/77c899997af21829. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/FilterUtils.java:106

  public static final Set<SqlKind> SUPPORTED_OPS = FILTERS.keySet();

  /**
   * Parses a SQL filter expression string and returns a set of all field names referenced within
   * it.
   */
  static Set<String> getReferencedFieldNames(@Nullable String filter) {
    if (filter == null || filter.trim().isEmpty()) {
      return new HashSet<>();
    }

    SqlParser parser = SqlParser.create(filter);
    try {
      SqlNode expression = parser.parseExpression();
      Set<String> fieldNames = new HashSet<>();
      extractFieldNames(expression, fieldNames);
      return fieldNames;
    } catch (Exception exception) {
      throw new RuntimeException(
          String.format("Encountered an error when parsing filter: '%s'", filter), exception);
    }
  }

  private static void extractFieldNames(SqlNode node, Set<String> fieldNames) {
    if (node instanceof SqlIdentifier) {
      fieldNames.add(getFieldName((SqlIdentifier) node));
    } else if (node instanceof SqlBasicCall) {
      // recursively check operands
      SqlBasicCall call = (SqlBasicCall) node;
      for (SqlNode operand : call.getOperandList()) {
        extractFieldNames(operand, fieldNames);
      }
    } else if (node instanceof SqlNodeList) {
      // For IN clauses, the right-hand side is a SqlNodeList, so iterate through its elements
      SqlNodeList nodeList = (SqlNodeList) node;
      for (SqlNode element : nodeList.getList()) {
        if (element != null) {

View on GitHub (pinned to 12126d8942)