apache/iceberg · error

Unable to parse the predicate expression: %s

Error message

Unable to parse the predicate expression: %s

What it means

Thrown when Spark's SQL parser cannot parse the predicate string into an Expression in getPartitionsByFilter(spark, table, predicate). SparkTableUtil wraps the ParseException as an unchecked exception including the raw predicate.

Source

Thrown at spark/v3.5/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java:244

   * @param table a table name and (optional) database
   * @param predicate a predicate on partition columns
   * @return matching table's partitions
   */
  public static List<SparkPartition> getPartitionsByFilter(
      SparkSession spark, String table, String predicate) {
    TableIdentifier tableIdent;
    try {
      tableIdent = spark.sessionState().sqlParser().parseTableIdentifier(table);
    } catch (ParseException e) {
      throw SparkExceptionUtil.toUncheckedException(
          e, "Unable to parse the table identifier: %s", table);
    }

    Expression unresolvedPredicateExpr;
    try {
      unresolvedPredicateExpr = spark.sessionState().sqlParser().parseExpression(predicate);
    } catch (ParseException e) {
      throw SparkExceptionUtil.toUncheckedException(
          e, "Unable to parse the predicate expression: %s", predicate);
    }

    Expression resolvedPredicateExpr = resolveAttrs(spark, table, unresolvedPredicateExpr);
    return getPartitionsByFilter(spark, tableIdent, resolvedPredicateExpr);
  }

  /**
   * Returns partitions that match the specified 'predicate'.
   *
   * @param spark a Spark session
   * @param tableIdent a table identifier
   * @param predicateExpr a predicate expression on partition columns
   * @return matching table's partitions
   */
  public static List<SparkPartition> getPartitionsByFilter(
      SparkSession spark, TableIdentifier tableIdent, Expression predicateExpr) {
    try {

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Validate the predicate parses first: spark.sessionState().sqlParser().parseExpression(predicate).toString().
2. Quote column names with backticks and use single quotes for string literals.
3. Use Spark-legal syntax (e.g. date '2024-01-01' instead of TO_DATE(...) variants the parser rejects).
4. Build the expression programmatically via functions.expr or Column APIs instead of raw strings when predicates are dynamic.

Example fix

// before
SparkTableUtil.getPartitionsByFilter(spark, "db.events", "date = 2024-01-01");
// after
SparkTableUtil.getPartitionsByFilter(spark, "db.events", "date = date '2024-01-01'");

try {
  spark.sessionState().sqlParser().parseExpression(predicate);
} catch (ParseException e) {
  throw new IllegalArgumentException("Invalid predicate: " + predicate, e);
}

try {
  SparkTableUtil.getPartitionsByFilter(spark, table, predicate);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Unable to parse the predicate")) {
    // log offending predicate and skip filtering or fail fast
  } else {
    throw e;
  }
}

When it happens

Trigger: Calling SparkTableUtil.getPartitionsByFilter with a predicate string that is not a valid Spark SQL boolean expression — malformed syntax, unsupported functions, unquoted string literals, or an empty string.

Common situations: Building predicates with string concatenation that produces invalid SQL; using engine-specific SQL that Spark's parser rejects; passing column names with special characters unquoted; locale/date formatting issues in literal values.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Unable to parse sortOrder: %s
• e.message (wrapped as IcebergParseException with command con
• msg (dynamic parse error message)
• Invalid transform argument
• Unable to parse the table identifier: %s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/de1a565a87033ffb. Report an issue: GitHub.