apache/iceberg · error · AnalysisException

Optional parameters must be after required ones but…

Error message

Optional parameters must be after required ones but $currentParam is after $previousParam

What it means

Thrown by ResolveProcedures when a procedure's parameter list places a required parameter after an optional one. Iceberg requires all optional (defaulted) parameters to come after required ones so positional invocation remains unambiguous.

Solutions

1. Reorder the parameter list so all required parameters precede optional ones
2. If the new parameter is truly optional, declare it with ProcedureParameter.optional instead of required
3. Use an ordered structure (List/Seq in declared order) when building the parameter list

Example fix

// before
Seq(ProcedureParameter.optional("where", StringType), ProcedureParameter.required("table", StringType))
// after
Seq(ProcedureParameter.required("table", StringType), ProcedureParameter.optional("where", StringType))

val idxOpt = params.zipWithIndex.find { case ((prev, cur)) => false }
// simpler: verify no required param follows an optional one
require(!params.sliding(2).exists { case Seq(p, c) => !p.required && c.required; case _ => false },
  "optional params must come last")

try { spark.sql(callSql) } catch { case e: AnalysisException if e.getMessage.contains("Optional parameters must be after required ones") => // reorder params }

When it happens

Trigger: Registering a procedure with params like [optional("a"), required("b")] and then resolving any CALL against it.

Common situations: Adding a new required parameter at the end of an existing signature that already had optional parameters; generating parameters from a map with non-deterministic ordering.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Duplicate parameter names
• CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS
• ALTER TABLE contains multiple distribution clauses
• ALTER TABLE contains multiple distribution clauses
• ALTER TABLE contains multiple ordering clauses

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/eb9dcf7ffdb7bf29. Report an issue: GitHub.

Appendix: source

Thrown at spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveProcedures.scala:70

      val normalizedArgs = normalizeArgs(args)
      Call(procedure, args = buildArgExprs(normalizedParams, normalizedArgs).toSeq)
  }

  private def validateParams(params: Seq[ProcedureParameter]): Unit = {
    // should not be any duplicate param names
    val duplicateParamNames = params.groupBy(_.name).collect {
      case (name, matchingParams) if matchingParams.length > 1 => name
    }

    if (duplicateParamNames.nonEmpty) {
      throw new AnalysisException(
        s"Duplicate parameter names: ${duplicateParamNames.mkString("[", ",", "]")}")
    }

    // optional params should be at the end
    params.sliding(2).foreach {
      case Seq(previousParam, currentParam) if !previousParam.required && currentParam.required =>
        throw new AnalysisException(
          s"Optional parameters must be after required ones but $currentParam is after $previousParam")
      case _ =>
    }
  }

  private def buildArgExprs(
      params: Seq[ProcedureParameter],
      args: Seq[CallArgument]): Seq[Expression] = {

    // build a map of declared parameter names to their positions
    val nameToPositionMap = params.map(_.name).zipWithIndex.toMap

    // build a map of parameter names to args
    val nameToArgMap = buildNameToArgMap(params, args, nameToPositionMap)

    // verify all required parameters are provided
    val missingParamNames = params.filter(_.required).collect {
      case param if !nameToArgMap.contains(param.name) => param.name

View on GitHub (pinned to 86d9c8fc54)