{"record":{"id":"eb9dcf7ffdb7bf29","repo":"apache/iceberg","slug":"optional-parameters-must-be-after-required-ones-bu","errorCode":null,"errorMessage":"Optional parameters must be after required ones but $currentParam is after $previousParam","messagePattern":"Optional parameters must be after required ones but \\$currentParam is after \\$previousParam","errorType":"exception","errorClass":"AnalysisException","httpStatus":null,"severity":"error","filePath":"spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveProcedures.scala","lineNumber":70,"sourceCode":"      val normalizedArgs = normalizeArgs(args)\n      Call(procedure, args = buildArgExprs(normalizedParams, normalizedArgs).toSeq)\n  }\n\n  private def validateParams(params: Seq[ProcedureParameter]): Unit = {\n    // should not be any duplicate param names\n    val duplicateParamNames = params.groupBy(_.name).collect {\n      case (name, matchingParams) if matchingParams.length > 1 => name\n    }\n\n    if (duplicateParamNames.nonEmpty) {\n      throw new AnalysisException(\n        s\"Duplicate parameter names: ${duplicateParamNames.mkString(\"[\", \",\", \"]\")}\")\n    }\n\n    // optional params should be at the end\n    params.sliding(2).foreach {\n      case Seq(previousParam, currentParam) if !previousParam.required && currentParam.required =>\n        throw new AnalysisException(\n          s\"Optional parameters must be after required ones but $currentParam is after $previousParam\")\n      case _ =>\n    }\n  }\n\n  private def buildArgExprs(\n      params: Seq[ProcedureParameter],\n      args: Seq[CallArgument]): Seq[Expression] = {\n\n    // build a map of declared parameter names to their positions\n    val nameToPositionMap = params.map(_.name).zipWithIndex.toMap\n\n    // build a map of parameter names to args\n    val nameToArgMap = buildNameToArgMap(params, args, nameToPositionMap)\n\n    // verify all required parameters are provided\n    val missingParamNames = params.filter(_.required).collect {\n      case param if !nameToArgMap.contains(param.name) => param.name","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v3.5/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveProcedures.scala#L52-L88","documentation":"Thrown by ResolveProcedures when a procedure's parameter list places a required parameter after an optional one. Iceberg requires all optional (defaulted) parameters to come after required ones so positional invocation remains unambiguous.","triggerScenarios":"Registering a procedure with params like [optional(\"a\"), required(\"b\")] and then resolving any CALL against it.","commonSituations":"Adding a new required parameter at the end of an existing signature that already had optional parameters; generating parameters from a map with non-deterministic ordering.","solutions":["Reorder the parameter list so all required parameters precede optional ones","If the new parameter is truly optional, declare it with ProcedureParameter.optional instead of required","Use an ordered structure (List/Seq in declared order) when building the parameter list"],"exampleFix":"// before\nSeq(ProcedureParameter.optional(\"where\", StringType), ProcedureParameter.required(\"table\", StringType))\n// after\nSeq(ProcedureParameter.required(\"table\", StringType), ProcedureParameter.optional(\"where\", StringType))","handlingStrategy":"validation","validationCode":"val idxOpt = params.zipWithIndex.find { case ((prev, cur)) => false }\n// simpler: verify no required param follows an optional one\nrequire(!params.sliding(2).exists { case Seq(p, c) => !p.required && c.required; case _ => false },\n  \"optional params must come last\")","typeGuard":null,"tryCatchPattern":"try { spark.sql(callSql) } catch { case e: AnalysisException if e.getMessage.contains(\"Optional parameters must be after required ones\") => // reorder params }","preventionTips":["Always append new optional parameters at the end of a procedure signature","Build parameter lists from an ordered Seq in declaration order","Add a signature-validation unit test for custom procedures"],"tags":["spark","sql","procedures","analysis"],"backgroundTag":"invalid-argument-value","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}