apache/iceberg · error · AnalysisException

CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS

CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS

Error message

CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS

What it means

When creating an Iceberg view, the declared column list has FEWER names than the view's query produces. CheckViews.verifyColumnCount throws Spark's standard CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS AnalysisException with viewName/viewColumns/dataColumns parameters.

Source

Thrown at spark/v4.1/spark-extensions/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckViews.scala:84

      case _ => // OK
    }
  }

  private def verifyColumnCount(
      ident: ResolvedIdentifier,
      columns: Seq[String],
      query: LogicalPlan): Unit = {
    if (columns.nonEmpty) {
      if (columns.length > query.output.length) {
        throw new AnalysisException(
          errorClass = "CREATE_VIEW_COLUMN_ARITY_MISMATCH.NOT_ENOUGH_DATA_COLUMNS",
          messageParameters = Map(
            "viewName" -> String.format("%s.%s", ident.catalog.name(), ident.identifier),
            "viewColumns" -> columns.mkString(", "),
            "dataColumns" -> query.output.map(c => c.name).mkString(", ")))
      } else if (columns.length < query.output.length) {
        throw new AnalysisException(
          errorClass = "CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS",
          messageParameters = Map(
            "viewName" -> String.format("%s.%s", ident.catalog.name(), ident.identifier),
            "viewColumns" -> columns.mkString(", "),
            "dataColumns" -> query.output.map(c => c.name).mkString(", ")))
      }
    }
  }

  private def checkCyclicViewReference(
      viewIdent: Seq[String],
      plan: LogicalPlan,
      cyclePath: Seq[Seq[String]]): Unit = {
    plan match {
      case sub @ SubqueryAlias(_, Project(_, _)) =>
        val currentViewIdent: Seq[String] = sub.identifier.qualifier :+ sub.identifier.name
        checkIfRecursiveView(viewIdent, currentViewIdent, cyclePath, sub.children)
      case v1View: View =>

View on GitHub (pinned to 86d9c8fc54)

Solutions

Add column names to the declared list so it matches the query output count
Remove columns from the SELECT to match the declared list
Drop the explicit column list and alias columns directly in the SELECT

Example fix

// before
CREATE VIEW v (a) AS SELECT id, name FROM t
// after
CREATE VIEW v (a, b) AS SELECT id, name FROM t

Defensive patterns

Strategy: validation

Validate before calling

val q = spark.sql(viewQuery); require(declaredCols.length == q.schema.length, s"view declares ${declaredCols.length} columns but query outputs ${q.schema.length}")

Try / catch

try { spark.sql(createViewDdl) } catch { case e: AnalysisException if e.getErrorClass.contains("CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS") => /* extend the column list */ }

Prevention

Regenerate column lists whenever the query changes
Drop explicit column lists when query aliases already name columns
Validate DDL in CI with a dry parse/schema check

When it happens

Trigger: `CREATE VIEW v (a) AS SELECT x, y, z FROM ...` — explicit column list length is less than the number of columns output by the query, detected on the Iceberg v2 view path.

Common situations: CREATE VIEW column lists not updated after the SELECT gains columns; copy-paste of view DDL between tables with different schemas.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f5997223f921b253. Report an issue: GitHub.