apache/iceberg · error

Unable to get partition spec for table

Error message

Unable to get partition spec for table: %s

What it means

Thrown by SparkTableUtil.importSparkTable when resolving the source table's partition spec fails with a Spark AnalysisException — typically because the source table cannot be analyzed (does not exist, unresolved, or has a schema/catalog problem). The exception is rewrapped with this message naming the source table. It means the import could not even inspect the source table's structure.

Solutions

1. Run the source table query directly (SELECT * FROM src LIMIT 1 or DESCRIBE TABLE) to see the underlying AnalysisException and fix its root cause.
2. Confirm the source is a real catalog table (not a view/temp view) and that it exists in the intended catalog and database.
3. Refresh the catalog cache (REFRESH TABLE / spark.catalog.refreshTable) or restart the session if the metastore state changed underneath Spark.
4. If migrating, ensure the source format support (e.g. Hive, Parquet, ORC) is available in the Spark session.

Example fix

// before
SparkTableUtil.importSparkTable(spark, TableIdentifier.apply("events", "mydb"), icebergTable);
// after: verify analyzability first
spark.sql("DESCRIBE TABLE mydb.events"); // fails with the real root cause if broken
SparkTableUtil.importSparkTable(spark, TableIdentifier.apply("events", "mydb"), icebergTable);

try {
  spark.sql("DESCRIBE TABLE " + sourceTableIdentWithDB);
} catch (Exception e) {
  throw new IllegalArgumentException("Source not analyzable: " + sourceTableIdentWithDB, e);
}
SparkTableUtil.importSparkTable(spark, ident, targetTable);

try {
  SparkTableUtil.importSparkTable(spark, ident, target);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Unable to get partition spec for table")) {
    // inspect cause AnalysisException; verify source is a real analyzable table
  } else throw e;
}

When it happens

Trigger: Calling SparkTableUtil.importSparkTable(spark, sourceTableIdent, targetTable, ...) where spark.sessionState.catalog / table lookup raises AnalysisException: source view unresolvable, source table missing from metastore, corrupted metastore entries, or unsupported source formats failing analysis.

Common situations: Importing from a temp view or a streaming source that cannot be analyzed; Hive metastore out of sync (table in catalog cache but deleted on disk); source is a Spark SQL view referencing dropped tables; Spark version drift where the source format plugin is missing.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• Cannot find source table
• Cannot rename as for backup. The backup table already…
• Cannot use non-v1 table
• CREATE_VIEW_COLUMN_ARITY_MISMATCH.TOO_MANY_DATA_COLUMNS
• Duplicate parameter names

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f5d04d075c6bcf26. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/SparkTableUtil.java:408

      } else {
        List<SparkPartition> sourceTablePartitions =
            getPartitions(spark, sourceTableIdent, partitionFilter);
        if (sourceTablePartitions.isEmpty()) {
          targetTable.newAppend().commit();
        } else {
          importSparkPartitions(
              spark,
              sourceTablePartitions,
              targetTable,
              spec,
              stagingDir,
              checkDuplicateFiles,
              ignoreMissingFiles,
              service);
        }
      }
    } catch (AnalysisException e) {
      throw SparkExceptionUtil.toUncheckedException(
          e, "Unable to get partition spec for table: %s", sourceTableIdentWithDB);
    }
  }

  /**
   * Import files from an existing Spark table to an Iceberg table.
   *
   * <p>The import uses the Spark session to get table metadata. It assumes no operation is going on
   * the original and target table and thus is not thread-safe.
   *
   * @param spark a Spark session
   * @param sourceTableIdent an identifier of the source Spark table
   * @param targetTable an Iceberg table where to import the data
   * @param stagingDir a staging directory to store temporary manifest files
   */
  public static void importSparkTable(
      SparkSession spark, TableIdentifier sourceTableIdent, Table targetTable, String stagingDir) {
    importSparkTable(

View on GitHub (pinned to 86d9c8fc54)