{"record":{"id":"f35f784038096b7e","repo":"apache/iceberg","slug":"unable-to-build-the-manifest-files-dataframe-the-f35f78","errorCode":null,"errorMessage":"Unable to build the manifest files dataframe. The end version in use may contain invalid snapshots. Please choose an earlier version without invalid snapshots.","messagePattern":"Unable to build the manifest files dataframe\\. The end version in use may contain invalid snapshots\\. Please choose an earlier version without invalid snapshots\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java","lineNumber":552,"sourceCode":"      Table endStaticTable = newStaticTable(endVersionName, table.io());\n      Dataset<Row> lastVersionFiles = manifestDS(endStaticTable).select(\"path\");\n      if (startMetadata == null) {\n        return Sets.newHashSet(lastVersionFiles.distinct().as(Encoders.STRING()).collectAsList());\n      } else {\n        Set<Long> deltaSnapshotIds =\n            deltaSnapshots.stream().map(Snapshot::snapshotId).collect(Collectors.toSet());\n        return Sets.newHashSet(\n            lastVersionFiles\n                .distinct()\n                .filter(\n                    functions\n                        .column(ManifestFile.SNAPSHOT_ID.name())\n                        .isInCollection(deltaSnapshotIds))\n                .as(Encoders.STRING())\n                .collectAsList());\n      }\n    } catch (Exception e) {\n      throw new UnsupportedOperationException(\n          \"Unable to build the manifest files dataframe. The end version in use may contain invalid snapshots. \"\n              + \"Please choose an earlier version without invalid snapshots.\",\n          e);\n    }\n  }\n\n  public static class RewriteContentFileResult extends RewriteResult<ContentFile<?>> {\n    @Override\n    public RewriteContentFileResult append(RewriteResult<ContentFile<?>> r1) {\n      this.copyPlan().addAll(r1.copyPlan());\n      this.toRewrite().addAll(r1.toRewrite());\n      r1.rewrittenManifestLengths().forEach(this::addRewrittenManifestLength);\n      return this;\n    }\n\n    public RewriteContentFileResult appendDataFile(RewriteResult<DataFile> r1) {\n      this.copyPlan().addAll(r1.copyPlan());\n      this.toRewrite().addAll(r1.toRewrite());","sourceCodeStart":534,"sourceCodeEnd":570,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java#L534-L570","documentation":"RewriteTablePathSparkAction.manifestsToRewrite builds a Spark DataFrame of manifest files to copy and wraps any failure (Spark query, snapshot lookup, etc.) in UnsupportedOperationException, advising the user to pick an earlier end version that contains no invalid snapshots.","triggerScenarios":"Calling manifestsToRewrite() with a rewriteTablePath end version whose snapshot chain contains invalid snapshots (e.g. dangling snapshot references, corrupted metadata) causing the manifest-collection query to throw.","commonSituations":"Migrating tables with interrupted expireSnapshots or partially-committed rewrites; end-version set to a snapshot with invalid ancestors; metadata corruption from failed operations.","solutions":["Choose an earlier end snapshot/version without invalid snapshots (e.g. via table.history() or rollBackTo).","Inspect snapshots with `spark.table(...).snapshots()` and remove/repair invalid snapshot references (e.g. removeSnapshots).","Read the wrapped cause (getCause()) to identify the concrete failing query and fix that underlying issue."],"exampleFix":"// before\nactions.rewriteTablePath(table).startVersion(\"v1\").endVersion(\"v9\").execute();\n// after\n// v9 contained invalid snapshots; use a known-good end version\nactions.rewriteTablePath(table).startVersion(\"v1\").endVersion(\"v7\").execute();","handlingStrategy":"validation","validationCode":"table.snapshots().forEach(s -> { /* validate each snapshot in [start, end] resolves */ });","typeGuard":null,"tryCatchPattern":"try { action.execute(); } catch (UnsupportedOperationException e) { Throwable cause = e.getCause(); /* inspect cause, choose earlier end version */ }","preventionTips":["Choose end versions from healthy snapshot chains (verify with snapshots() metadata table)","Avoid targeting versions after interrupted expire/rewrite operations","Read the cause chain to diagnose the actual failing query"],"tags":["spark","rewrite-table-path","invalid-snapshots"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}