{"record":{"id":"6cf76f2a5e292175","repo":"apache/iceberg","slug":"unable-to-build-the-manifest-files-dataframe-the-6cf76f","errorCode":null,"errorMessage":"Unable to build the manifest files dataframe. The end version in use may contain invalid snapshots. Please choose an earlier version without invalid snapshots.","messagePattern":"Unable to build the manifest files dataframe\\. The end version in use may contain invalid snapshots\\. Please choose an earlier version without invalid snapshots\\.","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java","lineNumber":552,"sourceCode":"      Table endStaticTable = newStaticTable(endVersionName, table.io());\n      Dataset<Row> lastVersionFiles = manifestDS(endStaticTable).select(\"path\");\n      if (startMetadata == null) {\n        return Sets.newHashSet(lastVersionFiles.distinct().as(Encoders.STRING()).collectAsList());\n      } else {\n        Set<Long> deltaSnapshotIds =\n            deltaSnapshots.stream().map(Snapshot::snapshotId).collect(Collectors.toSet());\n        return Sets.newHashSet(\n            lastVersionFiles\n                .distinct()\n                .filter(\n                    functions\n                        .column(ManifestFile.SNAPSHOT_ID.name())\n                        .isInCollection(deltaSnapshotIds))\n                .as(Encoders.STRING())\n                .collectAsList());\n      }\n    } catch (Exception e) {\n      throw new UnsupportedOperationException(\n          \"Unable to build the manifest files dataframe. The end version in use may contain invalid snapshots. \"\n              + \"Please choose an earlier version without invalid snapshots.\",\n          e);\n    }\n  }\n\n  public static class RewriteContentFileResult extends RewriteResult<ContentFile<?>> {\n    @Override\n    public RewriteContentFileResult append(RewriteResult<ContentFile<?>> r1) {\n      this.copyPlan().addAll(r1.copyPlan());\n      this.toRewrite().addAll(r1.toRewrite());\n      r1.rewrittenManifestLengths().forEach(this::addRewrittenManifestLength);\n      return this;\n    }\n\n    public RewriteContentFileResult appendDataFile(RewriteResult<DataFile> r1) {\n      this.copyPlan().addAll(r1.copyPlan());\n      this.toRewrite().addAll(r1.toRewrite());","sourceCodeStart":534,"sourceCodeEnd":570,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteTablePathSparkAction.java#L534-L570","documentation":"RewriteTablePathSparkAction.manifestsToRewrite builds a Spark DataFrame of manifest files for the version range being copied. If building/collecting that dataframe fails for any reason (invalid snapshots, missing metadata files in the copy range), it is rethrown as UnsupportedOperationException telling the user to pick an earlier end version without invalid snapshots.","triggerScenarios":"Calling rewriteTablePath() (manifestsToRewrite stage) with an end version whose snapshot chain contains invalid/unreadable snapshots, causing the manifest-files dataframe construction or collectAsList to throw.","commonSituations":"Copying a table path up to a snapshot ID referencing expired or corrupted metadata; using a snapshot range that spans manually deleted metadata files; malformed ancestor chain after failed commits.","solutions":["Choose an earlier end snapshot/version that predates the invalid snapshots","Validate the snapshot chain (table.snapshots(), metadata JSON) and repair or roll back to a valid snapshot (rollbackTo)","Restore missing metadata files from backup or re-register the table at a valid metadata location","Inspect the underlying cause exception chained in the UnsupportedOperationException"],"exampleFix":"// before\nSparkActions.get(table).rewriteTablePath(source, target).rewriteManifests(); // end version has invalid snapshot\n// after: roll back to a valid snapshot first\ntable.manageSnapshots().rollbackTo(validSnapshotId);\nSparkActions.get(table).rewriteTablePath(source, target).execute();","handlingStrategy":"validation","validationCode":"// validate snapshot chain before rewriting table path\ntable.snapshots().forEach(s -> {\n  Preconditions.checkArgument(s.snapshotId() > 0, \"Invalid snapshot in chain\");\n  // ensure metadata files referenced exist in the copied range\n});","typeGuard":null,"tryCatchPattern":"try { action.rewriteManifests(); } catch (UnsupportedOperationException e) { /* pick earlier end version per message */ }","preventionTips":["Verify all snapshots in the chosen version range are valid and their metadata files exist","Avoid deleting/expiring metadata files before a path rewrite","Roll back to a known-good snapshot before copying the table path"],"tags":["spark","rewrite-table-path","snapshots","metadata"],"backgroundTag":"invalid-state-transition","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}