{"record":{"id":"0dd15722f60fac09","repo":"apache/iceberg","slug":"ignoring-provided-staging-location-as-new-manifest-0dd157","errorCode":null,"errorMessage":"Ignoring provided staging location as new manifests will be committed directly","messagePattern":"Ignoring provided staging location as new manifests will be committed directly","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteManifestsSparkAction.java","lineNumber":168,"sourceCode":"  @Override\n  public RewriteManifestsSparkAction specId(int specId) {\n    Preconditions.checkArgument(table.specs().containsKey(specId), \"Invalid spec id %s\", specId);\n    this.spec = table.specs().get(specId);\n    return this;\n  }\n\n  @Override\n  public RewriteManifestsSparkAction rewriteIf(Predicate<ManifestFile> newPredicate) {\n    this.predicate = newPredicate;\n    return this;\n  }\n\n  @Override\n  public RewriteManifestsSparkAction stagingLocation(String newStagingLocation) {\n    if (shouldStageManifests) {\n      this.outputLocation = newStagingLocation;\n    } else {\n      LOG.warn(\"Ignoring provided staging location as new manifests will be committed directly\");\n    }\n    return this;\n  }\n\n  @Override\n  public RewriteManifests.Result execute() {\n    String desc = String.format(\"Rewriting manifests in %s\", table.name());\n    JobGroupInfo info = newJobGroupInfo(\"REWRITE-MANIFESTS\", desc);\n    return withJobGroupInfo(info, this::doExecute);\n  }\n\n  @Override\n  public RewriteManifestsSparkAction sortBy(List<String> partitionFields) {\n    // Collect set of available partition columns to cluster on\n    Set<String> availablePartitionNames =\n        spec.fields().stream().map(PartitionField::name).collect(Collectors.toSet());\n\n    // Identify specified partition fields that are not available in the spec","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteManifestsSparkAction.java#L150-L186","documentation":"This is a LOG.warn, not an exception. RewriteManifestsSparkAction only uses a custom staging location when manifests actually need to be staged (rewritten) before commit. When the plan determines new manifests can be committed directly (e.g. nothing to rewrite / writing directly to the table location), a user-provided stagingLocation cannot be applied and is silently ignored with this warning.","triggerScenarios":"Calling rewriteManifests(table).stagingLocation(\"s3://bucket/tmp/\").execute() when the action's shouldStageManifests flag is false, i.e. the rewrite plan commits new manifests directly without a staging phase.","commonSituations":"Operators configuring a staging dir for S3 write-committed correctness or cost reasons, unaware the action decided direct commit is sufficient; copying configuration from an older Iceberg version where staging was always used.","solutions":["Verify whether the action is staging manifests at all; the warning is benign if direct commit is expected — remove the stagingLocation call if it has no effect.","Check the rewrite plan (shouldStageManifests derivation) to understand why staging was skipped; adjust rewrite options so staging actually occurs if you require it.","If a specific staging location is mandatory for your storage, force staging behavior via the relevant rewrite.* table/action options rather than relying on stagingLocation alone."],"exampleFix":"// before\nactions.rewriteManifests(table).stagingLocation(\"s3://bucket/staging\").execute();\n// after\nRewriteManifestsSparkAction action = actions.rewriteManifests(table);\nif (action.willStageManifests()) { // check/ensure staging is planned\n  action.stagingLocation(\"s3://bucket/staging\");\n}\naction.execute();","handlingStrategy":"validation","validationCode":"// Verify the rewrite plan stages manifests before supplying a staging location\nif (!actionStagesManifests(table, options)) {\n  // skip .stagingLocation(...) — it will be ignored\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Only set stagingLocation when staging behavior is actually configured","Treat this warning as informational, not a failure","Check release notes for changes to when manifests are staged vs committed directly"],"tags":["spark","rewrite-manifests","ignored-parameter"],"backgroundTag":"ignored-configuration-parameter","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}