Ignoring provided staging location as new manifests will be…

Error message

Ignoring provided staging location as new manifests will be committed directly

What it means

This is a LOG.warn, not an exception. RewriteManifestsSparkAction only uses a custom staging location when manifests actually need to be staged (rewritten) before commit. When the plan determines new manifests can be committed directly (e.g. nothing to rewrite / writing directly to the table location), a user-provided stagingLocation cannot be applied and is silently ignored with this warning.

Solutions

Verify whether the action is staging manifests at all; the warning is benign if direct commit is expected — remove the stagingLocation call if it has no effect.
Check the rewrite plan (shouldStageManifests derivation) to understand why staging was skipped; adjust rewrite options so staging actually occurs if you require it.
If a specific staging location is mandatory for your storage, force staging behavior via the relevant rewrite.* table/action options rather than relying on stagingLocation alone.

Example fix

// before
actions.rewriteManifests(table).stagingLocation("s3://bucket/staging").execute();
// after
RewriteManifestsSparkAction action = actions.rewriteManifests(table);
if (action.willStageManifests()) { // check/ensure staging is planned
  action.stagingLocation("s3://bucket/staging");
}
action.execute();

Defensive patterns

Strategy: validation

Validate before calling

// Verify the rewrite plan stages manifests before supplying a staging location
if (!actionStagesManifests(table, options)) {
  // skip .stagingLocation(...) — it will be ignored
}

Prevention

Only set stagingLocation when staging behavior is actually configured
Treat this warning as informational, not a failure
Check release notes for changes to when manifests are staged vs committed directly

When it happens

Trigger: Calling rewriteManifests(table).stagingLocation("s3://bucket/tmp/").execute() when the action's shouldStageManifests flag is false, i.e. the rewrite plan commits new manifests directly without a staging phase.

Common situations: Operators configuring a staging dir for S3 write-committed correctness or cost reasons, unaware the action decided direct commit is sufficient; copying configuration from an older Iceberg version where staging was always used.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/0dd15722f60fac09. Report an issue: GitHub.

Appendix: source

Thrown at spark/v4.2/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteManifestsSparkAction.java:168

  @Override
  public RewriteManifestsSparkAction specId(int specId) {
    Preconditions.checkArgument(table.specs().containsKey(specId), "Invalid spec id %s", specId);
    this.spec = table.specs().get(specId);
    return this;
  }

  @Override
  public RewriteManifestsSparkAction rewriteIf(Predicate<ManifestFile> newPredicate) {
    this.predicate = newPredicate;
    return this;
  }

  @Override
  public RewriteManifestsSparkAction stagingLocation(String newStagingLocation) {
    if (shouldStageManifests) {
      this.outputLocation = newStagingLocation;
    } else {
      LOG.warn("Ignoring provided staging location as new manifests will be committed directly");
    }
    return this;
  }

  @Override
  public RewriteManifests.Result execute() {
    String desc = String.format("Rewriting manifests in %s", table.name());
    JobGroupInfo info = newJobGroupInfo("REWRITE-MANIFESTS", desc);
    return withJobGroupInfo(info, this::doExecute);
  }

  @Override
  public RewriteManifestsSparkAction sortBy(List<String> partitionFields) {
    // Collect set of available partition columns to cluster on
    Set<String> availablePartitionNames =
        spec.fields().stream().map(PartitionField::name).collect(Collectors.toSet());

    // Identify specified partition fields that are not available in the spec

View on GitHub (pinned to 86d9c8fc54)