apache/iceberg · info

Ignoring provided staging location as new manifests will be

Error message

Ignoring provided staging location as new manifests will be committed directly

What it means

WARN log from RewriteManifestsSparkAction.stagingLocation: the caller supplied a staging location, but the rewrite plan does not actually stage manifests (shouldStageManifests is false, e.g. when manifests are committed directly without a separate staging phase), so the provided location is ignored.

Source

Thrown at spark/v4.0/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteManifestsSparkAction.java:168

  @Override
  public RewriteManifestsSparkAction specId(int specId) {
    Preconditions.checkArgument(table.specs().containsKey(specId), "Invalid spec id %s", specId);
    this.spec = table.specs().get(specId);
    return this;
  }

  @Override
  public RewriteManifestsSparkAction rewriteIf(Predicate<ManifestFile> newPredicate) {
    this.predicate = newPredicate;
    return this;
  }

  @Override
  public RewriteManifestsSparkAction stagingLocation(String newStagingLocation) {
    if (shouldStageManifests) {
      this.outputLocation = newStagingLocation;
    } else {
      LOG.warn("Ignoring provided staging location as new manifests will be committed directly");
    }
    return this;
  }

  @Override
  public RewriteManifests.Result execute() {
    String desc = String.format("Rewriting manifests in %s", table.name());
    JobGroupInfo info = newJobGroupInfo("REWRITE-MANIFESTS", desc);
    return withJobGroupInfo(info, this::doExecute);
  }

  @Override
  public RewriteManifestsSparkAction sortBy(List<String> partitionFields) {
    // Collect set of available partition columns to cluster on
    Set<String> availablePartitionNames =
        spec.fields().stream().map(PartitionField::name).collect(Collectors.toSet());

    // Identify specified partition fields that are not available in the spec

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify whether the target Iceberg version still uses staged manifests; if not, drop the staging-location option.
2. If a custom storage location is required, upgrade/downgrade to a version whose rewrite path stages manifests, or manually rewrite manifests.
3. Confirm the commit succeeds despite the ignored option; this is purely informational.

Example fix

// before
actions.rewriteManifests(table).stagingLocation("s3://bucket/staging").execute();
// after: only set when staging is actually used
RewriteManifestsSparkAction a = actions.rewriteManifests(table);
if (usesStagedManifests(table)) { a.stagingLocation("s3://bucket/staging"); }
a.execute();

boolean stagingUsed = /* true only on versions/paths that stage manifests */; if (stagingUsed) action.stagingLocation(loc);

When it happens

Trigger: Calling RewriteManifestsSparkAction.stagingLocation("s3://...") when shouldStageManifests is false — i.e. the current rewrite strategy writes new manifests directly as part of the commit instead of staging them first.

Common situations: Copy-pasted config from older Iceberg versions where staging was always used; passing staging-location on newer catalogs/versions that commit manifests directly; users expecting replacement manifests to land in a custom prefix for lifecycle reasons.

Related errors

• max_concurrent_deletes only works with FileIOs that do not s
• Ignoring provided staging location as new manifests will be
• Unsupported manifest content type:
• Unknown manifest content: ${content}
• Unsupported manifest type: ${manifestFile.content()}

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/e9c2c2ca70cbaa08. Report an issue: GitHub.