apache/iceberg · info

Ignoring provided staging location as new manifests will be

Error message

Ignoring provided staging location as new manifests will be committed directly

What it means

RewriteManifestsSparkAction only uses a custom staging location when manifests must actually be rewritten as staged files; when it determines manifests can be committed directly (shouldStageManifests is false), a user-provided stagingLocation is ignored and this warning is logged. It is informational: the call has no effect on the outcome.

Source

Thrown at spark/v4.1/spark/src/main/java/org/apache/iceberg/spark/actions/RewriteManifestsSparkAction.java:168

  @Override
  public RewriteManifestsSparkAction specId(int specId) {
    Preconditions.checkArgument(table.specs().containsKey(specId), "Invalid spec id %s", specId);
    this.spec = table.specs().get(specId);
    return this;
  }

  @Override
  public RewriteManifestsSparkAction rewriteIf(Predicate<ManifestFile> newPredicate) {
    this.predicate = newPredicate;
    return this;
  }

  @Override
  public RewriteManifestsSparkAction stagingLocation(String newStagingLocation) {
    if (shouldStageManifests) {
      this.outputLocation = newStagingLocation;
    } else {
      LOG.warn("Ignoring provided staging location as new manifests will be committed directly");
    }
    return this;
  }

  @Override
  public RewriteManifests.Result execute() {
    String desc = String.format("Rewriting manifests in %s", table.name());
    JobGroupInfo info = newJobGroupInfo("REWRITE-MANIFESTS", desc);
    return withJobGroupInfo(info, this::doExecute);
  }

  @Override
  public RewriteManifestsSparkAction sortBy(List<String> partitionFields) {
    // Collect set of available partition columns to cluster on
    Set<String> availablePartitionNames =
        spec.fields().stream().map(PartitionField::name).collect(Collectors.toSet());

    // Identify specified partition fields that are not available in the spec

View on GitHub (pinned to 86d9c8fc54)

Solutions

1. Verify whether the action will stage manifests before setting a staging location
2. Remove the staging-location option if the warning appears; it has no effect
3. If a staged location is required, ensure conditions force manifest staging (e.g. use the API path that requires staging)

Example fix

// before
actions.rewriteManifests(table).stagingLocation("s3://bucket/tmp/manifests").execute();
// after
RewriteManifestsSparkAction rewrite = actions.rewriteManifests(table);
if (stagingRequired) { rewrite.stagingLocation("s3://bucket/tmp/manifests"); }
rewrite.execute();

if (actionWillStageManifests(table) && stagingLocation != null) { action.stagingLocation(stagingLocation); }

When it happens

Trigger: Calling stagingLocation(String) on RewriteManifestsSparkAction when shouldStageManifests is false, i.e. when the rewrite will write manifests directly rather than staging them.

Common situations: Users pass 'staging-location' expecting all manifest writes to go to a custom path, unaware the action decided direct commits are fine (e.g. all manifests already need rewriting with no partial groups).

Related errors

• Ignoring provided staging location as new manifests will be
• Unsupported manifest content type:
• Unknown manifest content: ${content}
• Unsupported manifest type: ${manifestFile.content()}
• max_concurrent_deletes only works with FileIOs that do not s

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/2307241aee5d9c11. Report an issue: GitHub.