What causes "Rewrite data file error." in apache/iceberg?

Inspect the chained cause for the actual failure (IO, schema, OOM) and fix that first.

apache/iceberg · error · java.lang.RuntimeException

Rewrite data file error.

Error message

Rewrite data file error.

What it means

Top-level wrapper exception for the rewrite-data-files action: RowDataRewriter.rewriteDataForTasks threw while rewriting combined scan tasks, and the action rethrows as RuntimeException('Rewrite data file error.') with the cause chained.

Source

Thrown at flink/v2.3/flink/src/main/java/org/apache/iceberg/flink/actions/RewriteDataFilesAction.java:62

        "Flink does not support compaction on row lineage enabled tables (V3+)");
  }

  @Override
  protected FileIO fileIO() {
    return table().io();
  }

  @Override
  protected List<DataFile> rewriteDataForTasks(List<CombinedScanTask> combinedScanTasks) {
    int size = combinedScanTasks.size();
    int parallelism = Math.min(size, maxParallelism);
    DataStream<CombinedScanTask> dataStream = env.fromData(combinedScanTasks);
    RowDataRewriter rowDataRewriter =
        new RowDataRewriter(table(), caseSensitive(), fileIO(), encryptionManager());
    try {
      return rowDataRewriter.rewriteDataForTasks(dataStream, parallelism);
    } catch (Exception e) {
      throw new RuntimeException("Rewrite data file error.", e);
    }
  }

  @Override
  protected RewriteDataFilesAction self() {
    return this;
  }

  public RewriteDataFilesAction maxParallelism(int parallelism) {
    Preconditions.checkArgument(parallelism > 0, "Invalid max parallelism %s", parallelism);
    this.maxParallelism = parallelism;
    return this;
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

Inspect the chained cause for the actual failure (IO, schema, OOM) and fix that first.
Verify FileIO credentials/permissions for both source and target locations.
Reduce parallelism or rewrite smaller task groups; rerun the action (it is retryable).

Defensive patterns

Strategy: retry

Validate before calling

table.io().newInputFile(...); // verify readable
table.io().newOutputFile(target); // verify writable via dry-run if available

Try / catch

try { result = action.execute(); } catch (RuntimeException e) { LOG.error("Rewrite failed", e.getCause()); /* retry or re-scope */ }

Prevention

Check FileIO credentials and target directory permissions first.
Run rewrites in smaller batches; monitor TaskManager memory.
The action is idempotent-ish per task group — safe to rerun after fixing the root cause.

When it happens

Trigger: Any failure inside the distributed rewrite of data files — writer failures, serialization/deserialization of tasks, IO errors writing new parquet files, or execution exceptions from the Flink datastream.

Common situations: Target location not writable (permissions/credentials); schema incompatibility when re-reading old files; task manager OOM during large rewrites.

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/f7ee2c04c82063f6. Report an issue: GitHub.