apache/iceberg · error · RuntimeException

Rewrite data file error.

Error message

Rewrite data file error.

What it means

RewriteDataFilesAction.rewriteDataForTasks submits combined scan tasks to the Flink runtime via RowDataRewriter; any exception surfaced from that pipeline is wrapped in a RuntimeException with the message 'Rewrite data file error.'. The original cause is attached and is what actually matters for diagnosis.

Source

Thrown at flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/actions/RewriteDataFilesAction.java:62

        "Flink does not support compaction on row lineage enabled tables (V3+)");
  }

  @Override
  protected FileIO fileIO() {
    return table().io();
  }

  @Override
  protected List<DataFile> rewriteDataForTasks(List<CombinedScanTask> combinedScanTasks) {
    int size = combinedScanTasks.size();
    int parallelism = Math.min(size, maxParallelism);
    DataStream<CombinedScanTask> dataStream = env.fromData(combinedScanTasks);
    RowDataRewriter rowDataRewriter =
        new RowDataRewriter(table(), caseSensitive(), fileIO(), encryptionManager());
    try {
      return rowDataRewriter.rewriteDataForTasks(dataStream, parallelism);
    } catch (Exception e) {
      throw new RuntimeException("Rewrite data file error.", e);
    }
  }

  @Override
  protected RewriteDataFilesAction self() {
    return this;
  }

  public RewriteDataFilesAction maxParallelism(int parallelism) {
    Preconditions.checkArgument(parallelism > 0, "Invalid max parallelism %s", parallelism);
    this.maxParallelism = parallelism;
    return this;
  }
}

View on GitHub (pinned to 86d9c8fc54)

Solutions

  1. Inspect getCause() of this RuntimeException to find the root failure
  2. Re-run the action; re-plan with a fresh TableScan if files changed between plan and rewrite
  3. Check Flink TaskManager logs for the failing subtask and underlying IO/codec errors
  4. Verify the table is not being concurrently modified during rewrite

Example fix

// before
RewriteDataFilesActionResult r = action.execute(); // wrapped RuntimeException
// after
try {
  RewriteDataFilesActionResult r = action.execute();
} catch (RuntimeException e) {
  LOG.error("Rewrite failed", e.getCause());
  throw e;
}
Defensive patterns

Strategy: try-catch

Try / catch

try { action.execute(); } catch (RuntimeException e) { Throwable root = e.getCause(); LOG.error("Rewrite data file failed: {}", root.getMessage(), root); throw e; }

Prevention

When it happens

Trigger: rewriteDataFiles() action execution where rewrite tasks fail: corrupt/missing data files, IO or deserialization failures in RowDataRewriter, Flink job execution exceptions, schema/encryption mismatches between plan and files.

Common situations: Files changed or deleted between scan planning and rewrite; checkpoint failures; storage (S3/HDFS) transient failures during task execution; concurrent table modification (snapshot conflicts).

Understand the failure class

Background: "failed to write file", "Could not save figure", "Error saving remote file" — file write failed: causes and fixes across languages and libraries — this error's family across 38 libraries.

Related errors

AI-assisted analysis of apache/iceberg@86d9c8fc54 (2026-09-12). Data as JSON: /api/errors/e6db408d506973d4. Report an issue: GitHub.