{"record":{"id":"b09de1170ea5e56d","repo":"apache/iceberg","slug":"rewrite-data-file-error-b09de1","errorCode":null,"errorMessage":"Rewrite data file error.","messagePattern":"Rewrite data file error\\.","errorType":"exception","errorClass":"java.lang.RuntimeException","httpStatus":null,"severity":"error","filePath":"flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/actions/RewriteDataFilesAction.java","lineNumber":62,"sourceCode":"        \"Flink does not support compaction on row lineage enabled tables (V3+)\");\n  }\n\n  @Override\n  protected FileIO fileIO() {\n    return table().io();\n  }\n\n  @Override\n  protected List<DataFile> rewriteDataForTasks(List<CombinedScanTask> combinedScanTasks) {\n    int size = combinedScanTasks.size();\n    int parallelism = Math.min(size, maxParallelism);\n    DataStream<CombinedScanTask> dataStream = env.fromData(combinedScanTasks);\n    RowDataRewriter rowDataRewriter =\n        new RowDataRewriter(table(), caseSensitive(), fileIO(), encryptionManager());\n    try {\n      return rowDataRewriter.rewriteDataForTasks(dataStream, parallelism);\n    } catch (Exception e) {\n      throw new RuntimeException(\"Rewrite data file error.\", e);\n    }\n  }\n\n  @Override\n  protected RewriteDataFilesAction self() {\n    return this;\n  }\n\n  public RewriteDataFilesAction maxParallelism(int parallelism) {\n    Preconditions.checkArgument(parallelism > 0, \"Invalid max parallelism %s\", parallelism);\n    this.maxParallelism = parallelism;\n    return this;\n  }\n}\n","sourceCodeStart":44,"sourceCodeEnd":77,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v2.1/flink/src/main/java/org/apache/iceberg/flink/actions/RewriteDataFilesAction.java#L44-L77","documentation":"RewriteDataFilesAction.execute() wraps any failure from the distributed RowDataRewriter (reading, rewriting, or committing data files) in a generic RuntimeException with the message 'Rewrite data file error.' It signals that a Flink job stage rewriting data files failed; the original cause is attached and must be inspected for the real problem (e.g. read errors, checkpoint failures, commit conflicts).","triggerScenarios":"Calling RewriteDataFilesAction (RewriteDataFiles Spark-style action in Flink) and the rewriteDataForTasks DataStream job throws any Exception: parquet/orc read failures, out-of-memory in RowDataRewriter, schema mismatch, or commit failures in the sink.","commonSituations":"Running the rewrite action on tables with corrupted files, on tables whose schema changed, or with insufficient task manager memory; also job-level failures like checkpoint loss or parallelism misconfiguration.","solutions":["Inspect the attached cause `e` (print full stack trace) — the message itself is generic and the real error is nested.","Verify task manager memory and parallelism are adequate for the data volume being rewritten.","Validate the table can be read fully (e.g. run a scan) to rule out corrupted/inaccessible data files.","Retry the action; Flink transient failures (network, checkpoint timeouts) are common causes.","Upgrade/check Flink-Iceberg version compatibility if the failure is reproducible on valid tables."],"exampleFix":"// before\nrewriteAction.execute();\n// after\ntry {\n  rewriteAction.execute();\n} catch (RuntimeException e) {\n  LOG.error(\"Rewrite failed\", e.getCause()); // inspect nested cause for the real error\n}","handlingStrategy":"try-catch","validationCode":"// before executing\nRewriteDataFilesAction action = actions.rewriteDataFiles(table);\ntable.refresh(); // ensure metadata is current\n// optionally validate readability:\ntable.newScan().planFiles().forEach(f -> Preconditions.checkNotNull(f));","typeGuard":null,"tryCatchPattern":"try {\n  action.execute();\n} catch (RuntimeException e) {\n  Throwable cause = e.getCause();\n  LOG.error(\"Rewrite data files failed: {}\", cause == null ? e : cause.getMessage(), cause);\n  throw cause instanceof RuntimeException ? (RuntimeException) cause : e;\n}","preventionTips":["Always log/unwrap the nested cause — the top message is generic.","Size task manager memory for the largest file groups being rewritten.","Verify all data files are readable (scan) before rewriting.","Keep flink/iceberg versions matched per the compatibility matrix."],"tags":["flink","rewrite","data-files","job-failure"],"backgroundTag":"unsupported-operation","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}