{"record":{"id":"e6db408d506973d4","repo":"apache/iceberg","slug":"rewrite-data-file-error","errorCode":null,"errorMessage":"Rewrite data file error.","messagePattern":"Rewrite data file error\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/actions/RewriteDataFilesAction.java","lineNumber":62,"sourceCode":"        \"Flink does not support compaction on row lineage enabled tables (V3+)\");\n  }\n\n  @Override\n  protected FileIO fileIO() {\n    return table().io();\n  }\n\n  @Override\n  protected List<DataFile> rewriteDataForTasks(List<CombinedScanTask> combinedScanTasks) {\n    int size = combinedScanTasks.size();\n    int parallelism = Math.min(size, maxParallelism);\n    DataStream<CombinedScanTask> dataStream = env.fromData(combinedScanTasks);\n    RowDataRewriter rowDataRewriter =\n        new RowDataRewriter(table(), caseSensitive(), fileIO(), encryptionManager());\n    try {\n      return rowDataRewriter.rewriteDataForTasks(dataStream, parallelism);\n    } catch (Exception e) {\n      throw new RuntimeException(\"Rewrite data file error.\", e);\n    }\n  }\n\n  @Override\n  protected RewriteDataFilesAction self() {\n    return this;\n  }\n\n  public RewriteDataFilesAction maxParallelism(int parallelism) {\n    Preconditions.checkArgument(parallelism > 0, \"Invalid max parallelism %s\", parallelism);\n    this.maxParallelism = parallelism;\n    return this;\n  }\n}\n","sourceCodeStart":44,"sourceCodeEnd":77,"githubUrl":"https://github.com/apache/iceberg/blob/86d9c8fc543e7c56c9f624eb725f76c9baff9570/flink/v1.20/flink/src/main/java/org/apache/iceberg/flink/actions/RewriteDataFilesAction.java#L44-L77","documentation":"RewriteDataFilesAction.rewriteDataForTasks submits combined scan tasks to the Flink runtime via RowDataRewriter; any exception surfaced from that pipeline is wrapped in a RuntimeException with the message 'Rewrite data file error.'. The original cause is attached and is what actually matters for diagnosis.","triggerScenarios":"rewriteDataFiles() action execution where rewrite tasks fail: corrupt/missing data files, IO or deserialization failures in RowDataRewriter, Flink job execution exceptions, schema/encryption mismatches between plan and files.","commonSituations":"Files changed or deleted between scan planning and rewrite; checkpoint failures; storage (S3/HDFS) transient failures during task execution; concurrent table modification (snapshot conflicts).","solutions":["Inspect getCause() of this RuntimeException to find the root failure","Re-run the action; re-plan with a fresh TableScan if files changed between plan and rewrite","Check Flink TaskManager logs for the failing subtask and underlying IO/codec errors","Verify the table is not being concurrently modified during rewrite"],"exampleFix":"// before\nRewriteDataFilesActionResult r = action.execute(); // wrapped RuntimeException\n// after\ntry {\n  RewriteDataFilesActionResult r = action.execute();\n} catch (RuntimeException e) {\n  LOG.error(\"Rewrite failed\", e.getCause());\n  throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { action.execute(); } catch (RuntimeException e) { Throwable root = e.getCause(); LOG.error(\"Rewrite data file failed: {}\", root.getMessage(), root); throw e; }","preventionTips":["Inspect e.getCause() before troubleshooting","Avoid concurrent table mutations during rewrite","Retry transient IO failures","Monitor TaskManager logs during compaction"],"tags":["flink","compaction","runtime-exception"],"backgroundTag":"file-write-failed","analyzedSha":"86d9c8fc543e7c56c9f624eb725f76c9baff9570","analyzedAt":"2026-09-12T00:46:39.097Z","contentChangedAt":"2026-09-12T00:46:39.097Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}