{"record":{"id":"fe9904272ff27391","repo":"apache/seatunnel","slug":"failed-to-read-manifest-assuming-files-not-yet","errorCode":null,"errorMessage":"Failed to read manifest {}, assuming files not yet committed","messagePattern":"Failed to read manifest (.+?), assuming files not yet committed","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/sink/commit/IcebergFilesCommitter.java","lineNumber":150,"sourceCode":"                            ManifestFiles.read(manifest, table.io())) {\n                        for (DataFile file : reader) {\n                            if (pendingPaths.contains(file.path().toString())) {\n                                return true;\n                            }\n                        }\n                    }\n                } else {\n                    try (CloseableIterable<DeleteFile> reader =\n                            ManifestFiles.readDeleteManifest(manifest, table.io(), table.specs())) {\n                        for (DeleteFile file : reader) {\n                            if (pendingPaths.contains(file.path().toString())) {\n                                return true;\n                            }\n                        }\n                    }\n                }\n            } catch (IOException e) {\n                log.warn(\n                        \"Failed to read manifest {}, assuming files not yet committed\",\n                        manifest.path(),\n                        e);\n            }\n        }\n        return false;\n    }\n\n    private void commit(\n            TableIdentifier tableIdentifier, List<WriteResult> results, long checkpointId) {\n        List<DataFile> dataFiles =\n                results.stream()\n                        .filter(payload -> payload.getDataFiles() != null)\n                        .flatMap(payload -> payload.getDataFiles().stream())\n                        .filter(dataFile -> dataFile.recordCount() > 0)\n                        .collect(toList());\n\n        List<DeleteFile> deleteFiles =","sourceCodeStart":132,"sourceCodeEnd":168,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-iceberg/src/main/java/org/apache/seatunnel/connectors/seatunnel/iceberg/sink/commit/IcebergFilesCommitter.java#L132-L168","documentation":"This warning is logged during Iceberg commit deduplication (areFilesAlreadyCommitted) when a manifest file cannot be read due to an IOException. The committer treats unreadable manifests as evidence that the files were not yet committed (conservative fallback) so recovery/commit proceeds instead of failing. This typically happens when a manifest listed in a snapshot was deleted by expiry or is not yet visible (eventual consistency on object stores).","triggerScenarios":"During isAlreadyCommitted after restore/restart, the committer iterates snapshot manifests and IOException occurs reading a manifest (manifest expired/removed by table maintenance, transient S3/HDFS read failure, or permissions issue).","commonSituations":"Job restored from checkpoint after snapshot expiry removed old manifests; object-store eventual consistency causing a just-written manifest to be temporarily invisible; HDFS NameNode failover or transient network errors.","solutions":["Verify the Iceberg table's snapshot expiry/retention settings and ensure they are longer than the job's checkpoint interval","Check filesystem/object-store permissions and network stability for the warehouse path","Retry the job — the warning is non-fatal by design and the commit proceeds","If it recurs constantly, inspect table metadata for orphaned manifest references (e.g. after a failed expireSnapshots)"],"exampleFix":"// before\n// table expires snapshots aggressively\nexpireSnapshots.olderThan(60_000).execute();\n// after\n// retain snapshots longer than job checkpoint interval\nexpireSnapshots.olderThan(System.currentTimeMillis() - 86400_000).execute();","handlingStrategy":"retry","validationCode":"// before committing, check manifest readability\nManifestFile mf = ...;\ntry (FileIO io = table.io()) {\n    io.newInputFile(mf.path());\n} catch (IOException e) { /* treat as not committed / retry */ }","typeGuard":null,"tryCatchPattern":"try { areFilesAlreadyCommitted(...); } catch (IOException e) {\n    log.warn(\"Manifest unreadable, will retry commit\", e);\n    retryWithBackoff(() -> areFilesAlreadyCommitted(...));\n}","preventionTips":["Keep snapshot retention longer than job checkpoint/restart intervals","Avoid aggressive expireSnapshots while dependent jobs run","Use stable object-store connectivity; enable retries in Hadoop/S3 filesystem configs"],"tags":["iceberg","io","manifest","commit"],"backgroundTag":"file-read-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}