{"record":{"id":"08937e7edae23632","repo":"apache/flink","slug":"the-src-file-with-length-does-not-match-the","errorCode":null,"errorMessage":"The src file {} with length {} does not match the expected length {}","messagePattern":"The src file (.+?) with length (.+?) does not match the expected length (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java","lineNumber":275,"sourceCode":"            try {\n                srcStatus = fs.getFileStatus(src);\n            } catch (FileNotFoundException fnfe) {\n                // srcStatus will be null\n            } catch (IOException e) {\n                throw new IOException(\"Cannot clean commit: Staging file does not exist.\");\n            }\n            if (srcStatus != null) {\n                LOG.debug(\n                        \"The srcStatus is {} and exp length is {}\",\n                        srcStatus.getLen(),\n                        expectedLength);\n                if (srcStatus.getLen() != expectedLength) {\n                    LOG.error(\n                            \"The src file {} with length {} does not match the expected length {}\",\n                            src,\n                            srcStatus.getLen(),\n                            expectedLength);\n                    throw new IOException(\n                            \"The src file \"\n                                    + src\n                                    + \" with length \"\n                                    + srcStatus.getLen()\n                                    + \" \"\n                                    + \"does not match the expected length \"\n                                    + expectedLength);\n                }\n                try {\n                    fs.rename(src, dest);\n                } catch (IOException e) {\n                    throw new IOException(\n                            \"Committing file by rename failed: \" + src + \" to \" + dest, e);\n                }\n            } else if (!fs.exists(dest)) {\n                // neither exists - that can be a sign of\n                //   - (1) a serious problem (file system loss of data)\n                //   - (2) a recovery of a savepoint that is some time old and the users","sourceCodeStart":257,"sourceCodeEnd":293,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-azure-fs-hadoop/src/main/java/org/apache/flink/fs/azurefs/AzureBlobFsRecoverableDataOutputStream.java#L257-L293","documentation":"Thrown by the AzureBlobFsRecoverableDataOutputStream Committer during commit() when the staging file exists but its length differs from the expected length recorded in the recoverable (the offset at persist time). A length mismatch means the staging file was modified (appended to or truncated) after the recoverable was created, so renaming it to the target would commit wrong data.","triggerScenarios":"commit() after task recovery where another attempt of the same operator already appended to the same temp file, or where the file was truncated by a previous failed recovery attempt; also when the same checkpoint state is committed twice with intervening writes.","commonSituations":"Two job attempts running concurrently against one checkpoint (zombie TaskManager); inconsistent output-path configuration so two sinks share temp file names; recovery racing with cleanup.","solutions":["Ensure exactly one active attempt per operator instance (proper failure detection, no duplicate JobManagers)","Give each sink/attempt unique output or temp-dir prefixes to avoid shared temp files","Restart cleanly from the last consistent checkpoint so pending files match their recoverables","Compare the three values in the log line (src path, actual length, expected length) to identify which attempt wrote the extra/missing bytes"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before committing, verify staging length matches the recoverable offset\norg.apache.hadoop.fs.FileStatus st = fs.getFileStatus(recoverable.tempFile());\nif (st.getLen() != recoverable.offset()) {\n    // another attempt modified the file: do not commit, alert\n}","typeGuard":null,"tryCatchPattern":"try {\n    committer.commit();\n} catch (java.io.IOException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"does not match the expected length\")) {\n        alertOnConcurrentModification();\n    } else { throw e; }\n}","preventionTips":["Ensure unique temp file names per job attempt (unique-part-dir / attempt counters)","Prevent zombie TaskManagers from writing while a new attempt recovers","Never point two running jobs at the same output checkpoint prefix"],"tags":["azure","abfs","commit","consistency","file-sink"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}