{"record":{"id":"5106a4c0e518440a","repo":"apache/seatunnel","slug":"file-checksum-is-not-available-fallback-to-conten","errorCode":null,"errorMessage":"File checksum is not available, fallback to content comparison. source={}, target={}","messagePattern":"File checksum is not available, fallback to content comparison\\. source=(.+?), target=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/AbstractReadStrategy.java","lineNumber":958,"sourceCode":"                logUpdateModeSkip(\n                        sourceFilePath, targetFilePath, \"strict len_mtime: len and mtime equal\");\n                return false;\n            }\n            if (compareMode == FileCompareMode.CHECKSUM) {\n                FileChecksum sourceChecksum = null;\n                FileChecksum targetChecksum = null;\n                Exception checksumException = null;\n                try {\n                    sourceChecksum = hadoopFileSystemProxy.getFileChecksum(sourceFilePath);\n                    targetChecksum = targetHadoopFileSystemProxy.getFileChecksum(targetFilePath);\n                } catch (Exception e) {\n                    checksumException = e;\n                }\n\n                if (checksumException != null || sourceChecksum == null || targetChecksum == null) {\n                    if (!checksumUnavailableWarned) {\n                        if (checksumException == null) {\n                            log.warn(\n                                    \"File checksum is not available, fallback to content comparison. source={}, target={}\",\n                                    maskUriUserInfo(sourceFilePath),\n                                    maskUriUserInfo(targetFilePath));\n                        } else {\n                            log.warn(\n                                    \"File checksum is not available, fallback to content comparison. source={}, target={}\",\n                                    maskUriUserInfo(sourceFilePath),\n                                    maskUriUserInfo(targetFilePath),\n                                    checksumException);\n                        }\n                        checksumUnavailableWarned = true;\n                    }\n                    try {\n                        boolean sameContent = fileContentEquals(sourceFilePath, targetFilePath);\n                        if (sameContent) {\n                            logUpdateModeSkip(\n                                    sourceFilePath,\n                                    targetFilePath,","sourceCodeStart":940,"sourceCodeEnd":976,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/AbstractReadStrategy.java#L940-L976","documentation":"Warning logged when a strict checksum-based copy-comparison cannot obtain file checksums (or checksum computation threw, with checksumException == null meaning no exception but a checksum was null). The reader falls back to comparing file contents directly instead of using the filesystem checksum API. This is a performance/degradation notice, not a failure.","triggerScenarios":"During strict-mode file sync comparison, FileSystem.getFileChecksum() returns null or throws for the source or target file — e.g. local files, S3/OSS filesystems that don't implement checksums, or differing checksum algorithms between source and target clusters.","commonSituations":"Copying between HDFS and object storage (S3/OSS) where one side has no checksum; copying between Hadoop clusters with different checksum algorithms (e.g. CRC32 vs MD5); small files where checksum API is unavailable.","solutions":["Expect slower content-based comparison; ensure network/disk capacity allows full file reads during sync.","Prefer source/target filesystem pairs that both support checksums (HDFS-to-HDFS with same algorithm).","If this is noisy, switch the sync/comparison mode away from strict checksum so content comparison is used directly.","Verify Hadoop checksum policy settings (io.bytes.per.checksum / checksum type) are identical on both sides."],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"// check checksum availability before strict mode\nFileChecksum src = fs.getFileChecksum(srcPath);\nFileChecksum dst = targetFs.getFileChecksum(dstPath);\nboolean strictUsable = src != null && dst != null\n    && src.getAlgorithmName().equals(dst.getAlgorithmName());","typeGuard":null,"tryCatchPattern":"try {\n    FileChecksum c = fs.getFileChecksum(path);\n    if (c == null) { /* plan content comparison up front */ }\n} catch (IOException e) { /* plan content comparison up front */ }","preventionTips":["Use source/target filesystem pairs with matching checksum support.","Don't assume checksums exist on S3/OSS/local filesystems.","Budget for slower full-content comparison when strict mode falls back."],"tags":["hadoop","checksum","fallback","file-sync"],"backgroundTag":"checksum-mismatch","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}