{"record":{"id":"edaba1fbe371e654","repo":"apache/seatunnel","slug":"file-checksum-is-not-available-fallback-to-conten-edaba1","errorCode":null,"errorMessage":"File checksum is not available, fallback to content comparison. source={}, target={}","messagePattern":"File checksum is not available, fallback to content comparison\\. source=(.+?), target=(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java","lineNumber":1956,"sourceCode":"            String targetPath = config.get(FileBaseSourceOptions.TARGET_PATH);\n            return buildTargetFilePath(targetPath, relativePath);\n        }\n\n        private boolean fileContentEquals(String sourceFilePath, String targetFilePath)\n                throws IOException {\n            try (InputStream sourceIn = sourceFs.getInputStream(sourceFilePath);\n                    InputStream targetIn = targetFs.getInputStream(targetFilePath)) {\n                return IOUtils.contentEquals(sourceIn, targetIn);\n            }\n        }\n\n        private void warnChecksumUnavailableOnce(\n                String sourceFilePath, String targetFilePath, Exception checksumException) {\n            if (checksumUnavailableWarned) {\n                return;\n            }\n            if (checksumException == null) {\n                log.warn(\n                        \"File checksum is not available, fallback to content comparison. source={}, target={}\",\n                        maskUriUserInfo(sourceFilePath),\n                        maskUriUserInfo(targetFilePath));\n            } else {\n                log.warn(\n                        \"File checksum is not available, fallback to content comparison. source={}, target={}\",\n                        maskUriUserInfo(sourceFilePath),\n                        maskUriUserInfo(targetFilePath),\n                        checksumException);\n            }\n            checksumUnavailableWarned = true;\n        }\n\n        private boolean filterByPattern(FileStatus fileStatus) {\n            if (pattern == null || fileBasePath == null) {\n                return true;\n            }\n            if (pattern.pattern().startsWith(fileBasePath)) {","sourceCodeStart":1938,"sourceCodeEnd":1974,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java#L1938-L1974","documentation":"This WARN is logged once (guarded by checksumUnavailableWarned) by warnChecksumUnavailableOnce when the filesystem/file API provides no checksum for the source or target file, so the enumerator cannot use a cheap checksum-based equality check and instead falls back to a byte-by-byte content comparison. It is informational about degraded (more expensive) comparison, not a failure.","triggerScenarios":"First time a file pair is compared in a sync cycle where obtaining a file checksum returns null/unavailable for the storage backend — e.g. object stores (S3/OSS) or local filesystems that do not expose checksums through the Hadoop FileSystem API used here.","commonSituations":"Using S3, OSS, or local FS targets that don't return checksums; HDFS erasure-coded or encrypted zones without checksum support; comparing files across heterogeneous filesystems where one side lacks checksum support.","solutions":["No action strictly required — the sync continues correctly with content comparison.","If performance matters, store files on a filesystem that exposes checksums (e.g. HDFS with checksums enabled) so equality checks stay cheap.","Expect this WARN only once per enumerator lifecycle; if it recurs the enumator was likely restarted (new instance)."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Check checksum support before enabling checksum-based sync\nFileChecksum ck = fs.getFileChecksum(path);\nboolean checksumSupported = ck != null;","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Choose checksum-capable storage (e.g. HDFS) for sync targets when comparing frequently.","Treat this WARN as a performance hint, not an error — content comparison is correct but slower.","For large files on non-checksum filesystems, compare size+mtime first to short-circuit comparisons."],"tags":["file","checksum","fallback","performance","log-warning"],"backgroundTag":"checksum-mismatch","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}