apache/seatunnel · warning

File checksum is not available, fallback to content comparis

Error message

File checksum is not available, fallback to content comparison. source={}, target={}

What it means

Warning logged when a strict checksum-based copy-comparison cannot obtain file checksums (or checksum computation threw, with checksumException == null meaning no exception but a checksum was null). The reader falls back to comparing file contents directly instead of using the filesystem checksum API. This is a performance/degradation notice, not a failure.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/reader/AbstractReadStrategy.java:958

                logUpdateModeSkip(
                        sourceFilePath, targetFilePath, "strict len_mtime: len and mtime equal");
                return false;
            }
            if (compareMode == FileCompareMode.CHECKSUM) {
                FileChecksum sourceChecksum = null;
                FileChecksum targetChecksum = null;
                Exception checksumException = null;
                try {
                    sourceChecksum = hadoopFileSystemProxy.getFileChecksum(sourceFilePath);
                    targetChecksum = targetHadoopFileSystemProxy.getFileChecksum(targetFilePath);
                } catch (Exception e) {
                    checksumException = e;
                }

                if (checksumException != null || sourceChecksum == null || targetChecksum == null) {
                    if (!checksumUnavailableWarned) {
                        if (checksumException == null) {
                            log.warn(
                                    "File checksum is not available, fallback to content comparison. source={}, target={}",
                                    maskUriUserInfo(sourceFilePath),
                                    maskUriUserInfo(targetFilePath));
                        } else {
                            log.warn(
                                    "File checksum is not available, fallback to content comparison. source={}, target={}",
                                    maskUriUserInfo(sourceFilePath),
                                    maskUriUserInfo(targetFilePath),
                                    checksumException);
                        }
                        checksumUnavailableWarned = true;
                    }
                    try {
                        boolean sameContent = fileContentEquals(sourceFilePath, targetFilePath);
                        if (sameContent) {
                            logUpdateModeSkip(
                                    sourceFilePath,
                                    targetFilePath,

View on GitHub (pinned to cf67b549a7)

Solutions

1. Expect slower content-based comparison; ensure network/disk capacity allows full file reads during sync.
2. Prefer source/target filesystem pairs that both support checksums (HDFS-to-HDFS with same algorithm).
3. If this is noisy, switch the sync/comparison mode away from strict checksum so content comparison is used directly.
4. Verify Hadoop checksum policy settings (io.bytes.per.checksum / checksum type) are identical on both sides.

// check checksum availability before strict mode
FileChecksum src = fs.getFileChecksum(srcPath);
FileChecksum dst = targetFs.getFileChecksum(dstPath);
boolean strictUsable = src != null && dst != null
    && src.getAlgorithmName().equals(dst.getAlgorithmName());

try {
    FileChecksum c = fs.getFileChecksum(path);
    if (c == null) { /* plan content comparison up front */ }
} catch (IOException e) { /* plan content comparison up front */ }

When it happens

Trigger: During strict-mode file sync comparison, FileSystem.getFileChecksum() returns null or throws for the source or target file — e.g. local files, S3/OSS filesystems that don't implement checksums, or differing checksum algorithms between source and target clusters.

Common situations: Copying between HDFS and object storage (S3/OSS) where one side has no checksum; copying between Hadoop clusters with different checksum algorithms (e.g. CRC32 vs MD5); small files where checksum API is unavailable.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• Fallback content comparison failed, fallback to COPY. source
• Fallback content comparison failed, fallback to COPY. source
• File checksum is not available, fallback to content comparis
• Failed to derive qualified default table location, fallback
• Circular condition chain detected: '%s' already exists in th

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/5106a4c0e518440a. Report an issue: GitHub.