Fallback content comparison failed, fallback to COPY. source

Error message

Fallback content comparison failed, fallback to COPY. source={}, target={}

What it means

This is a WARN log emitted by ContinuousMultipleTableFileSourceSplitEnumerator when the incremental file-sync logic tried to compare a source file and its target copy by content and the comparison itself threw an exception. Since equality cannot be proven, the enumerator conservatively treats the files as different (returns true) so the file is re-copied via COPY — favoring correctness over skipping. It is a degraded-path warning, not a job-failing error.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java:1911

                    }

                    if (checksumException != null
                            || sourceChecksum == null
                            || targetChecksum == null) {
                        warnChecksumUnavailableOnce(
                                sourceFilePath, targetFilePath, checksumException);
                        try {
                            boolean sameContent = fileContentEquals(sourceFilePath, targetFilePath);
                            if (sameContent && log.isDebugEnabled()) {
                                log.debug(
                                        "Update sync mode skipped file: source={}, target={}, reason={}",
                                        maskUriUserInfo(sourceFilePath),
                                        maskUriUserInfo(targetFilePath),
                                        "strict checksum: content equal (checksum unavailable)");
                            }
                            return !sameContent;
                        } catch (Exception e) {
                            log.warn(
                                    "Fallback content comparison failed, fallback to COPY. source={}, target={}",
                                    maskUriUserInfo(sourceFilePath),
                                    maskUriUserInfo(targetFilePath),
                                    e);
                            return true;
                        }
                    }
                    if (checksumEquals(sourceChecksum, targetChecksum)) {
                        if (log.isDebugEnabled()) {
                            log.debug(
                                    "Update sync mode skipped file: source={}, target={}, reason={}",
                                    maskUriUserInfo(sourceFilePath),
                                    maskUriUserInfo(targetFilePath),
                                    "strict checksum: checksum equal");
                        }
                        return false;
                    }
                    return true;

View on GitHub (pinned to cf67b549a7)

Solutions

Inspect the full stack trace attached to this WARN to find the underlying IO/permission error on source or target path.
Verify the target directory/path is stable and not being concurrently modified or cleaned by another process.
Check read permissions and connectivity for both sourceFilePath and targetFilePath URIs.
If checksums can be enabled/supported by the filesystem, prefer the checksum path so content comparison (and this failure mode) is avoided.

Example fix

// before: content comparison throws, file is redundantly re-copied every cycle
return !sameContent;
// after: fix the underlying cause, e.g. ensure target is not concurrently removed
// (no code change required; the library already falls back safely to COPY)

Defensive patterns

Strategy: fallback

Validate before calling

// Verify both paths are readable before running the sync job
FileSystem src = sourceFilePath.getFileSystem(conf);
FileSystem tgt = targetFilePath.getFileSystem(conf);
if (!src.exists(sourceFilePath) || !tgt.exists(targetFilePath)) {
    throw new IOException("source or target missing before comparison");
}
try (FSDataInputStream a = src.open(sourceFilePath);
     FSDataInputStream b = tgt.open(targetFilePath)) { /* probe read */ }

Try / catch

try {
    boolean different = compareContents(sourceFilePath, targetFilePath);
} catch (Exception e) {
    log.warn("content comparison failed, defaulting to COPY", e);
    boolean different = true; // safe fallback
}

Prevention

Avoid concurrent writers/deleters on the sync target directory.
Ensure read permissions on both source and target URIs.
Prefer filesystems with checksum support to avoid the comparison path entirely.
Monitor this WARN's attached stack trace for recurring IO/permission root causes.

When it happens

Trigger: Occurs during the 'should copy?' decision path when file checksums are unavailable and the code falls back to a direct content comparison (streaming read of source and target), and that comparison throws — e.g. the target file disappears mid-read, a filesystem/IO error occurs while reading either URI, or the storage backend rejects the read (permission/timeout).

Common situations: Target file deleted or truncated by another job while the enumerator compares; transient HDFS/S3/OSS read failures; permission issues on the target path; network blips to object storage during content streaming.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/90aafa86c10656f0. Report an issue: GitHub.