apache/seatunnel · warning

Continuous discovery scan failed, will retry in next interva

Error message

Continuous discovery scan failed, will retry in next interval.

What it means

safeScanOnce wraps the periodic scanOnce() call: if scanOnce throws any Exception (e.g. an IOException from filesystem listing/reading), it logs this warning with the exception and returns, letting the next scheduler interval retry. The continuous discovery loop is designed to be resilient; one failed scan does not fail the job.

Source

Thrown at seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java:402

                return false;
            }
            inFlightSplitContexts.remove(splitId);
            inFlightSplits.removeIf(s -> Objects.equals(s.splitId(), splitId));
            if (operationState != null) {
                finishedAwaitingCheckpoint.add(operationState);
            }
            return true;
        }
    }

    private void safeScanOnce() {
        if (closed) {
            return;
        }
        try {
            scanOnce();
        } catch (Exception e) {
            log.warn("Continuous discovery scan failed, will retry in next interval.", e);
        }
    }

    @VisibleForTesting
    void scanOnceForTest() throws IOException {
        scanOnce();
    }

    private void scanOnce() throws IOException {
        int scanned = 0;
        int queued = 0;
        Set<String> activeKnownSplitIds = new HashSet<>();
        for (TableScanContext ctx : tableScanContexts) {
            List<FileStatus> files = ctx.listFiles(ctx.rootPath);
            scanned += files.size();
            for (FileStatus fileStatus : files) {
                if (!ctx.shouldProcess(fileStatus, jobStartTimeMillis, startMode)) {
                    clearKnownVersionIfPresent(ctx.tableId, fileStatus.getPath().toString());

View on GitHub (pinned to cf67b549a7)

Solutions

1. Inspect the logged cause for the underlying filesystem error and address it (network, credentials, permissions)
2. Verify the source and target filesystems are reachable and paths exist with correct permissions
3. If failures persist every interval, fix the persistent filesystem issue; one-off occurrences can be ignored since the next tick retries

// Pre-check reachability of source/target paths before scheduling scans
for (String p : java.util.Arrays.asList(sourcePath, targetPath)) {
    if (!fs.exists(new org.apache.hadoop.fs.Path(p))) {
        log.warn("Path not reachable before scan: {}", p);
    }
}

try {
    scanOnce();
} catch (Exception e) {
    log.warn("Continuous discovery scan failed, will retry in next interval.", e);
    // next scheduler tick retries automatically
}

When it happens

Trigger: Any exception thrown from scanOnce — filesystem connectivity loss, permission errors on source/target paths, transient HDFS/S3 outages, path renamed/deleted between existence check and read — while a scheduler tick fires.

Common situations: S3/HDFS temporarily unreachable during continuous discovery; source directory permissions changed; flaky network between the SeaTunnel worker and the file system.

Related errors

• Post-sync operation failed and will be retried: action={}, s
• Continuous discovery scheduler does not terminate in 5 secon
• Failed to evaluate recovered split {}, re-enqueue it conserv
• Post-sync delete: rename-to-trash failed, will retry: source
• Circular condition chain detected: '%s' already exists in th

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/9fea6548c27ab7f6. Report an issue: GitHub.