{"record":{"id":"9fea6548c27ab7f6","repo":"apache/seatunnel","slug":"continuous-discovery-scan-failed-will-retry-in-ne","errorCode":null,"errorMessage":"Continuous discovery scan failed, will retry in next interval.","messagePattern":"Continuous discovery scan failed, will retry in next interval\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java","lineNumber":402,"sourceCode":"                return false;\n            }\n            inFlightSplitContexts.remove(splitId);\n            inFlightSplits.removeIf(s -> Objects.equals(s.splitId(), splitId));\n            if (operationState != null) {\n                finishedAwaitingCheckpoint.add(operationState);\n            }\n            return true;\n        }\n    }\n\n    private void safeScanOnce() {\n        if (closed) {\n            return;\n        }\n        try {\n            scanOnce();\n        } catch (Exception e) {\n            log.warn(\"Continuous discovery scan failed, will retry in next interval.\", e);\n        }\n    }\n\n    @VisibleForTesting\n    void scanOnceForTest() throws IOException {\n        scanOnce();\n    }\n\n    private void scanOnce() throws IOException {\n        int scanned = 0;\n        int queued = 0;\n        Set<String> activeKnownSplitIds = new HashSet<>();\n        for (TableScanContext ctx : tableScanContexts) {\n            List<FileStatus> files = ctx.listFiles(ctx.rootPath);\n            scanned += files.size();\n            for (FileStatus fileStatus : files) {\n                if (!ctx.shouldProcess(fileStatus, jobStartTimeMillis, startMode)) {\n                    clearKnownVersionIfPresent(ctx.tableId, fileStatus.getPath().toString());","sourceCodeStart":384,"sourceCodeEnd":420,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-connectors-v2/connector-file/connector-file-base/src/main/java/org/apache/seatunnel/connectors/seatunnel/file/source/split/ContinuousMultipleTableFileSourceSplitEnumerator.java#L384-L420","documentation":"safeScanOnce wraps the periodic scanOnce() call: if scanOnce throws any Exception (e.g. an IOException from filesystem listing/reading), it logs this warning with the exception and returns, letting the next scheduler interval retry. The continuous discovery loop is designed to be resilient; one failed scan does not fail the job.","triggerScenarios":"Any exception thrown from scanOnce — filesystem connectivity loss, permission errors on source/target paths, transient HDFS/S3 outages, path renamed/deleted between existence check and read — while a scheduler tick fires.","commonSituations":"S3/HDFS temporarily unreachable during continuous discovery; source directory permissions changed; flaky network between the SeaTunnel worker and the file system.","solutions":["Inspect the logged cause for the underlying filesystem error and address it (network, credentials, permissions)","Verify the source and target filesystems are reachable and paths exist with correct permissions","If failures persist every interval, fix the persistent filesystem issue; one-off occurrences can be ignored since the next tick retries"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Pre-check reachability of source/target paths before scheduling scans\nfor (String p : java.util.Arrays.asList(sourcePath, targetPath)) {\n    if (!fs.exists(new org.apache.hadoop.fs.Path(p))) {\n        log.warn(\"Path not reachable before scan: {}\", p);\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    scanOnce();\n} catch (Exception e) {\n    log.warn(\"Continuous discovery scan failed, will retry in next interval.\", e);\n    // next scheduler tick retries automatically\n}","preventionTips":["Monitor filesystem/network health between workers and HDFS/S3","Verify read permissions on source directories before job start","Treat isolated warnings as benign; investigate only when every interval fails"],"tags":["file-connector","filesystem","retry","scheduler"],"backgroundTag":"network-request-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}