{"record":{"id":"f14f8d0d7a02ac99","repo":"apache/seatunnel","slug":"file-discovery-failed","errorCode":null,"errorMessage":"File discovery failed","messagePattern":"File discovery failed","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"seatunnel-edge-agent/seatunnel-edge-agent-connector/src/main/java/org/apache/seatunnel/edge/agent/connector/file/FileCollectReader.java","lineNumber":274,"sourceCode":"        try {\n            List<Path> newFiles = globResolver.resolveNew();\n            for (Path file : newFiles) {\n                if (activeCursors.containsKey(file)) {\n                    continue;\n                }\n                String pathStr = file.toAbsolutePath().toString();\n                EdgeSourcePosition pos = resolvePosition(pathStr);\n\n                FileTailCursor cursor = openCursor(file, pos);\n                activeCursors.put(file, cursor);\n                lineCounters.put(file, restoredLineNumber(pos));\n                LOG.info(\"Discovered new file: {}\", file);\n            }\n        } catch (Exception e) {\n            if (config.isSkipOnError()) {\n                LOG.warn(\"Error during file discovery\", e);\n            } else {\n                throw new RuntimeException(\"File discovery failed\", e);\n            }\n        }\n    }\n\n    private void closeInactiveCursors(long nowMs) {\n        Iterator<Map.Entry<Path, FileTailCursor>> it = activeCursors.entrySet().iterator();\n        while (it.hasNext()) {\n            Map.Entry<Path, FileTailCursor> entry = it.next();\n            Path filePath = entry.getKey();\n            FileTailCursor cursor = entry.getValue();\n            if (nowMs - cursor.lastActivityMs() > config.getCloseInactiveMs()) {\n                LOG.debug(\"Closing inactive cursor: {}\", filePath);\n                clearFileState(filePath, \"inactive timeout\");\n                try {\n                    cursor.close();\n                } catch (Exception ignored) {\n                }\n                globResolver.forget(filePath);","sourceCodeStart":256,"sourceCodeEnd":292,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-edge-agent/seatunnel-edge-agent-connector/src/main/java/org/apache/seatunnel/edge/agent/connector/file/FileCollectReader.java#L256-L292","documentation":"FileCollectReader.discoverNewFiles scans the configured glob for new files inside poll(). If discovery throws and the input's on-error policy is \"fail\" (i.e. not skip-on-error), it wraps the cause in a RuntimeException(\"File discovery failed\") to propagate the failure to the reader loop.","triggerScenarios":"A glob pointing to a deleted or permission-denied directory, filesystem I/O errors during listing, or any exception thrown by the path matcher while poll() is running.","commonSituations":"Network mounts going offline, removable media unmounted, directories removed while the agent runs, or glob patterns with invalid syntax.","solutions":["Inspect the wrapped cause exception for the underlying filesystem error","Set input.on-error = \"skip\" to log and continue instead of failing","Verify the glob path exists and the agent user has read permission","Reconnect/remount the storage the glob points to"],"exampleFix":"# before\ninput.on-error = \"fail\"\n# after\ninput.on-error = \"skip\"  # tolerate transient discovery errors","handlingStrategy":"try-catch","validationCode":"Path dir = Paths.get(globDir);\nif (!Files.isDirectory(dir)) throw new IllegalStateException(\"glob directory missing: \" + dir);\nFiles.isDirectory(dir) && Files.isReadable(dir); // probe access before starting the reader","typeGuard":null,"tryCatchPattern":"try { reader.poll(); } catch (RuntimeException e) { if (\"File discovery failed\".equals(e.getMessage())) { log.error(\"discovery failed\", e.getCause()); /* remediate path/permissions or restart */ } else { throw e; } }","preventionTips":["Pre-deploy check that the glob base directory exists and is readable","Prefer on-error = \"skip\" for inputs on unreliable mounts","Monitor agent logs for 'Error during file discovery' warnings before they escalate","Avoid globs on removable/network storage without remount handling"],"tags":["filesystem","io","runtime"],"backgroundTag":"file-read-failed","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}