{"record":{"id":"cfe7fee8236e835b","repo":"apache/seatunnel","slug":"discarding-pending-multiline-line-s-for","errorCode":null,"errorMessage":"Discarding {} pending multiline line(s) for {} ({})","messagePattern":"Discarding (.+?) pending multiline line\\(s\\) for (.+?) \\((.+?)\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"seatunnel-edge-agent/seatunnel-edge-agent-connector/src/main/java/org/apache/seatunnel/edge/agent/connector/file/FileCollectReader.java","lineNumber":302,"sourceCode":"            FileTailCursor cursor = entry.getValue();\n            if (nowMs - cursor.lastActivityMs() > config.getCloseInactiveMs()) {\n                LOG.debug(\"Closing inactive cursor: {}\", filePath);\n                clearFileState(filePath, \"inactive timeout\");\n                try {\n                    cursor.close();\n                } catch (Exception ignored) {\n                }\n                globResolver.forget(filePath);\n                it.remove();\n            }\n        }\n    }\n\n    private void clearFileState(Path filePath, String reason) {\n        MultilineAssembler assembler = multilineAssemblers.remove(filePath);\n        if (assembler != null && assembler.hasPending()) {\n            List<MultilineAssembler.LineElement> discarded = assembler.flush();\n            LOG.warn(\n                    \"Discarding {} pending multiline line(s) for {} ({})\",\n                    discarded.size(),\n                    filePath,\n                    reason);\n        }\n        lineCounters.remove(filePath);\n    }\n\n    private MultilineAssembler getOrCreateAssembler(Path filePath) {\n        return multilineAssemblers.computeIfAbsent(\n                filePath,\n                k ->\n                        new MultilineAssembler(\n                                config.getMultilinePattern(),\n                                multilineMatchMode,\n                                config.isMultilineNegate(),\n                                config.getMultilineMaxLines()));\n    }","sourceCodeStart":284,"sourceCodeEnd":320,"githubUrl":"https://github.com/apache/seatunnel/blob/cf67b549a7a6c35fa0beb12d83c62892427ea919/seatunnel-edge-agent/seatunnel-edge-agent-connector/src/main/java/org/apache/seatunnel/edge/agent/connector/file/FileCollectReader.java#L284-L320","documentation":"clearFileState removes a file's tracking state (multiline assembler, line counters). If a MultilineAssembler still holds unfinished partial multiline records, those lines are flushed and discarded with a warning stating how many lines were dropped and why. This prevents orphaned partial records from leaking after the file is evicted.","triggerScenarios":"Called from readLines, handleReadException, or closeInactiveCursors: a read error occurs on a file, a cursor goes inactive (file deleted/rotated/idle timeout), or an exception during line reading forces state cleanup while a multiline record was mid-assembly.","commonSituations":"Log rotation cutting a multiline stack trace in half, files removed while a stack trace was still being assembled, or repeated read exceptions (permissions, truncation) discarding buffered lines.","solutions":["Check the warn message: it reports the count of discarded lines, the file, and the cleanup reason.","Tune multiline settings (pattern, timeout) so records complete before files are rotated or closed.","Recover the data from the original log file if the discarded lines matter.","Fix the underlying read error (the 'reason' field) to avoid repeated forced cleanups."],"exampleFix":"// before: multiline pattern too strict, records never close\nmultiline {\n  pattern = \"^\\\\s+at \"\n}\n// after: add timeout so partial records flush before eviction\nmultiline {\n  pattern = \"^\\\\s+at \"\n  timeout = 5000\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Wrap reads so pending multiline data can be preserved or accounted for\ntry {\n    reader.readLines();\n} catch (Exception e) {\n    LOG.warn(\"read failed; expect pending multiline discard warning if assembler held data\", e);\n    // optionally flush assembler contents to a dead-letter store before clearFileState runs\n}","preventionTips":["Configure multiline timeout so partial records flush before files go inactive.","Match the multiline pattern against real log formats before production.","Rotate logs gently (copytruncate alternatives) to avoid splitting multiline records.","Treat the discarded-lines count in the warning as a data-loss metric to monitor."],"tags":["file","multiline","data-loss","cleanup"],"backgroundTag":"internal-invariant-violation","analyzedSha":"cf67b549a7a6c35fa0beb12d83c62892427ea919","analyzedAt":"2026-09-10T21:44:55.265Z","contentChangedAt":"2026-09-10T21:44:55.265Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}