apache/seatunnel · warning

Discarding pending multiline line(s) for ( )

Error message

Discarding {} pending multiline line(s) for {} ({})

What it means

clearFileState removes a file's tracking state (multiline assembler, line counters). If a MultilineAssembler still holds unfinished partial multiline records, those lines are flushed and discarded with a warning stating how many lines were dropped and why. This prevents orphaned partial records from leaking after the file is evicted.

Solutions

1. Check the warn message: it reports the count of discarded lines, the file, and the cleanup reason.
2. Tune multiline settings (pattern, timeout) so records complete before files are rotated or closed.
3. Recover the data from the original log file if the discarded lines matter.
4. Fix the underlying read error (the 'reason' field) to avoid repeated forced cleanups.

Example fix

// before: multiline pattern too strict, records never close
multiline {
  pattern = "^\\s+at "
}
// after: add timeout so partial records flush before eviction
multiline {
  pattern = "^\\s+at "
  timeout = 5000
}

// Wrap reads so pending multiline data can be preserved or accounted for
try {
    reader.readLines();
} catch (Exception e) {
    LOG.warn("read failed; expect pending multiline discard warning if assembler held data", e);
    // optionally flush assembler contents to a dead-letter store before clearFileState runs
}

When it happens

Trigger: Called from readLines, handleReadException, or closeInactiveCursors: a read error occurs on a file, a cursor goes inactive (file deleted/rotated/idle timeout), or an exception during line reading forces state cleanup while a multiline record was mid-assembly.

Common situations: Log rotation cutting a multiline stack trace in half, files removed while a stack trace was still being assembled, or repeated read exceptions (permissions, truncation) discarding buffered lines.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• delete transaction directory
• Failed to clean empty transaction parent directory
• Failed to clean empty transaction parent directory
• CLOSE_CONNECTION_FAILED
• close easysearch connection error

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/cfe7fee8236e835b. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-edge-agent/seatunnel-edge-agent-connector/src/main/java/org/apache/seatunnel/edge/agent/connector/file/FileCollectReader.java:302

            FileTailCursor cursor = entry.getValue();
            if (nowMs - cursor.lastActivityMs() > config.getCloseInactiveMs()) {
                LOG.debug("Closing inactive cursor: {}", filePath);
                clearFileState(filePath, "inactive timeout");
                try {
                    cursor.close();
                } catch (Exception ignored) {
                }
                globResolver.forget(filePath);
                it.remove();
            }
        }
    }

    private void clearFileState(Path filePath, String reason) {
        MultilineAssembler assembler = multilineAssemblers.remove(filePath);
        if (assembler != null && assembler.hasPending()) {
            List<MultilineAssembler.LineElement> discarded = assembler.flush();
            LOG.warn(
                    "Discarding {} pending multiline line(s) for {} ({})",
                    discarded.size(),
                    filePath,
                    reason);
        }
        lineCounters.remove(filePath);
    }

    private MultilineAssembler getOrCreateAssembler(Path filePath) {
        return multilineAssemblers.computeIfAbsent(
                filePath,
                k ->
                        new MultilineAssembler(
                                config.getMultilinePattern(),
                                multilineMatchMode,
                                config.isMultilineNegate(),
                                config.getMultilineMaxLines()));
    }

View on GitHub (pinned to cf67b549a7)