What causes "Python source reader has already been closed" in apache/seatunnel?

Create a new PythonSourceReader instance instead of re-closing/opening the old one

apache/seatunnel · error · IOException

Python source reader has already been closed

Error message

Python source reader has already been closed

What it means

Thrown by PythonSourceReader.open() when the reader was already closed (closeRequested flag set) and open() is called afterwards. The lifecycle lock rejects reopening a closed reader because its process handle and buffers are torn down.

Source

Thrown at seatunnel-connectors-v2/connector-python/src/main/java/org/apache/seatunnel/connectors/seatunnel/python/source/PythonSourceReader.java:126

    private boolean closeComplete;

    public PythonSourceReader(
            PythonSourceConfig sourceConfig,
            CatalogTable catalogTable,
            SingleSplitReaderContext readerContext) {
        this.sourceConfig = sourceConfig;
        this.catalogTable = catalogTable;
        this.readerContext = readerContext;
        this.deserializationSchema = createDeserializationSchema(sourceConfig, catalogTable);
        this.recentStderrLines = new ArrayDeque<>(STDERR_HISTORY_LIMIT);
        this.stdoutLines = new ArrayBlockingQueue<>(STDOUT_QUEUE_CAPACITY);
    }

    @Override
    public void open() throws Exception {
        synchronized (lifecycleLock) {
            if (closeRequested) {
                throw new IOException("Python source reader has already been closed");
            }
        }
        Path scriptPath = validateScriptPath().toAbsolutePath().normalize();
        Path resolvedExecutable =
                PythonSourceExecutionPolicy.resolveExecutable(sourceConfig.getPythonExecutable());
        LOG.warn(
                "Python source runs unsandboxed external code. Resolved executable='{}', scriptOrigin='python.script.path={}', guarded by system properties '{}','{}'.",
                resolvedExecutable,
                scriptPath,
                PythonSourceExecutionPolicy.PYTHON_SOURCE_ENABLED_PROPERTY,
                PythonSourceExecutionPolicy.PYTHON_ALLOWED_EXECUTABLES_PROPERTY);
        ProcessBuilder processBuilder =
                new ProcessBuilder(resolvedExecutable.toString(), scriptPath.toString());
        configureWorkingDirectory(processBuilder, scriptPath);

        synchronized (lifecycleLock) {
            if (closeRequested) {
                throw new IOException("Python source reader has already been closed");

View on GitHub (pinned to cf67b549a7)

Solutions

Create a new PythonSourceReader instance instead of re-closing/opening the old one
Ensure close() is only called at final teardown, never between open attempts
In retry logic, rebuild the reader from source config rather than reusing it

Example fix

// before
reader.close();
reader.open(); // throws
// after
reader.close();
reader = new PythonSourceReader(...);
reader.open();

Defensive patterns

Strategy: try-catch

Validate before calling

if (reader.isClosed()) { reader = createNewReader(config); }

Try / catch

try {
    reader.open();
} catch (java.io.IOException e) {
    if (e.getMessage().contains("already been closed")) {
        reader = createNewReader(config);
        reader.open();
    } else throw e;
}

Prevention

Treat readers as single-use: one open per instance
Never call open() after close(); build a fresh reader instead
Guard retry logic to recreate the reader

When it happens

Trigger: Calling open() after close() on the same reader instance; engine/task lifecycle bugs that retry open after a close; test code reusing a reader fixture across cases.

Common situations: Custom integration code that restarts a reader by calling open() again; task retry logic recreating state incorrectly; test teardown order issues.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/6bfacec1fb9ea6a6. Report an issue: GitHub.