apache/seatunnel · error · IllegalStateException

Failed to close auxiliary state store

Error message

Failed to close auxiliary state store

What it means

DefaultAuxiliaryStateStores.closeIfPossible closes an auxiliary state store via AutoCloseable.close(); if close() throws any Exception it wraps it in an IllegalStateException with the message 'Failed to close auxiliary state store'. This surfaces resource-cleanup failures (e.g., releasing file/network handles in the state store) instead of silently swallowing them.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/common/statestore/DefaultAuxiliaryStateStores.java:63

    @Override
    public CheckpointOverviewStateStore checkpointOverviewStateStore() {
        return checkpointOverviewStateStore;
    }

    @Override
    public void close() {
        closeIfPossible(checkpointOverviewStateStore);
        closeIfPossible(metricsSnapshotStore);
    }

    private void closeIfPossible(Object store) {
        if (!(store instanceof AutoCloseable)) {
            return;
        }
        try {
            ((AutoCloseable) store).close();
        } catch (Exception e) {
            throw new IllegalStateException("Failed to close auxiliary state store", e);
        }
    }
}

View on GitHub (pinned to cf67b549a7)

Solutions

1. Inspect the wrapped cause (getCause()) — it contains the actual close failure (IOException, etc.) from the underlying store
2. Check disk space, mounts, and file permissions for the state store directory
3. Ensure no other process holds the state files open (especially on Windows) before engine shutdown
4. Retry engine shutdown once the underlying storage is reachable; close failures are often transient storage issues
5. If a custom AuxiliaryStateStore is in use, make its close() idempotent and avoid throwing on already-closed resources

Example fix

// before
// custom store
public void close() throws Exception {
    outStream.flush(); // throws if disk full/handle closed
}

// after
@Override
public void close() throws Exception {
    try {
        outStream.flush();
    } finally {
        outStream.close();
    }
}

// pre-shutdown sanity: state dir writable and has free space
Files.isWritable(stateDir);
long usable = Files.getFileStore(stateDir).getUsableSpace();

try {
    auxiliaryStateStores.close();
} catch (IllegalStateException e) {
    LOG.error("Auxiliary state store close failed: {}", e.getCause(), e);
}

When it happens

Trigger: Calling close() on a DefaultAuxiliaryStateStores whose underlying store's AutoCloseable.close() throws — I/O errors while flushing/closing local state files, locked files, or network state-store clients failing to disconnect.

Common situations: State directory on a full/unmounted disk or NFS hang at shutdown; file handles still held by another process on Windows; state store backend (e.g., local file or object storage client) network failure during engine shutdown or job cleanup.

Related errors

• failed to close arrow stream reader.
• Error while closing Databend source reader
• Closing streamLoadHttpClient failed.
• Closing httpClient failed.
• Failed to close Neo4j source reader.

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/fba3a1020bf9c9f1. Report an issue: GitHub.