What causes "Failed to close allocator: {}" in apache/seatunnel?

Ensure every ValueVector/batch allocated from the allocator is released before close

Failed to close allocator

Error message

Failed to close allocator: {}

What it means

LanceSinkWriter.close also closes the Arrow BufferAllocator. If allocator.close() throws (typically because buffers allocated from it were not released), the writer logs this warning and nulls the reference. This usually signals a memory leak of Arrow buffers.

Solutions

Ensure every ValueVector/batch allocated from the allocator is released before close
Check for exception paths in write/flushBatch that skip buffer release; wrap them in try/finally with release
Look at the logged message to identify the leaked reference count
Use Arrow's BaseAllocator DEBUG mode to find the leak stack trace

Example fix

// before
VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
root.clear(); // missing release of underlying buffers in some path
// after
VectorSchemaRoot root = VectorSchemaRoot.create(schema, allocator);
try {
    // write data
} finally {
    root.clear();
}
allocator.close();

Defensive patterns

Strategy: try-catch

Validate before calling

// check allocator state before close
if (allocator.getAllocatedMemory() > 0) System.out.println("Warning: un-released Arrow memory before close");

Try / catch

try { writer.close(); } catch (Exception e) { log.warn("Arrow allocator leak suspected: {}", e.getMessage()); }

Prevention

Release every ValueVector/root allocated from the writer's allocator
Use try/finally around buffer usage
Enable Arrow allocator DEBUG accounting in dev to catch leaks early

When it happens

Trigger: Closing the sink writer while some Arrow ValueVector/ByteBuf allocated from the allocator was never released, e.g. an exception path skipped release, or allocator.close invoked while batches are still referenced.

Common situations: Job failure mid-flush leaving vectors un-released; custom code retaining buffers after writer.close(); Arrow reference-counting bugs in long-running pipelines.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/4f03d7f493ce4afe. Report an issue: GitHub.

Appendix: source

Thrown at seatunnel-connectors-v2/connector-lance/src/main/java/org/apache/seatunnel/connectors/seatunnel/lance/sink/LanceSinkWriter.java:212

    @Override
    public void close() throws IOException {
        try {
            flushBatch();
        } finally {
            if (dataset != null) {
                try {
                    dataset.close();
                } catch (Exception e) {
                    log.warn("Failed to close dataset: {}", e.getMessage());
                }
                dataset = null;
            }

            if (allocator != null) {
                try {
                    allocator.close();
                } catch (Exception e) {
                    log.warn("Failed to close allocator: {}", e.getMessage());
                }
                allocator = null;
            }
        }
    }
}

View on GitHub (pinned to cf67b549a7)