apache/flink · error · TableException

Exception in close

Error message

Exception in close

What it means

FileSystemOutputFormat.close closes the internal PartitionWriter if it is not null. Any exception during writer close (finalizing part files, flushing buffers, renaming temp files to final names) is caught and re-thrown as a TableException with message 'Exception in close'. This is the finalization step for each subtask.

Source

Thrown at flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemOutputFormat.java:217

    }

    @Override
    public void writeRecord(T record) {
        try {
            writer.write(record);
        } catch (Exception e) {
            throw new TableException("Exception in writeRecord", e);
        }
    }

    @Override
    public void close() throws IOException {
        try {
            if (writer != null) {
                writer.close();
            }
        } catch (Exception e) {
            throw new TableException("Exception in close", e);
        }
    }

    /** Builder to build {@link FileSystemOutputFormat}. */
    public static class Builder<T> {

        private String[] partitionColumns;
        private OutputFormatFactory<T> formatFactory;
        private TableMetaStoreFactory metaStoreFactory;
        private Path stagingPath;

        private LinkedHashMap<String, String> staticPartitions = new LinkedHashMap<>();
        private boolean dynamicGrouped = false;
        private boolean overwrite = false;
        private boolean isToLocal = false;
        private FileSystemFactory fileSystemFactory = FileSystem::get;

        private PartitionComputer<T> computer;

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Examine the chained cause of the TableException to find the specific close failure.
2. Verify file system health and that the output directory remains writable throughout the job lifecycle.
3. Check for disk space during the close phase (buffer flush + file finalization).
4. If using S3, be aware that rename operations may be copy-and-delete; ensure no throttling or timeout.

try {
    outputFormat.close();
} catch (TableException e) {
    Throwable root = e.getCause();
    // Close failure may leave partial files; investigate root cause
    LOG.error("Output format close failed", root);
    throw e;
}

When it happens

Trigger: The PartitionWriter.close call fails: file system rename of part files from temporary to final names fails, final flush of buffered data encounters I/O error, or the format writer fails to finalize (e.g. Parquet footer write failure).

Common situations: File system rename failure (cross-filesystem rename on S3, HDFS safe mode). Parquet/ORC footer write failure due to corrupt internal state or I/O error. Output directory permissions changed mid-job. Disk full during final flush.

Related errors

• Exception in open
• Exception in finalizeGlobal
• Exception in writeRecord
• An IO error occurred while accessing the staging FileSystem.
• Can not find format factory.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/7f5da3db44d31e67. Report an issue: GitHub.