apache/flink · error · TableException

Exception in writeRecord

Error message

Exception in writeRecord

What it means

FileSystemOutputFormat.writeRecord delegates to the internal PartitionWriter.write. Any exception during record serialization, partition computation, or file I/O within the writer is caught and re-thrown as a TableException with message 'Exception in writeRecord'. This is the per-record write path, so failures here typically indicate data or serialization issues.

Source

Thrown at flink-connectors/flink-connector-files/src/main/java/org/apache/flink/connector/file/table/FileSystemOutputFormat.java:206

                                    partitionColumns.length - staticPartitions.size() > 0,
                                    dynamicGrouped,
                                    staticPartitions)
                            .create(
                                    writerContext,
                                    fileManager,
                                    computer,
                                    new PartitionWriter.DefaultPartitionWriterListener());
        } catch (Exception e) {
            throw new TableException("Exception in open", e);
        }
    }

    @Override
    public void writeRecord(T record) {
        try {
            writer.write(record);
        } catch (Exception e) {
            throw new TableException("Exception in writeRecord", e);
        }
    }

    @Override
    public void close() throws IOException {
        try {
            if (writer != null) {
                writer.close();
            }
        } catch (Exception e) {
            throw new TableException("Exception in close", e);
        }
    }

    /** Builder to build {@link FileSystemOutputFormat}. */
    public static class Builder<T> {

        private String[] partitionColumns;

View on GitHub (pinned to 2f3c205e92)

Solutions

  1. Examine the chained cause of the TableException to identify the serialization or I/O failure.
  2. Validate input data types match the table schema; check for nulls in partition columns.
  3. If the cause is file system I/O, check disk space, HDFS/S3 health, and write quotas.
  4. If serialization-specific, ensure the format writer can handle the record type.
Defensive patterns

Strategy: try-catch

Try / catch

try {
    outputFormat.writeRecord(record);
} catch (TableException e) {
    Throwable root = e.getCause();
    // Log the failing record for debugging
    LOG.error("Failed to write record: {}", record, root);
    throw e;
}

Prevention

When it happens

Trigger: The PartitionWriter.write call fails: record serialization fails (format-specific encoding error), the partition value computation throws (e.g. null partition key with no default handling), or a file system write error occurs on the current part file.

Common situations: Data type mismatch between the table schema and the incoming record. Serialization failure (e.g. invalid UTF-8 in CSV, schema evolution in Parquet). File system write failure (disk full, HDFS block size exceeded, S3 throttling). Null value in a partition column that triggers a NullPointerException in the partition computer.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/809870db7f4d402a. Report an issue: GitHub.