alibaba/DataX · error · IOException

Writing records to Doris failed.

Error message

Writing records to Doris failed.

What it means

DorisWriterManager.write() is the per-record hot path: it serializes the record to UTF-8 bytes, appends to the buffer, and flushes when batchCount >= batchRows or batchSize >= batchSize. Any exception in that path (record.getBytes, buffer add, or the triggered flush enqueuing to flushQueue) is wrapped in this generic IOException.

Source

Thrown at doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisWriterManager.java:83

            scheduledFuture.cancel(false);
            this.scheduler.shutdown();
        }
    }

    public final synchronized void writeRecord(String record) throws IOException {
        checkFlushException();
        try {
            byte[] bts = record.getBytes(StandardCharsets.UTF_8);
            buffer.add(bts);
            batchCount++;
            batchSize += bts.length;
            if (batchCount >= options.getBatchRows() || batchSize >= options.getBatchSize()) {
                String label = createBatchLabel();
                LOG.debug(String.format("Doris buffer Sinking triggered: rows[%d] label[%s].", batchCount, label));
                flush(label, false);
            }
        } catch (Exception e) {
            throw new IOException("Writing records to Doris failed.", e);
        }
    }

    public synchronized void flush(String label, boolean waitUtilDone) throws Exception {
        checkFlushException();
        if (batchCount == 0) {
            if (waitUtilDone) {
                waitAsyncFlushingDone();
            }
            return;
        }
        flushQueue.put(new WriterTuple (label, batchSize,  new ArrayList<>(buffer)));
        if (waitUtilDone) {
            // wait the last flush
            waitAsyncFlushingDone();
        }
        buffer.clear();
        batchCount = 0;

View on GitHub (pinned to 80ec23d5c5)

Solutions

1. Read the cause chain: the 'e' in 'Writing records to Doris failed., e' holds the real failure (usually a stream-load error from the observer)
2. Fix the root stream-load issue (FE connectivity, schema, auth) — this wrapper disappears once flushes succeed
3. Validate transform output column types match the writer's column list before starting the job
4. Increase batchSize/batchRows or maxRetries if the cause is transient backpressure

try {
    manager.write(record);
} catch (IOException e) {
    Throwable root = com.google.common.base.Throwables.getRootCause(e);
    log.error("write failed, root cause: {}", root.getMessage(), root);
    // root is the real stream-load / conversion error; act on it, not on the wrapper
    throw e;
}

When it happens

Trigger: A Record whose getBytes(UTF_8) throws (unsupported/null column conversion), a flush() inside write() that throws (flushQueue full, checkFlushException firing because the async writer previously failed), or buffer.add failing. The original exception is attached as the cause 'e'.

Common situations: A prior async flush failed and checkFlushException() rethrows on the next write — so this message often masks the real DorisStreamLoadObserver error as its cause; dirty transform output (unexpected column type) reaching the writer; flushQueue backpressure when Doris is slow.

Related errors

• Failed to parse delimiter: `Hex str is empty`
• Failed to parse delimiter: `Hex str length error`
• Failed to parse delimiter: `Hex str format error`
• Failed to create row serializer, unsupported `format` from s
• load_url cannot be empty, or the host cannot connect.Please

AI-assisted analysis of alibaba/DataX@80ec23d5c5 (2026-08-14). Data as JSON: /api/errors/a509e2f99c2d45b2. Report an issue: GitHub.