{"record":{"id":"a509e2f99c2d45b2","repo":"alibaba/DataX","slug":"writing-records-to-doris-failed","errorCode":null,"errorMessage":"Writing records to Doris failed.","messagePattern":"Writing records to Doris failed\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisWriterManager.java","lineNumber":83,"sourceCode":"            scheduledFuture.cancel(false);\n            this.scheduler.shutdown();\n        }\n    }\n\n    public final synchronized void writeRecord(String record) throws IOException {\n        checkFlushException();\n        try {\n            byte[] bts = record.getBytes(StandardCharsets.UTF_8);\n            buffer.add(bts);\n            batchCount++;\n            batchSize += bts.length;\n            if (batchCount >= options.getBatchRows() || batchSize >= options.getBatchSize()) {\n                String label = createBatchLabel();\n                LOG.debug(String.format(\"Doris buffer Sinking triggered: rows[%d] label[%s].\", batchCount, label));\n                flush(label, false);\n            }\n        } catch (Exception e) {\n            throw new IOException(\"Writing records to Doris failed.\", e);\n        }\n    }\n\n    public synchronized void flush(String label, boolean waitUtilDone) throws Exception {\n        checkFlushException();\n        if (batchCount == 0) {\n            if (waitUtilDone) {\n                waitAsyncFlushingDone();\n            }\n            return;\n        }\n        flushQueue.put(new WriterTuple (label, batchSize,  new ArrayList<>(buffer)));\n        if (waitUtilDone) {\n            // wait the last flush\n            waitAsyncFlushingDone();\n        }\n        buffer.clear();\n        batchCount = 0;","sourceCodeStart":65,"sourceCodeEnd":101,"githubUrl":"https://github.com/alibaba/DataX/blob/80ec23d5c5328eb90ca364d2749e92dfaf44541e/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisWriterManager.java#L65-L101","documentation":"DorisWriterManager.write() is the per-record hot path: it serializes the record to UTF-8 bytes, appends to the buffer, and flushes when batchCount >= batchRows or batchSize >= batchSize. Any exception in that path (record.getBytes, buffer add, or the triggered flush enqueuing to flushQueue) is wrapped in this generic IOException.","triggerScenarios":"A Record whose getBytes(UTF_8) throws (unsupported/null column conversion), a flush() inside write() that throws (flushQueue full, checkFlushException firing because the async writer previously failed), or buffer.add failing. The original exception is attached as the cause 'e'.","commonSituations":"A prior async flush failed and checkFlushException() rethrows on the next write — so this message often masks the real DorisStreamLoadObserver error as its cause; dirty transform output (unexpected column type) reaching the writer; flushQueue backpressure when Doris is slow.","solutions":["Read the cause chain: the 'e' in 'Writing records to Doris failed., e' holds the real failure (usually a stream-load error from the observer)","Fix the root stream-load issue (FE connectivity, schema, auth) — this wrapper disappears once flushes succeed","Validate transform output column types match the writer's column list before starting the job","Increase batchSize/batchRows or maxRetries if the cause is transient backpressure"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    manager.write(record);\n} catch (IOException e) {\n    Throwable root = com.google.common.base.Throwables.getRootCause(e);\n    log.error(\"write failed, root cause: {}\", root.getMessage(), root);\n    // root is the real stream-load / conversion error; act on it, not on the wrapper\n    throw e;\n}","preventionTips":["Always inspect the cause chain — this wrapper is never the root problem","Pre-validate record column types between reader and writer configuration","Watch for the first Doris error in logs; subsequent write() failures are just checkFlushException echoes"],"tags":["doris","datax","wrapper-exception","buffering"],"backgroundTag":null,"analyzedSha":"80ec23d5c5328eb90ca364d2749e92dfaf44541e","analyzedAt":"2026-08-14T15:33:51.187Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}