{"record":{"id":"afedc702173fa13a","repo":"alibaba/DataX","slug":"failed-to-flush-data-to-doris-s","errorCode":null,"errorMessage":"Failed to flush data to Doris.\n%s","messagePattern":"Failed to flush data to Doris\\.\n(.+?)","errorType":"http","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisStreamLoadObserver.java","lineNumber":72,"sourceCode":"            throw new IOException (\"load_url cannot be empty, or the host cannot connect.Please check your configuration.\");\n        }\n        String loadUrl = new StringBuilder(host)\n                .append(\"/api/\")\n                .append(options.getDatabase())\n                .append(\"/\")\n                .append(options.getTable())\n                .append(\"/_stream_load\")\n                .toString();\n        LOG.info(\"Start to join batch data: rows[{}] bytes[{}] label[{}].\", data.getRows().size(), data.getBytes(), data.getLabel());\n        Map<String, Object> loadResult = put(loadUrl, data.getLabel(), addRows(data.getRows(), data.getBytes().intValue()));\n        LOG.info(\"StreamLoad response :{}\",JSON.toJSONString(loadResult));\n        final String keyStatus = \"Status\";\n        if (null == loadResult || !loadResult.containsKey(keyStatus)) {\n            throw new IOException(\"Unable to flush data to Doris: unknown result status.\");\n        }\n        LOG.debug(\"StreamLoad response:{}\",JSON.toJSONString(loadResult));\n        if (RESULT_FAILED.equals(loadResult.get(keyStatus))) {\n            throw new IOException(\n                    new StringBuilder(\"Failed to flush data to Doris.\\n\").append(JSON.toJSONString(loadResult)).toString()\n            );\n        } else if (RESULT_LABEL_EXISTED.equals(loadResult.get(keyStatus))) {\n            LOG.debug(\"StreamLoad response:{}\",JSON.toJSONString(loadResult));\n            checkStreamLoadState(host, data.getLabel());\n        }\n    }\n\n    private void checkStreamLoadState(String host, String label) throws IOException {\n        int idx = 0;\n        while(true) {\n            try {\n                TimeUnit.SECONDS.sleep(Math.min(++idx, 5));\n            } catch (InterruptedException ex) {\n                break;\n            }\n            try (CloseableHttpClient httpclient = HttpClients.createDefault()) {\n                HttpGet httpGet = new HttpGet(new StringBuilder(host).append(\"/api/\").append(options.getDatabase()).append(\"/get_load_state?label=\").append(label).toString());","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/alibaba/DataX/blob/80ec23d5c5328eb90ca364d2749e92dfaf44541e/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisStreamLoadObserver.java#L54-L90","documentation":"DorisStreamLoadObserver.streamLoad throws this IOException when the stream load response's Status is 'Fail' — Doris explicitly rejected the load. The full response JSON (including the Status, Message, and error URL fields) is appended to the exception so the exact server-side failure reason travels with it.","triggerScenarios":"Any Doris-side load failure: quality errors exceeding max_filter_ratio (bad rows), schema mismatch (wrong column count/order versus the table), unsupported types, invalid CSV due to a separator present in the data, or table not found. The response body in the message pinpoints which.","commonSituations":"Data containing the column separator itself, column list in the writer config not matching the Doris table order, NULL handling in non-nullable columns, or exceeding the filter ratio with dirty source rows.","solutions":["Read the appended JSON in the exception — the 'Message' and 'ErrorURL' fields state the Doris-side reason; open the ErrorURL to see offending rows.","For quality errors, clean the data or raise max_filter_ratio in loadProps if acceptable.","Align the writer 'column' list (order and count) with the Doris table schema, or set the columns explicitly in loadProps.","If the separator appears in data, switch to a hex separator like \"\\\\x01\" or to JSON format."],"exampleFix":"// before: CSV data contains the ',' separator\n\"loadProps\": { \"format\": \"csv\", \"column_separator\": \",\" }\n// after\n\"loadProps\": { \"format\": \"csv\", \"column_separator\": \"\\\\x01\", \"max_filter_ratio\": \"0.01\" }","handlingStrategy":"fallback","validationCode":"// reduce row rejects before they reach Doris: choose a delimiter absent from data\nString sample = fetchSampleRows();\nfor (String cand : List.of(\"\\\\x01\", \"\\\\x1f\", \"\\\\x02\")) if (!sample.contains(cand)) { chosenSep = cand; break; }","typeGuard":null,"tryCatchPattern":"catch (IOException e) when message starts with 'Failed to flush data to Doris.': parse the appended JSON, read Status/Message/ErrorURL, fetch ErrorURL for offending rows, route them to a side output and re-flush the remainder.","preventionTips":["Use a control-character separator (\\\\x01) or JSON format when data may contain the delimiter.","Set a sane max_filter_ratio and monitor quality metrics so bad rows surface early.","Keep the writer 'column' list exactly aligned with the target table schema."],"tags":["datax","doris","writer","stream-load","data-quality"],"backgroundTag":null,"analyzedSha":"80ec23d5c5328eb90ca364d2749e92dfaf44541e","analyzedAt":"2026-08-14T15:33:51.187Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}