{"record":{"id":"8bc65436143fc053","repo":"alibaba/DataX","slug":"failed-to-join-rows-data-unsupported-format-fro","errorCode":null,"errorMessage":"Failed to join rows data, unsupported `format` from stream load properties:","messagePattern":"Failed to join rows data, unsupported `format` from stream load properties:","errorType":"validation","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisStreamLoadObserver.java","lineNumber":153,"sourceCode":"            return bos.array();\n        }\n\n        if (Keys.StreamLoadFormat.JSON.equals(options.getStreamLoadFormat())) {\n            ByteBuffer bos = ByteBuffer.allocate(totalBytes + (rows.isEmpty() ? 2 : rows.size() + 1));\n            bos.put(\"[\".getBytes(StandardCharsets.UTF_8));\n            byte[] jsonDelimiter = \",\".getBytes(StandardCharsets.UTF_8);\n            boolean isFirstElement = true;\n            for (byte[] row : rows) {\n                if (!isFirstElement) {\n                    bos.put(jsonDelimiter);\n                }\n                bos.put(row);\n                isFirstElement = false;\n            }\n            bos.put(\"]\".getBytes(StandardCharsets.UTF_8));\n            return bos.array();\n        }\n        throw new RuntimeException(\"Failed to join rows data, unsupported `format` from stream load properties:\");\n    }\n    private Map<String, Object> put(String loadUrl, String label, byte[] data) throws IOException {\n        LOG.info(String.format(\"Executing stream load to: '%s', size: '%s'\", loadUrl, data.length));\n        final HttpClientBuilder httpClientBuilder = HttpClients.custom()\n                .setRedirectStrategy(new DefaultRedirectStrategy () {\n                    @Override\n                    protected boolean isRedirectable(String method) {\n                        return true;\n                    }\n                });\n        try ( CloseableHttpClient httpclient = httpClientBuilder.build()) {\n            HttpPut httpPut = new HttpPut(loadUrl);\n            httpPut.removeHeaders(HttpHeaders.CONTENT_LENGTH);\n            httpPut.removeHeaders(HttpHeaders.TRANSFER_ENCODING);\n            List<String> cols = options.getColumns();\n            if (null != cols && !cols.isEmpty() && Keys.StreamLoadFormat.CSV.equals(options.getStreamLoadFormat())) {\n                httpPut.setHeader(\"columns\", String.join(\",\", cols.stream().map(f -> String.format(\"`%s`\", f)).collect(Collectors.toList())));\n            }","sourceCodeStart":135,"sourceCodeEnd":171,"githubUrl":"https://github.com/alibaba/DataX/blob/80ec23d5c5328eb90ca364d2749e92dfaf44541e/doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisStreamLoadObserver.java#L135-L171","documentation":"DorisStreamLoadObserver.addRows() only knows how to join buffered rows for two formats: CSV (joins with line_delimiter) and JSON (wraps rows into a JSON array with ',' separators). If options.getStreamLoadFormat() is anything else, this RuntimeException is thrown. Note the message does not interpolate the offending format — the '%s'-less string ends with a colon, so the bad value is not printed.","triggerScenarios":"Setting \"format\" under loadProps (or the streamLoadFormat option) to a value other than 'csv' or 'json' — e.g. 'json_lines', 'arrow', or a typo like 'CSV ' with trailing whitespace. addRows() falls through both if-branches and hits the unconditional throw.","commonSituations":"Copy-pasting a Doris stream-load curl example that uses a format DataX's doriswriter does not implement; assuming the plugin supports every format the Doris stream load itself supports; case/whitespace mismatch in config.","solutions":["Set the format to \"csv\" or \"json\" exactly (lowercase) in the doriswriter job configuration","For line-delimited JSON needs, use \"json\" format — the plugin wraps rows into a JSON array which Doris accepts","If you truly need another format, extend DorisStreamLoadObserver.addRows() with a branch for it and rebuild the plugin"],"exampleFix":"// before\n\"loadProps\": { \"format\": \"json_lines\" }\n// after\n\"loadProps\": { \"format\": \"json\" }","handlingStrategy":"validation","validationCode":"// validate format before the job starts\nString fmt = String.valueOf(loadProps.get(\"format\")).trim().toLowerCase(Locale.ROOT);\nif (!(fmt.equals(\"csv\") || fmt.equals(\"json\"))) {\n    throw new IllegalArgumentException(\"doriswriter supports only 'csv' or 'json', got: \" + loadProps.get(\"format\"));\n}","typeGuard":"boolean isSupportedFormat(String f) {\n    String s = f == null ? \"csv\" : f.trim().toLowerCase(Locale.ROOT);\n    return s.equals(\"csv\") || s.equals(\"json\");\n}","tryCatchPattern":null,"preventionTips":["Copy the format value from the doriswriter README, not from generic Doris stream-load docs","Add a job-startup assertion that format is csv/json before any data moves"],"tags":["doris","config","format","datax"],"backgroundTag":null,"analyzedSha":"80ec23d5c5328eb90ca364d2749e92dfaf44541e","analyzedAt":"2026-08-14T15:33:51.187Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}