alibaba/DataX · error · IOException

Failed to flush data to Doris, Error could not get the final

Error message

Failed to flush data to Doris, Error could not get the final state of label[%s].

What it means

DorisStreamLoadObserver.checkStreamLoadState throws this IOException while polling a duplicate load label: the get_load_state HTTP call answered, but the response entity was null so no state could be parsed. It occurs after Doris reported the label already existed, while the observer retries with backoff (sleep capped at 5s per iteration) to learn whether that earlier load actually committed.

Source

Thrown at doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisStreamLoadObserver.java:97

    }

    private void checkStreamLoadState(String host, String label) throws IOException {
        int idx = 0;
        while(true) {
            try {
                TimeUnit.SECONDS.sleep(Math.min(++idx, 5));
            } catch (InterruptedException ex) {
                break;
            }
            try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
                HttpGet httpGet = new HttpGet(new StringBuilder(host).append("/api/").append(options.getDatabase()).append("/get_load_state?label=").append(label).toString());
                httpGet.setHeader("Authorization", getBasicAuthHeader(options.getUsername(), options.getPassword()));
                httpGet.setHeader("Connection", "close");

                try (CloseableHttpResponse resp = httpclient.execute(httpGet)) {
                    HttpEntity respEntity = getHttpEntity(resp);
                    if (respEntity == null) {
                        throw new IOException(String.format("Failed to flush data to Doris, Error " +
                                "could not get the final state of label[%s].\n", label), null);
                    }
                    Map<String, Object> result = (Map<String, Object>)JSON.parse(EntityUtils.toString(respEntity));
                    String labelState = (String)result.get("data");
                    if (null == labelState) {
                        throw new IOException(String.format("Failed to flush data to Doris, Error " +
                                "could not get the final state of label[%s]. response[%s]\n", label, EntityUtils.toString(respEntity)), null);
                    }
                    LOG.info(String.format("Checking label[%s] state[%s]\n", label, labelState));
                    switch(labelState) {
                        case LAEBL_STATE_VISIBLE:
                        case LAEBL_STATE_COMMITTED:
                            return;
                        case RESULT_LABEL_PREPARE:
                            continue;
                        case RESULT_LABEL_ABORTED:
                            throw new DorisWriterExcetion (String.format("Failed to flush data to Doris, Error " +
                                    "label[%s] state[%s]\n", label, labelState), null, true);

View on GitHub (pinned to 80ec23d5c5)

Solutions

1. Check FE health/stability at the time of the error (fe.audit/fe.log, leader elections).
2. Retry the DataX job — label state will resolve to VISIBLE/COMMITTED or the label will expire; ensure labelPrefix differs per retry if your wrapper reuses labels.
3. Ensure loadUrl points directly at FEs (not a body-stripping proxy) and the http_port is correct.
4. Verify in Doris (SHOW LOAD WHERE LABEL = '...') what final state the label reached before re-running.

Example fix

// before: retrying a job with identical labelPrefix can hit label-exists polling
"labelPrefix": "datax_job_20240101"
// after: unique prefix per run
"labelPrefix": "datax_job_20240101_run2"

catch (IOException e) { if (e.getMessage().contains("could not get the final state of label")) { wait briefly, then query SHOW LOAD WHERE LABEL = '<label>' and treat VISIBLE/COMMITTED as success before re-running; } }

When it happens

Trigger: A stream load response Status of 'Label Already Exists' (e.g. job retried after a timeout with the same labelPrefix) enters checkStreamLoadState; the subsequent GET /api/<db>/get_load_state?label=... returns 200 with a null entity, triggering this error. The FE being restarted mid-poll is a typical cause.

Common situations: Task-level retries reusing labels after network hiccups, FE leader re-election during the check, or a proxy stripping response bodies. Companion message with 'response[%s]' covers the parsed-but-null state case; this one is the null-entity case.

Related errors

• Failed to create row serializer, unsupported `format` from s
• load_url cannot be empty, or the host cannot connect.Please
• Unable to flush data to Doris: unknown result status.
• Failed to flush data to Doris. %s
• Failed to parse delimiter: `Hex str is empty`

AI-assisted analysis of alibaba/DataX@80ec23d5c5 (2026-08-14). Data as JSON: /api/errors/c8bbc2eb47a8df02. Report an issue: GitHub.