alibaba/DataX · error · IOException

Failed to flush data to Doris, Error could not get the final

Error message

Failed to flush data to Doris, Error could not get the final state of label[%s]. response[%s]

What it means

Thrown by DorisStreamLoadObserver after a stream-load PUT when it polls the label state via HTTP GET and the JSON response either has no body or its parsed 'data' field (the label state) is null. It means the job could not determine whether the load labeled 'label' succeeded, so the flush is treated as failed. The full response body is embedded to help diagnose what the FE actually returned.

Source

Thrown at doriswriter/src/main/java/com/alibaba/datax/plugin/writer/doriswriter/DorisStreamLoadObserver.java:103

                TimeUnit.SECONDS.sleep(Math.min(++idx, 5));
            } catch (InterruptedException ex) {
                break;
            }
            try (CloseableHttpClient httpclient = HttpClients.createDefault()) {
                HttpGet httpGet = new HttpGet(new StringBuilder(host).append("/api/").append(options.getDatabase()).append("/get_load_state?label=").append(label).toString());
                httpGet.setHeader("Authorization", getBasicAuthHeader(options.getUsername(), options.getPassword()));
                httpGet.setHeader("Connection", "close");

                try (CloseableHttpResponse resp = httpclient.execute(httpGet)) {
                    HttpEntity respEntity = getHttpEntity(resp);
                    if (respEntity == null) {
                        throw new IOException(String.format("Failed to flush data to Doris, Error " +
                                "could not get the final state of label[%s].\n", label), null);
                    }
                    Map<String, Object> result = (Map<String, Object>)JSON.parse(EntityUtils.toString(respEntity));
                    String labelState = (String)result.get("data");
                    if (null == labelState) {
                        throw new IOException(String.format("Failed to flush data to Doris, Error " +
                                "could not get the final state of label[%s]. response[%s]\n", label, EntityUtils.toString(respEntity)), null);
                    }
                    LOG.info(String.format("Checking label[%s] state[%s]\n", label, labelState));
                    switch(labelState) {
                        case LAEBL_STATE_VISIBLE:
                        case LAEBL_STATE_COMMITTED:
                            return;
                        case RESULT_LABEL_PREPARE:
                            continue;
                        case RESULT_LABEL_ABORTED:
                            throw new DorisWriterExcetion (String.format("Failed to flush data to Doris, Error " +
                                    "label[%s] state[%s]\n", label, labelState), null, true);
                        case RESULT_LABEL_UNKNOWN:
                        default:
                            throw new IOException(String.format("Failed to flush data to Doris, Error " +
                                    "label[%s] state[%s]\n", label, labelState), null);
                    }
                }

View on GitHub (pinned to 80ec23d5c5)

Solutions

1. Inspect the embedded response[%s] in the log message: if it is HTML/auth error, fix the FE address or credentials in the job config
2. Verify the FE http port configured for loadUrl matches the FE you query (default 8030) and the label was created on that FE
3. If the label is genuinely unknown but data may have committed, manually check the label state on the FE (SHOW TRANSACTION / stream load state API) before re-running
4. Re-run the job with a new label prefix (labels are immutable in Doris; a retry with the same label will always fail)

Example fix

// before: label prefix reused across retries causes unknown state
"labelPrefix": "myjob_20240101_"
// after: unique prefix per run so a retry never collides with a committed label
"labelPrefix": "myjob_20240101_run2_"

// before flush, verify the FE label-state endpoint is reachable and returns JSON
HttpGet probe = new HttpGet(loadUrl.replace("/api/" + db + "/_stream_load", "/api/" + db + "/" + label + "/_get_state"));
probe.setHeader("Authorization", basicAuth);
try (CloseableHttpResponse r = httpclient.execute(probe)) {
    String body = EntityUtils.toString(r.getEntity());
    if (!body.trim().startsWith("{")) throw new IOException("FE returned non-JSON, check loadUrl/port: " + body);
}

try {
    observer.flush();
} catch (IOException e) {
    if (e.getMessage() != null && e.getMessage().contains("could not get the final state")) {
        // indeterminate outcome: manually verify label state on FE before re-running
        verifyLabelOnFeManually(label);
    } else { throw e; }
}

When it happens

Trigger: Calling flush() -> the observer GETs the label-state endpoint (e.g. http://fe:8030/api/{db}/{label}/_get_state) and the FE returns JSON whose 'data' key is missing/null, or getHttpEntity(resp) yields null (empty entity) so an earlier sibling check fires. Happens when the label never existed on the FE (already cleaned up), the FE restarted, or a non-Doris endpoint (proxy/error page) answered.

Common situations: Querying a label created by a different FE than the one handling the state request; label expired from FE memory; wrong loadUrl/port configured in the job (httpPort vs queryPort); an intermediate proxy returning HTML so JSON.parse yields a map without 'data'.

Related errors

• load_url cannot be empty, or the host cannot connect.Please
• Failed to create row serializer, unsupported `format` from s
• Unable to flush data to Doris: unknown result status.
• Failed to flush data to Doris. %s
• Failed to flush data to Doris, Error could not get the final

AI-assisted analysis of alibaba/DataX@80ec23d5c5 (2026-08-14). Data as JSON: /api/errors/089606e9f39d3459. Report an issue: GitHub.