apache/seatunnel · error · IOException

HTTP ${code} from ${url}: ${errorBody}

Error message

HTTP ${code} from ${url}: ${errorBody}

What it means

LoggerLevelService.request performs an HTTP call to a remote node's log-level REST endpoint and throws an IOException whenever the response status is not 200, embedding the status code, the URL, and the error-stream body in the message. It is propagated by fanOut when aggregating log-level changes across cluster nodes, so a single unhealthy node surfaces this error.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/service/LoggerLevelService.java:229

        HttpURLConnection connection = (HttpURLConnection) new URL(url).openConnection();
        try {
            connection.setRequestMethod(method);
            connection.setConnectTimeout(REQUEST_TIMEOUT_MS);
            connection.setReadTimeout(REQUEST_TIMEOUT_MS);
            if (httpConfig.isEnableBasicAuth()) {
                String credentials =
                        httpConfig.getBasicAuthUsername() + ":" + httpConfig.getBasicAuthPassword();
                connection.setRequestProperty(
                        "Authorization",
                        "Basic "
                                + Base64.getEncoder()
                                        .encodeToString(
                                                credentials.getBytes(StandardCharsets.UTF_8)));
            }
            connection.connect();
            int code = connection.getResponseCode();
            if (code != HttpURLConnection.HTTP_OK) {
                throw new IOException(
                        "HTTP " + code + " from " + url + ": " + read(connection.getErrorStream()));
            }
            return Json.parse(read(connection.getInputStream())).asObject();
        } finally {
            connection.disconnect();
        }
    }

    private JsonObject loggerJson(String name, Level level) {
        JsonObject logger =
                new JsonObject()
                        .add(NAME, name)
                        .add(LEVEL, level == null ? null : level.name())
                        .add(ORIGIN, LogLevels.origin(name));
        Level fileLevel = LogLevels.levelBeforeOverride(name);
        if (fileLevel != null) {
            logger.add(FILE_LEVEL, fileLevel.name());
        }

View on GitHub (pinned to cf67b549a7)

Solutions

1. Read the errorBody embedded in the message to identify the failing node and status; fix that node first.
2. Verify all nodes run the same SeaTunnel version and expose the log-level REST endpoint on the configured http port.
3. Check network/firewall and auth (basic credentials) consistency across the cluster.
4. Retry the operation once the offending node is healthy; fanOut errors are per-node and transient 5xx may succeed on retry.

Example fix

// before
curl -X POST 'http://old-worker:8080/log-level' ...  -> 404 on upgraded-mixed cluster
// after
# upgrade all nodes, verify endpoint, then re-run cluster-wide level change
curl -X POST 'http://worker:8080/log-level?logger=root&level=WARN'

// pre-check each node before fan-out
nodes.forEach(n => fetch("http://" + n + ":" + port + "/health") .then(r => { if (!r.ok) throw new Error("node " + n + " unhealthy: " + r.status); }));

try { fanOut(nodes, payload); } catch (IOException e) {
  if (e.getMessage().matches("HTTP \\d+ .*")) {
    // parse code/url from message; retry transient 5xx, alert on 4xx
    retryWithBackoff(excludingPersistentlyFailingNodes(e));
  } else throw e;
}

When it happens

Trigger: Changing or querying logger levels cluster-wide where any node answers with a non-200 status: 404 (node running an older version without the log-level endpoint), 401 (auth mismatch), 500 (node-side error), or connection to a wrong port/host.

Common situations: Rolling upgrades where some nodes lack the new log-level API; httpConfig port changed on some workers but not others; security/auth settings differing between nodes; a node in a crashed or overloaded state returning 5xx.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

• Failed to fetch metadata from Gravitino for metadata: %s
• fail get tableSchema:
• SEND_RESPONSE_FAILED
• REST_SERVICE_FAILED
• Failed to get response from Doris

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/3bc35032e4194653. Report an issue: GitHub.