apache/seatunnel · warning

Logger level request to member {} failed

Error message

Logger level request to member {} failed

What it means

LoggerLevelService.fanOut sends logger-level GET/SET/RESET requests to each cluster member over HTTP. If a member request throws, this warning is logged (with the throwable) and a per-node JSON entry with STATUS=FAILURE and the error message is added to the response, so the caller sees which node failed while other nodes still report.

Source

Thrown at seatunnel-engine/seatunnel-engine-server/src/main/java/org/apache/seatunnel/engine/server/rest/service/LoggerLevelService.java:189

                                NodeEngineUtil.sendOperationToMemberNode(
                                                nodeEngine,
                                                new GetNodeHttpPortOperation(),
                                                member.getAddress())
                                        .get();
                memberId = member.getAddress().getHost() + ":" + httpPort;
                String url =
                        "http://"
                                + member.getAddress().getHost()
                                + ":"
                                + httpPort
                                + httpConfig.getContextPath()
                                + RestConstant.REST_URL_LOGGERS
                                + (name == null ? "" : "/" + encode(name))
                                + (level == null ? "" : "?" + LEVEL + "=" + encode(level.name()));
                nodes.add(request(url, method, httpConfig));
                reached++;
            } catch (Throwable t) {
                log.warn("Logger level request to member {} failed", memberId, t);
                nodes.add(
                        new JsonObject()
                                .add(NODE, memberId)
                                .add(STATUS, FAILURE)
                                .add(ERROR, errorMessage(t)));
                failed++;
            }
        }
        String status = failed == 0 ? SUCCESS : (reached == 0 ? FAILURE : PARTIAL_FAILURE);
        JsonObject response = new JsonObject().add(SCOPE, SCOPE_CLUSTER).add(STATUS, status);
        if (name != null) {
            response.add(NAME, name);
        }
        if (level != null) {
            response.add(LEVEL, level.name());
        }
        return response.add(NODES, nodes);
    }

View on GitHub (pinned to cf67b549a7)

Solutions

  1. Inspect the per-node ERROR field in the returned JSON to see why that member failed.
  2. Verify the failing member is up and its REST port is reachable; restart or fix networking.
  3. Check basic-auth configuration consistency across nodes.
  4. Use a valid level enum supported by the target logging implementation; avoid mixed versions in the cluster.

Example fix

// before
// POST /loggers/MyLogger?level=VERBOSE
// after (valid level)
// POST /loggers/MyLogger?level=DEBUG
Defensive patterns

Strategy: try-catch

Validate before calling

// before fan-out, validate level and member reachability
// if (!VALID_LEVELS.contains(level.name())) throw new IllegalArgumentException("bad level");
// if (!isNodeReachable(memberId)) { preMarkFailure(memberId); }

Try / catch

try {
    nodes.add(request(url, method, httpConfig));
    reached++;
} catch (Throwable t) {
    log.warn("Logger level request to member {} failed", memberId, t);
    nodes.add(perNodeFailure(memberId, t)); // node entry with STATUS=FAILURE
}

Prevention

When it happens

Trigger: Calling cluster loggers / clusterSetLevel / clusterResetLevel REST endpoints when a member is unreachable, its HTTP port is wrong, basic-auth fails, or the target logger name/level is rejected by that node's endpoint.

Common situations: One node down or restarting during the fan-out; per-node auth credentials mismatch; mixed-version cluster where the /loggers endpoint differs; invalid level name for that node's logging framework.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/e89cbfeda4079df7. Report an issue: GitHub.