apache/seatunnel · warning

Failed to clear scroll ID: ${scrollId}

Error message

Failed to clear scroll ID: ${scrollId}

What it means

clearScroll's catch-all: any Exception during the DELETE /_search/scroll exchange (ConnectException, SocketTimeout, response parsing failure, EntityUtils IO error) is logged with the scrollId and the method returns false. Cleanup failure is deliberately non-fatal — the scroll will still expire via its TTL.

Source

Thrown at seatunnel-connectors-v2/connector-elasticsearch/src/main/java/org/apache/seatunnel/connectors/seatunnel/elasticsearch/client/EsRestClient.java:343

                log.warn("DELETE {} response null for scroll ID: {}", endpoint, scrollId);
                return false;
            }
            String entity = EntityUtils.toString(response.getEntity(), StandardCharsets.UTF_8);
            if (response.getStatusLine().getStatusCode() == HttpStatus.SC_OK) {
                JsonNode jsonNode = JsonUtils.parseObject(entity);
                boolean succeeded = jsonNode.get("succeeded").asBoolean();
                return succeeded;
            } else {
                log.warn(
                        "DELETE {} response status code={}, body={} for scroll ID: {}",
                        endpoint,
                        response.getStatusLine().getStatusCode(),
                        entity,
                        scrollId);
                return false;
            }
        } catch (Exception ex) {
            log.warn("Failed to clear scroll ID: " + scrollId, ex);
            return false;
        }
    }

    /**
     * Close SQL cursor to release server-side resources.
     *
     * @param cursor The SQL cursor to close
     * @return True if the cursor was successfully closed
     */
    public boolean closeSqlCursor(String cursor) {
        if (StringUtils.isEmpty(cursor)) {
            log.debug("SQL cursor is empty; skip closing.");
            return false;
        }

        String endpoint = "/_sql/close";
        Request request = new Request("POST", endpoint);

View on GitHub (pinned to cf67b549a7)

Solutions

1. Confirm the cluster is reachable at cleanup time; check proxies/firewalls between SeaTunnel and ES
2. Bound scroll TTL on the initial search so uncleaned scrolls expire anyway (cleanup is best-effort)
3. If clearing during close, ensure the RestClient is still open before calling clearScroll (order teardown correctly)
4. Retry once on transient IO errors; clearing is idempotent
5. Inspect the logged exception stack to distinguish transport vs parse errors

Example fix

// before: clear scroll after closing the client
esClient.close();
esClient.clearScroll(scrollId);
// after: clear while client is still usable
esClient.clearScroll(scrollId);
esClient.close();

// one retry, then give up gracefully — scroll expires via TTL
if (!esClient.clearScroll(scrollId)) {
    Thread.sleep(1000);
    esClient.clearScroll(scrollId); // idempotent best-effort
}

When it happens

Trigger: Network failure, connection reset, timeout, or JSON/entity parse error while performing or reading the DELETE /_search/scroll response.

Common situations: Elasticsearch unreachable mid-job (network partition, node restart); scroll clear fired during shutdown while the client connection is already closed; malformed server response from a proxy/gateway (e.g. HTML error page).

Related errors

• Error clearing scrollId
• Failed to clear Easysearch scrollId:
• DELETE {} response status code={}, body={} for scroll ID: {}
• Failed to close SQL cursor: {}
• Failed to clear scroll ID: ${scrollId}

AI-assisted analysis of apache/seatunnel@cf67b549a7 (2026-09-10). Data as JSON: /api/errors/93d3f62cb1f37f8d. Report an issue: GitHub.