apache/beam · warning

Support for Elasticsearch cluster version

Error message

Support for Elasticsearch cluster version {} will be dropped in a future release of the Apache Beam SDK

What it means

ElasticsearchIO maintains a list of Elasticsearch major cluster versions scheduled for removal from support. When the sink connects and reads the cluster version, it checks that list and logs this deprecation warning so operators can plan an upgrade before the SDK drops compatibility.

Solutions

1. Upgrade the Elasticsearch cluster to a supported major version
2. Pin an older Beam SDK release if an immediate cluster upgrade is impossible
3. Track the Beam release notes for the date support for that cluster version is removed

Example fix

// before
// pipeline writes to Elasticsearch 2.4 cluster
ElasticsearchIO.write().withConnectionConfiguration(DiscoveryAndFiltering.of("es-legacy-host", 9200))
// after
// upgrade cluster to a supported version first, then same code emits no warning
ElasticsearchIO.write().withConnectionConfiguration(DiscoveryAndFiltering.of("es-host", 9200))

// check cluster version before deploying
GET / returned version.major; if (deprecatedVersions.contains(major)) { planUpgrade(); }

When it happens

Trigger: Running an ElasticsearchIO read/write whose connection targets an Elasticsearch cluster whose major version is in DEPRECATED_CLUSTER_VERSIONS (e.g. version 2 or other legacy majors).

Common situations: Teams running older self-managed Elasticsearch clusters (or legacy managed offerings) upgrade the Beam SDK but not the database, and see the warning at pipeline startup after version sniffing.

Understand the failure class

Background: "is deprecated and will be removed" — deprecation warnings for old API names, keywords, and options, and how to migrate before the removal release — this error's family across 29 libraries.

Related errors

• Use of withMaxBufferingDuration requires…
• A function must be provided to convert the input type into…
• A PValue contained in
• A schema was provided without a data format (or viceversa)…
• All inherited interfaces of

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/2052a788003db33d. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java:2998

          } else {
            LOG.warn("ES Cluster is responding with HTP 429 - TOO_MANY_REQUESTS.");
          }
        }
        throw new IOException(String.format(RETRY_FAILED_LOG, attempt));
      }

      @Teardown
      public void closeClient() throws IOException {
        if (restClient != null) {
          restClient.close();
        }
      }
    }
  }

  private static void maybeLogVersionDeprecationWarning(int clusterVersion) {
    if (DEPRECATED_CLUSTER_VERSIONS.contains(clusterVersion)) {
      LOG.warn(
          "Support for Elasticsearch cluster version {} will be dropped in a future release of "
              + "the Apache Beam SDK",
          clusterVersion);
    }
  }

  static int getBackendVersion(RestClient restClient) {
    try {
      Request request = new Request("GET", "");
      Response response = restClient.performRequest(request);
      JsonNode jsonNode = parseResponse(response.getEntity());
      int backendVersion =
          Integer.parseInt(jsonNode.path("version").path("number").asText().substring(0, 1));
      checkArgument(
          VALID_CLUSTER_VERSIONS.contains(backendVersion),
          "The Elasticsearch version to connect to is %s.x. "
              + "This version of the ElasticsearchIO is only compatible with "
              + "Elasticsearch "

View on GitHub (pinned to 12126d8942)