apache/beam · error

Painless scripts are not supported on Elasticsearch…

Error message

Painless scripts are not supported on Elasticsearch clusters before version 5.0

What it means

Error raised by ElasticsearchIO's DocToBulk path when scripted upserts are configured (withUpsertScript) but the target Elasticsearch cluster reports a version below 5.0. Painless scripting did not exist before ES 5.x, so the configured upsert script cannot be executed against that cluster and the write is rejected.

Solutions

1. Upgrade the Elasticsearch cluster to 5.0 or later to use painless upsert scripts
2. Drop withUpsertScript and use plain document appends (appendOnly) on pre-5.0 clusters

When it happens

Trigger: Thrown at sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java:1740 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/baa90aa73d0b0d06. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java:1740

     *
     * <p>When the documents contain
     *
     * @param appendOnly set to true to allow only document appending
     * @return the {@link DocToBulk} with the-append only control set
     */
    public DocToBulk withAppendOnly(boolean appendOnly) {
      return builder().setAppendOnly(appendOnly).build();
    }

    /**
     * Whether to use scripted updates and what script to use.
     *
     * @param source set to the value of the script source, painless lang
     * @return the {@link DocToBulk} with the scripted updates set
     */
    public DocToBulk withUpsertScript(String source) {
      if (getBackendVersion() == null || getBackendVersion() == 2) {
        LOG.warn("Painless scripts are not supported on Elasticsearch clusters before version 5.0");
      }
      return builder().setUsePartialUpdate(false).setUpsertScript(source).build();
    }

    /**
     * Provide a function to extract the doc version from the document. This version number will be
     * used as the document version in Elasticsearch. Should the function throw an Exception then
     * the batch will fail and the exception propagated. Incompatible with update operations and
     * should only be used with withUsePartialUpdate(false)
     *
     * @param docVersionFn to extract the document version
     * @return the {@link DocToBulk} with the function set
     */
    public DocToBulk withDocVersionFn(Write.FieldValueExtractFn docVersionFn) {
      checkArgument(docVersionFn != null, "docVersionFn must not be null");
      return builder().setDocVersionFn(docVersionFn).build();
    }

View on GitHub (pinned to 12126d8942)