{"record":{"id":"08de992bef818094","repo":"apache/beam","slug":"could-not-extract-error-codes-from-responseentity","errorCode":null,"errorMessage":"Could not extract error codes from responseEntity {}","messagePattern":"Could not extract error codes from responseEntity (.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java","lineNumber":1548,"sourceCode":"      }\n\n      DefaultRetryPredicate() {\n        this(429);\n      }\n\n      /** Returns true if the response has the error code for any mutation. */\n      private static boolean errorCodePresent(HttpEntity responseEntity, int errorCode) {\n        try {\n          JsonNode json = parseResponse(responseEntity);\n          if (json.path(\"errors\").asBoolean()) {\n            for (JsonNode item : json.path(\"items\")) {\n              if (item.findValue(\"status\").asInt() == errorCode) {\n                return true;\n              }\n            }\n          }\n        } catch (IOException e) {\n          LOG.warn(\"Could not extract error codes from responseEntity {}\", responseEntity);\n        }\n        return false;\n      }\n\n      @Override\n      public boolean test(HttpEntity responseEntity) {\n        return errorCodePresent(responseEntity, errorCode);\n      }\n    }\n  }\n\n  /** A {@link PTransform} converting docs to their Bulk API counterparts. */\n  @AutoValue\n  public abstract static class DocToBulk\n      extends PTransform<PCollection<String>, PCollection<Document>> {\n\n    private static final ObjectMapper OBJECT_MAPPER = new ObjectMapper();\n    private static final int DEFAULT_RETRY_ON_CONFLICT = 5; // race conditions on updates","sourceCodeStart":1530,"sourceCodeEnd":1566,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/java/io/elasticsearch/src/main/java/org/apache/beam/sdk/io/elasticsearch/ElasticsearchIO.java#L1530-L1566","documentation":"In ElasticsearchIO's RetryConfiguration error-code extractor, the code tries to parse the error response entity to find per-item HTTP status values and compare them to a predicate error code. If reading the entity content throws IOException, parsing fails and it logs this warning, returning false (the response is not treated as retryable). The actual write error is surfaced elsewhere; this warning means the retry predicate could not classify the error.","triggerScenarios":"An entity's getContent() stream throws IOException while scanning bulk items for findValue(\"status\").asInt() == errorCode — typically a truncated or already-consumed response stream during retry evaluation.","commonSituations":"Connection reset while reading the bulk error response; response body drained by a previous attempt; proxy terminating the response early; very large error payloads cut off by timeouts.","solutions":["Check network stability between the Beam worker and Elasticsearch; truncated responses cause this.","Inspect the logged responseEntity to understand the underlying server error.","Adjust RetryConfiguration to retry on exceptions (e.g. retryPredicates covering IOExceptions) instead of relying only on status codes.","Ensure Elasticsearch returns well-formed error bodies (check server/proxy logs for truncation)."],"exampleFix":"// before: only classifies by parsed status codes, warns on parse failure\nRetryConfiguration.retryPredicates();\n// after: also retry on IO errors explicitly\n.withRetryStrategy(new ElasticsearchIO.RetryStrategy(10, 1000))\n.withMaxCumulativeAttempts(10); // plus ensure network stability","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// the retry predicate swallows the IOException internally; configure resilient retries at the sink\n.withRetryConfiguration(ElasticsearchIO.RetryConfiguration.create(10, 1000))\n.withMaxBatchSize(1000L)","preventionTips":["Ensure stable network paths to Elasticsearch; truncated responses break entity parsing.","Set explicit RetryConfiguration instead of relying on default error-code extraction.","Keep batch sizes moderate so error responses stay small and fully readable."],"tags":["elasticsearch","http","retry","io"],"backgroundTag":"api-error-response","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}