{"record":{"id":"e9c0556f1d10e4b9","repo":"apache/hadoop","slug":"request-execution-with-deadline-failed","errorCode":null,"errorMessage":"Request execution with deadline failed","messagePattern":"Request execution with deadline failed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsApacheHttpClient.java","lineNumber":215,"sourceCode":"        .setSocketTimeout(readTimeout);\n    httpRequest.setConfig(requestConfigBuilder.build());\n    ExecutorService executor = Executors.newSingleThreadExecutor();\n    Future<HttpResponse> future = executor.submit(() ->\n        httpClient.execute(httpRequest, abfsHttpClientContext)\n    );\n\n    try {\n      return future.get(deadlineMillis, TimeUnit.MILLISECONDS);\n    } catch (TimeoutException e) {\n      /* Deadline exceeded, abort the request.\n       * This will also kill the underlying socket exception in the HttpClient.\n       * Connection will be marked stale and won't be returned back to KAC for reuse.\n       */\n      httpRequest.abort();\n      throw new TailLatencyRequestTimeoutException(e);\n    } catch (Exception e) {\n      // Any other exception from execution should be thrown as IOException.\n      throw new IOException(\"Request execution with deadline failed\", e);\n    } finally {\n      executor.shutdownNow();\n    }\n  }\n\n  /**\n   * Creates the socket factory registry for HTTP and HTTPS.\n   *\n   * @param sslSocketFactory SSL socket factory.\n   * @return Socket factory registry.\n   */\n  private Registry<ConnectionSocketFactory> createSocketFactoryRegistry(\n      ConnectionSocketFactory sslSocketFactory) {\n    if (sslSocketFactory == null) {\n      return RegistryBuilder.<ConnectionSocketFactory>create()\n          .register(HTTP_SCHEME,\n              PlainConnectionSocketFactory.getSocketFactory())\n          .build();","sourceCodeStart":197,"sourceCodeEnd":233,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-tools/hadoop-azure/src/main/java/org/apache/hadoop/fs/azurebfs/services/AbfsApacheHttpClient.java#L197-L233","documentation":"The catch-all branch of AbfsApacheHttpClient.executeRequest wraps every non-timeout failure of the underlying HttpClient execution (connection acquisition errors, SSL failures, socket resets, InterruptedException from executor shutdown) into IOException('Request execution with deadline failed') with the real cause chained. The message itself is only a wrapper — the actionable diagnosis is always in the cause chain.","triggerScenarios":"httpClient.execute(...) throwing anything other than TimeoutException on the worker thread: UnknownHostException, SSLException, ConnectException, connection-pool exhaustion, or the executor being interrupted (shutdownNow). Distinct from 4802, which fires on deadline expiry only.","commonSituations":"Misconfigured proxy or missing SSL truststore entries; keep-alive connections reset by middleboxes; socket/handle exhaustion under very high concurrency; races between job close and in-flight requests.","solutions":["Unwrap the chain (e.getCause(), often twice) and fix the underlying connectivity error identified there","For DNS/proxy/TLS causes, correct the endpoint URL, proxy settings, or truststore rather than retrying","For pool or fd exhaustion, tune fs.azure.io.* concurrency settings or reduce parallelism","Retry idempotent reads — connection-level failures are frequently transient"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  abfsClient.executeRequest(...);\n} catch (IOException e) {\n  Throwable root = e;\n  while (root.getCause() != null) root = root.getCause();\n  // diagnose root: UnknownHost / SSL / Connect / pool exhaustion, then act or retry\n}","preventionTips":["Never debug from the wrapper message alone — always unwrap to the root cause","Validate proxy, DNS and truststore configuration before running large jobs","Keep an eye on connection/fd usage when raising client concurrency"],"tags":["azure","abfs","http","io","wrapper-exception","network"],"backgroundTag":"http-request-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}