{"record":{"id":"95c103cd6c2a3b51","repo":"apache/hadoop","slug":"millis-timeout-while-waiting-for-channel-to-be","errorCode":null,"errorMessage":"{} millis timeout while waiting for channel to be ready for {}. ch : {}","messagePattern":"(.+?) millis timeout while waiting for channel to be ready for (.+?)\\. ch : (.+?)","errorType":"exception","errorClass":"SocketTimeoutException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java","lineNumber":163,"sourceCode":"        }\n      } catch (IOException e) {\n        if (!channel.isOpen()) {\n          closed = true;\n        }\n        throw e;\n      }\n\n      //now wait for socket to be ready.\n      int count = 0;\n      try {\n        count = SelectorPool.select(channel, ops, timeout);\n      } catch (IOException e) { //unexpected IOException.\n        closed = true;\n        throw e;\n      } \n\n      if (count == 0) {\n        throw new SocketTimeoutException(timeoutExceptionString(channel,\n                                                                timeout, ops));\n      }\n      // otherwise the socket should be ready for io.\n    }\n    \n    return 0; // does not reach here.\n  }\n  \n  /**\n   * The contract is similar to {@link SocketChannel#connect(SocketAddress)} \n   * with a timeout.\n   * \n   * @see SocketChannel#connect(SocketAddress)\n   * \n   * @param channel - this should be a {@link SelectableChannel}\n   * @param endpoint\n   * @throws IOException\n   */","sourceCodeStart":145,"sourceCodeEnd":181,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/net/SocketIOWithTimeout.java#L145-L181","documentation":"Inside doIO, when a non-blocking operation returns 0 the code parks in SelectorPool.select(channel, ops, timeout). If select reports zero ready operations by the deadline, SocketTimeoutException is thrown with a message built by timeoutExceptionString naming the operation (read/write) and channel. This is the generic IO-readiness timeout of the stream wrappers.","triggerScenarios":"A read on SocketInputStream or write on SocketOutputStream where the channel never becomes ready within this.timeout (set via constructor or setTimeout) — the peer stalls mid-response, stops draining its receive window, or the network drops packets.","commonSituations":"Hung or GC-paused datanode/name-node peers; network partitions mid-transfer; client socket timeouts (e.g. dfs.client.socket-timeout) set below real transfer latencies; large block reads over slow links.","solutions":["Raise the timeout (setTimeout / dfs.client.socket-timeout and related ipc properties) to cover worst-case latency","Check peer health and network throughput — a timeout here usually means a stalled sender, not slow data","Catch SocketTimeoutException at the call site and retry/fail over per client policy instead of treating it as fatal corruption"],"exampleFix":"// before\nSocketInputStream in = new SocketInputStream(channel, 5_000); // 5s too small\n\n// after\nSocketInputStream in = new SocketInputStream(channel, 60_000);\nin.setTimeout(120_000); // extend for long transfers","handlingStrategy":"retry","validationCode":"long timeout = Math.max(60_000, expectedMaxTransferMillis());\nstream.setTimeout(timeout);","typeGuard":null,"tryCatchPattern":"catch (SocketTimeoutException e) {\n  // readiness timeout: peer stalled. Verify peer, then retry with backoff\n  if (--attempts > 0 && peerHealthy()) {\n    reopenStreamAndRetry();\n  } else {\n    throw e;\n  }\n}","preventionTips":["Size timeouts to worst-case latency of the workload, not the happy path","Monitor peer GC/disk stalls; a readiness timeout is usually a symptom, not the disease"],"tags":["hadoop-common","network","timeout","nio","java"],"backgroundTag":"socket-read-write-timeout","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}