{"record":{"id":"560c5d745391694d","repo":"apache/hadoop","slug":"join-on-writer-thread-thread-timed-out","errorCode":null,"errorMessage":"Join on writer thread ${thread} timed out","messagePattern":"Join on writer thread (.+?) timed out","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/LocalReplicaInPipeline.java","lineNumber":276,"sourceCode":"    while (true) {\n      Thread thread = writer.get();\n      if ((thread == null) || (thread == Thread.currentThread()) ||\n          (!thread.isAlive())) {\n        if (writer.compareAndSet(thread, null)) {\n          return; // Done\n        }\n        // The writer changed.  Go back to the start of the loop and attempt to\n        // stop the new writer.\n        continue;\n      }\n      thread.interrupt();\n      try {\n        thread.join(xceiverStopTimeout);\n        if (thread.isAlive()) {\n          // Our thread join timed out.\n          final String msg = \"Join on writer thread \" + thread + \" timed out\";\n          DataNode.LOG.warn(msg + \"\\n\" + StringUtils.getStackTrace(thread));\n          throw new IOException(msg);\n        }\n      } catch (InterruptedException e) {\n        throw new IOException(\"Waiting for writer thread is interrupted.\");\n      }\n    }\n  }\n\n  @Override  // Object\n  public int hashCode() {\n    return super.hashCode();\n  }\n\n  @Override // ReplicaInPipeline\n  public ReplicaOutputStreams createStreams(boolean isCreate,\n      DataChecksum requestedChecksum) throws IOException {\n    final File blockFile = getBlockFile();\n    final File metaFile = getMetaFile();\n    if (DataNode.LOG.isDebugEnabled()) {","sourceCodeStart":258,"sourceCodeEnd":294,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/LocalReplicaInPipeline.java#L258-L294","documentation":"Error \"Join on writer thread ${thread} timed out\" thrown in apache/hadoop.","triggerScenarios":"Joining the replica writer thread times out after a write or finalize, because the writer is stalled.","commonSituations":"The replica writer thread did not stop within the join timeout, typically due to a blocked disk I/O. Check for slow or failing disks.","solutions":["Investigate why the writer thread did not finish (slow disk, downstream stall); check disk metrics and downstream DataNode logs.","Increase timeouts only after ruling out hardware and network stalls."],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}