apache/hadoop · error · IOException

Interrupted waiting for lockSharedStorage() response

Error message

Interrupted waiting for lockSharedStorage() response

What it means

Thrown while canRollBack() was blocked in waitFor() and the waiting thread was interrupted. Note the message text says 'lockSharedStorage()' — a copy-paste quirk in the source; the operation actually in flight is the canRollBack quorum call. InterruptedException is converted to an IOException without restoring the interrupt flag.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/qjournal/client/QuorumJournalManager.java:744

          "lockSharedStorage");
      
      if (call.countExceptions() > 0) {
        call.rethrowException("Could not check if roll back possible for"
            + " one or more JournalNodes");
      }
      
      // Either they all return the same thing or this call fails, so we can
      // just return the first result.
      try {
        DFSUtil.assertAllResultsEqual(call.getResults().values());
      } catch (AssertionError ae) {
        throw new IOException("Results differed for canRollBack", ae);
      }
      for (Boolean result : call.getResults().values()) {
        return result;
      }
    } catch (InterruptedException e) {
      throw new IOException("Interrupted waiting for lockSharedStorage() " +
          "response");
    } catch (TimeoutException e) {
      throw new IOException("Timed out waiting for lockSharedStorage() " +
          "response");
    }
    
    throw new AssertionError("Unreachable code.");
  }

  @Override
  public void doRollback() throws IOException {
    QuorumCall<AsyncLogger, Void> call = loggers.doRollback();
    try {
      call.waitFor(loggers.size(), loggers.size(), 0, timeoutMs,
          "doRollback");
      
      if (call.countExceptions() > 0) {
        call.rethrowException("Could not perform rollback of one or more JournalNodes");

View on GitHub (pinned to 2add963021)

Solutions

Check what interrupted the thread: look for a preceding NN shutdown/failover or operator cancellation in the logs — the interrupt is the symptom, not the fault.
If the interruption was accidental (tooling timeout on the outer step), widen that outer limit or remove the interrupt and re-run the rollback check.
Re-run the canRollBack/rollback sequence once the cluster is in a stable, single-writer state.
If you are calling this API directly in your own tooling, never interrupt the thread performing quorum waits; cancel at a checkpoint between calls instead.

Defensive patterns

Strategy: try-catch

Type guard

static boolean wasInterruptedWait(IOException ioe) {
  return ioe.getCause() instanceof InterruptedException
      || ioe.getMessage().startsWith("Interrupted waiting");
}

Try / catch

try {
  qjm.canRollBack(storage, prevStorage, targetLayoutVersion);
} catch (IOException ioe) {
  if (wasInterruptedWait(ioe)) {
    Thread.currentThread().interrupt(); // restore the flag, then unwind cleanly
    return OperationStatus.CANCELLED;
  }
  throw ioe;
}

Prevention

Never interrupt a thread in a QJM quorum wait — cancel between operations instead.
In automation, set outer timeouts larger than the worst-case dfs.qjm.operations.timeout so tooling does not interrupt the NN mid-check.
Note the message quirk: 'lockSharedStorage()' here actually means the canRollBack wait.

When it happens

Trigger: Thread interrupt arrives during canRollBack(): the NameNode is shutting down mid-rollback-check, an HA failover transitions the NN out of the executing state, or caller code cancels the operation by interrupting it.

Common situations: 'hdfs namenode -rollback' interrupted by operator Ctrl-C or a shutdown hook; automated upgrade tooling that times out the overall procedure and interrupts the worker thread; JVM shutdown during HA state transitions.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/42a6eefa2e31ebe3. Report an issue: GitHub.