apache/hadoop · warning · InvalidRequestException

{this}: slot {slotIdx} does not exist.

Error message

{this}: slot {slotIdx} does not exist.

What it means

ShortCircuitShm.getSlot(slotIdx) returns a previously registered slot; on the datanode it is reached through ShortCircuitRegistry.unregisterSlot when a client releases a short-circuit slot. If the index's bit is not set in the segment's allocatedSlots BitSet, the slot was never registered (or was already unregistered) on this side, and InvalidRequestException '<shm>: slot <idx> does not exist.' is thrown - a client/datanode bookkeeping mismatch on the shared-memory registry.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/shortcircuit/ShortCircuitShm.java:555

    if (idx >= slots.length) {
      throw new RuntimeException(this + ": no more slots are available.");
    }
    allocatedSlots.set(idx, true);
    Slot slot = new Slot(calculateSlotAddress(idx), blockId);
    slot.clear();
    slot.makeValid();
    slots[idx] = slot;
    if (LOG.isTraceEnabled()) {
      LOG.trace(this + ": allocAndRegisterSlot " + idx + ": allocatedSlots=" + allocatedSlots +
                  StringUtils.getStackTrace(Thread.currentThread()));
    }
    return slot;
  }

  synchronized public final Slot getSlot(int slotIdx)
      throws InvalidRequestException {
    if (!allocatedSlots.get(slotIdx)) {
      throw new InvalidRequestException(this + ": slot " + slotIdx +
          " does not exist.");
    }
    return slots[slotIdx];
  }

  /**
   * Register a slot.
   *
   * This function looks at a slot which has already been initialized (by
   * another process), and registers it with us.  Then, it returns the
   * relevant Slot object.
   *
   * @return    The slot.
   *
   * @throws InvalidRequestException
   *            If the slot index we're trying to allocate has not been
   *            initialized, or is already in use.
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Restart long-lived client applications after a datanode restart so they re-establish shm segments
2. Check for version skew between client and datanode; the shm slot protocol must match
3. Treat as transient - clients re-request a new shm after InvalidRequestException and recover; confirm the flood stops in datanode logs
4. If persistent, gather datanode log context around opRequestShortCircuitFds / opReleaseShortCircuitFds and file a JIRA

try {
  // short-circuit operations carrying a SlotId
} catch (InvalidRequestException e) {
  // slot bookkeeping is stale: drop the cached shm/slot and re-establish the short-circuit session
}

When it happens

Trigger: A datanode processing a client's slot release / short-circuit cleanup that references a slot index it never registered in that shm: stale SlotId after the datanode restarted and recreated its registry, or a client referencing a slot it already released.

Common situations: Datanode restarts while long-lived clients kept their shm segments; rolling upgrades; client and datanode Hadoop version skew in the shm-slot protocol; duplicated release requests.

Related errors

• {this}: invalid negative slot index {slotIdx}
• {this}: invalid slot index {slotIdx}
• {this}: slot {slotIdx} is already in use.
• {this}: no more slots are available.
• {this}: slot {slotIdx} is not marked as valid.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/632ea52b17d57843. Report an issue: GitHub.