{"record":{"id":"c4ea91ea82c37932","repo":"apache/hadoop","slug":"there-is-no-shared-memory-segment-registered-with","errorCode":null,"errorMessage":"there is no shared memory segment registered with shmId {}","messagePattern":"there is no shared memory segment registered with shmId (.+?)","errorType":"exception","errorClass":"InvalidRequestException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ShortCircuitRegistry.java","lineNumber":352,"sourceCode":"    if (LOG.isTraceEnabled()) {\n      LOG.trace(\"createNewMemorySegment: created \" + info.shmId);\n    }\n    return info;\n  }\n  \n  public synchronized void registerSlot(ExtendedBlockId blockId, SlotId slotId,\n      boolean isCached) throws InvalidRequestException {\n    if (!enabled) {\n      if (LOG.isTraceEnabled()) {\n        LOG.trace(this + \" can't register a slot because the \" +\n            \"ShortCircuitRegistry is not enabled.\");\n      }\n      throw new UnsupportedOperationException();\n    }\n    ShmId shmId = slotId.getShmId();\n    RegisteredShm shm = segments.get(shmId);\n    if (shm == null) {\n      throw new InvalidRequestException(\"there is no shared memory segment \" +\n          \"registered with shmId \" + shmId);\n    }\n    Slot slot = shm.registerSlot(slotId.getSlotIdx(), blockId);\n    if (isCached) {\n      slot.makeAnchorable();\n    } else {\n      slot.makeUnanchorable();\n    }\n    boolean added = slots.put(blockId, slot);\n    Preconditions.checkState(added);\n    if (LOG.isTraceEnabled()) {\n      LOG.trace(this + \": registered \" + blockId + \" with slot \" +\n        slotId + \" (isCached=\" + isCached + \")\");\n    }\n  }\n  \n  public synchronized void unregisterSlot(SlotId slotId)\n      throws InvalidRequestException {","sourceCodeStart":334,"sourceCodeEnd":370,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/ShortCircuitRegistry.java#L334-L370","documentation":"ShortCircuitRegistry.registerSlot() maps a client's shared-memory slot (SlotId = shmId + slotIdx) to an ExtendedBlockId. It first looks up the ShmId in its segments map; an unknown shmId means the DataNode has no registered short-circuit shared memory segment with that id, so it throws InvalidRequestException, which is serialized back to the DFS client over the domain socket / client-datanode protocol.","triggerScenarios":"A client sends a slot registration (block request with a SlotId) whose shmId was never established via a successful requestShortCircuitShm handshake - e.g., the segment was already torn down (DN restart, registry shutdown/re-enable, idle eviction) and the client retries with the stale shmId, or a fabricated/duplicated slot request.","commonSituations":"Client holds a cached ShortCircuitSharedMemorySegment across a DataNode restart or ShortCircuitRegistry flush; races between 'shutdown()' (which clears segments) and in-flight slot requests; tests hammering short-circuit reads (dfs.client.read.shortcircuit=true with dfs.domain.socket.path configured) against a restarted DN.","solutions":["Client side: discard the stale shared-memory segment on InvalidRequestException and re-establish it (new requestShortCircuitShm) before retrying the short-circuit read","Falling back to normal (non-short-circuit) reads after this error is safe and automatic in most client versions - keep the fallback enabled","Operator side: if this floods logs after a DN restart, that is expected churn; ensure clients eventually refresh rather than pinning dead segments"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// client side: only use SlotIds from segments you successfully created\nif (shm == null || !shm.isValid()) { shm = requestShortCircuitShm(dn); } // re-establish before registering slots","typeGuard":null,"tryCatchPattern":"catch (InvalidRequestException e) { // from requestShortCircuitFds with a SlotId\n  if (e.getMessage().contains(\"no shared memory segment\")) {\n    discardShmAndReestablish(); // drop stale shmId, requestShortCircuitShm again, then retry once\n  } else throw e;\n}","preventionTips":["Invalidate cached ShortCircuitSharedMemorySegment on any DN restart or InvalidRequestException","Keep dfs.domain.socket.path stable across restarts so segment handoff stays valid"],"tags":["hdfs","datanode","short-circuit","shared-memory","domain-socket","client"],"backgroundTag":"stale-session-id","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}