{"record":{"id":"f16b791a286b08d1","repo":"apache/hadoop","slug":"got-error-status-status-status-message-mess","errorCode":null,"errorMessage":"Got error, status=${status}, status message ${message}, ${logInfo}","messagePattern":"Got error, status=(.+?), status message (.+?), (.+?)","errorType":"exception","errorClass":"BlockPinningException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/DataTransferProtoUtil.java","lineNumber":118,"sourceCode":"\n  public static void checkBlockOpStatus(\n          BlockOpResponseProto response,\n          String logInfo) throws IOException {\n    checkBlockOpStatus(response, logInfo, false);\n  }\n\n  public static void checkBlockOpStatus(BlockOpResponseProto response,\n      String logInfo, boolean checkBlockPinningErr) throws IOException {\n    if (response.getStatus() != Status.SUCCESS) {\n      if (response.getStatus() == Status.ERROR_ACCESS_TOKEN) {\n        throw new InvalidBlockTokenException(\n          \"Got access token error\"\n          + \", status message \" + response.getMessage()\n          + \", \" + logInfo\n        );\n      } else if (checkBlockPinningErr\n          && response.getStatus() == Status.ERROR_BLOCK_PINNED) {\n        throw new BlockPinningException(\n            \"Got error\"\n            + \", status=\" + response.getStatus().name()\n            + \", status message \" + response.getMessage()\n            + \", \" + logInfo\n          );\n      } else {\n        throw new IOException(\n          \"Got error\"\n          + \", status=\" + response.getStatus().name()\n          + \", status message \" + response.getMessage()\n          + \", \" + logInfo\n        );\n      }\n    }\n  }\n}\n","sourceCodeStart":100,"sourceCodeEnd":135,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/DataTransferProtoUtil.java#L100-L135","documentation":"In checkBlockOpStatus(), when the caller passed checkBlockPinningErr=true and the DataNode returned Status.ERROR_BLOCK_PINNED, a BlockPinningException is thrown. This path applies to writers that requested blocks pinned in memory (lazyPersist/RAM_DISK, via the pinned flag in the op request); the DataNode could not honor pinning, typically because not enough locked memory is available for RAM_DISK replicas.","triggerScenarios":"Creating a file with LAZY_PERSIST storage policy or the pinned-blocks option (dfs.client.blockWriter... / create(...).setPinned/flag) while the DataNode lacks locked memory (dfs.datanode.max.locked.memory too small or 0), RAM_DISK volumes absent, or the OS cannot lock memory (ulimit memlock).","commonSituations":"Enabling centralized cache/lazy-persist features without sizing dfs.datanode.max.locked.memory; nodes missing RAM_DISK volume config; memlock ulimit not raised for the DataNode user; heterogeneous cluster where only some nodes support pinning.","solutions":["On the DataNode set dfs.datanode.max.locked.memory to a positive value within the OS limit and configure a RAM_DISK volume (e.g. [RAM_DISK]/mnt/ram_disk).","Raise the memlock ulimit for the DataNode process or systemd LimitMEMLOCK=infinity.","If pinning is optional, fall back to a normal (disk) storage policy instead of pinning.","Verify the node's health via DataNode JMX (lockedMemoryBudget) before routing lazy-persist writes to it."],"exampleFix":"// hdfs-site.xml (DataNode)\n<!-- before: no locked-memory budget -->\n\n<!-- after -->\n<property>\n  <name>dfs.datanode.max.locked.memory</name>\n  <value>1073741824</value>\n</property>\n<!-- and a RAM_DISK storage dir -->\n<property>\n  <name>dfs.datanode.data.dir</name>\n  <value>[RAM_DISK]/mnt/ram_disk,[DISK]/mnt/disk1</value>\n</property>","handlingStrategy":"fallback","validationCode":"// Before lazy-persist writes, confirm the target has locked-memory budget via DataNode JMX\n// (curl dn:9864/jmx -> FSDataset \"LockedMemoryUsed\"/capacity) or check dfs.datanode.max.locked.memory > 0 in config","typeGuard":null,"tryCatchPattern":"try {\n  out = dfs.create(path, EnumSet.of(CreateFlag.LAZY_PERSIST));\n} catch (BlockPinningException e) {\n  // node cannot pin in memory — fall back to normal DISK policy\n  out = dfs.create(path);\n}","preventionTips":["Size dfs.datanode.max.locked.memory and configure RAM_DISK volumes before enabling lazy persist.","Raise memlock limits (systemd LimitMEMLOCK / ulimit -l) on DataNode hosts.","Monitor locked-memory usage JMX so pinning requests do not outpace the budget."],"tags":["hdfs","datanode","lazy-persist","ram-disk","memory"],"backgroundTag":"resource-unavailable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}