{"record":{"id":"61386a53c536000a","repo":"apache/hadoop","slug":"offset-out-of-the-range-0-path-61386a","errorCode":null,"errorMessage":"Offset={} out of the range [0, {}); {}, path={}","messagePattern":"Offset=(.+?) out of the range \\[0, (.+?)\\); (.+?), path=(.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterWebHdfsMethods.java","lineNumber":521,"sourceCode":"      } else if (op == PutOpParam.Op.CREATE && !ns.equals(resolvedNs)) {\n        // for CREATE, the dest dn should be in the resolved ns\n        excludes.add(dn);\n      }\n    }\n\n    if (op == GetOpParam.Op.OPEN ||\n        op == PostOpParam.Op.APPEND ||\n        op == GetOpParam.Op.GETFILECHECKSUM) {\n      // Choose a datanode containing a replica\n      final ClientProtocol cp = getRpcClientProtocol();\n      final HdfsFileStatus status = cp.getFileInfo(path);\n      if (status == null) {\n        throw new FileNotFoundException(\"File \" + path + \" not found.\");\n      }\n      final long len = status.getLen();\n      if (op == GetOpParam.Op.OPEN) {\n        if (openOffset < 0L || (openOffset >= len && len > 0)) {\n          throw new IOException(\"Offset=\" + openOffset\n              + \" out of the range [0, \" + len + \"); \" + op + \", path=\" + path);\n        }\n      }\n\n      if (len > 0) {\n        final long offset = op == GetOpParam.Op.OPEN ? openOffset : len - 1;\n        final LocatedBlocks locations = cp.getBlockLocations(path, offset, 1);\n        final int count = locations.locatedBlockCount();\n        if (count > 0) {\n          LocatedBlock location0 = locations.get(0);\n          return bestNode(location0.getLocations(), excludes);\n        }\n      }\n    }\n\n    return getRandomDatanode(dns, excludes);\n  }\n","sourceCodeStart":503,"sourceCodeEnd":539,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/java/org/apache/hadoop/hdfs/server/federation/router/RouterWebHdfsMethods.java#L503-L539","documentation":"RouterWebHdfsMethods.chooseDatanode validates the client-supplied offset for WebHDFS OPEN: it must satisfy 0 <= offset < fileLength (when length > 0). A negative offset, or one at/after EOF, throws IOException with the offending offset, the valid range, the op and the path. This mirrors the NameNode-side check but runs on the Router before it picks a replica DataNode for the redirect.","triggerScenarios":"WebHDFS OPEN with offset=<negative> or offset >= length; reading at EOF instead of using length as end-of-stream signal; clients computing offsets from a stale file length after the file was truncated/rewritten.","commonSituations":"Custom readers that request offset == len to 'probe' EOF; files truncated concurrently so the cached length is now larger than reality; unit tests passing 0-length and boundary offsets.","solutions":["Clamp the request to 0 <= offset < len, fetching the current length via GETFILESTATUS first","Treat offset == len as clean EOF on the client side instead of issuing a request","Re-read the file status after concurrent writes/truncations and recompute offsets from the fresh length"],"exampleFix":"// before\nInputStream in = openPath(urlWithOffset(\"offset=\" + offset)); // offset may be >= len\n\n// after\nHdfsFileStatus st = getFileInfo(path);\nlong len = st.getLen();\nif (offset < 0 || (offset >= len && len > 0)) {\n  offset = Math.max(0, len - 1); // or signal EOF to the caller\n}\nInputStream in = openPath(urlWithOffset(\"offset=\" + offset));","handlingStrategy":"validation","validationCode":"HdfsFileStatus st = cp.getFileInfo(path);\nlong len = (st == null) ? 0 : st.getLen();\nboolean valid = offset >= 0 && (len == 0 || offset < len);\nif (!valid) {\n  // treat as EOF or bad request; do not issue the WebHDFS OPEN\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage().startsWith(\"Offset=\")\n      && e.getMessage().contains(\"out of the range\")) {\n    // refresh file length and re-clamp offset before retrying once\n  }\n}","preventionTips":["Always fetch current file length before computing read offsets","Handle offset == length as EOF client-side instead of sending it to the server","In tests, cover offset boundaries 0, len-1, len for WebHDFS reads"],"tags":["hadoop","hdfs","rbf","webhdfs","offset","range-validation","open"],"backgroundTag":"offset-out-of-range","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}