{"record":{"id":"10106063a2623818","repo":"apache/hadoop","slug":"offset-exceeds-file-length","errorCode":null,"errorMessage":"Offset: {} exceeds file length: {}","messagePattern":"Offset: (.+?) exceeds file length: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java","lineNumber":553,"sourceCode":"      return locatedBlocks.get(targetBlockIdx);\n    }\n  }\n\n  /**\n   * Get blocks in the specified range.\n   * Fetch them from the namenode if not cached. This function\n   * will not get a read request beyond the EOF.\n   * @param offset starting offset in file\n   * @param length length of data\n   * @return consequent segment of located blocks\n   * @throws IOException\n   */\n  private List<LocatedBlock> getBlockRange(long offset,\n      long length)  throws IOException {\n    // getFileLength(): returns total file length\n    // locatedBlocks.getFileLength(): returns length of completed blocks\n    if (offset >= getFileLength()) {\n      throw new IOException(\"Offset: \" + offset +\n        \" exceeds file length: \" + getFileLength());\n    }\n\n    synchronized(infoLock) {\n      final List<LocatedBlock> blocks;\n      final long lengthOfCompleteBlk = locatedBlocks.getFileLength();\n      final boolean readOffsetWithinCompleteBlk = offset < lengthOfCompleteBlk;\n      final boolean readLengthPastCompleteBlk = offset + length > lengthOfCompleteBlk;\n\n      if (readOffsetWithinCompleteBlk) {\n        //get the blocks of finalized (completed) block range\n        blocks = getFinalizedBlockRange(offset,\n          Math.min(length, lengthOfCompleteBlk - offset));\n      } else {\n        blocks = new ArrayList<>(1);\n      }\n\n      // get the blocks from incomplete block range","sourceCodeStart":535,"sourceCodeEnd":571,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L535-L571","documentation":"getBlockRange() guards the start of every read range: offset must be strictly below the stream's current file length (completed blocks plus last block being written). Reading at or beyond that throws this IOException. On a healthy file it indicates caller math errors; on a shrinking file it indicates a truncate raced the reader.","triggerScenarios":"read(buffer, off, len) or readFully positioned at offset >= getFileLength(); the sequential reader's pos advanced past the file's actual length after concurrent truncation.","commonSituations":"Split readers in compute frameworks computing offsets from stale file lengths; tailers racing log truncation; unit tests reading empty files with nonzero offsets.","solutions":["Validate offset+length against a fresh getFileStatus().getLen() before issuing reads","Return EOF cleanly for pos >= len instead of calling read at all","Re-open the stream after external truncate events to refresh locatedBlocks"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"long len = fs.getFileStatus(path).getLen();\nif (offset >= len) {\n  return -1; // EOF by contract\n}\n// only now issue the read that calls getBlockRange internally","typeGuard":null,"tryCatchPattern":"try {\n  in.readFully(buf, off, len);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"exceeds file length\")) {\n    // stale length: refresh and clamp\n    long cur = fs.getFileStatus(path).getLen();\n    in.readFully(buf, off, (int) Math.min(len, cur - in.getPos()));\n  } else throw e;\n}","preventionTips":["Compute read ranges from freshly fetched lengths, not cached ones","Handle pos >= len as EOF before calling read","Re-open streams after external truncate events"],"tags":["hdfs","offset","validation","read"],"backgroundTag":"offset-out-of-range","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}