{"record":{"id":"6d125b36fd02153a","repo":"apache/hadoop","slug":"file-had-invalid-length","errorCode":null,"errorMessage":"File {} had invalid length: {}","messagePattern":"File (.+?) had invalid length: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/BestEffortLongFile.java","lineNumber":92,"sourceCode":"    value = newVal;\n  }\n  \n  private void lazyOpen() throws IOException {\n    if (ch != null) {\n      return;\n    }\n\n    // Load current value.\n    byte[] data = null;\n    try {\n      data = Files.toByteArray(file);\n    } catch (FileNotFoundException fnfe) {\n      // Expected - this will use default value.\n    }\n\n    if (data != null && data.length != 0) {\n      if (data.length != Longs.BYTES) {\n        throw new IOException(\"File \" + file + \" had invalid length: \" +\n            data.length);\n      }\n      value = Longs.fromByteArray(data);\n    } else {\n      value = defaultVal;\n    }\n    \n    // Now open file for future writes.\n    RandomAccessFile raf = new RandomAccessFile(file, \"rw\");\n    try {\n      ch = raf.getChannel();\n    } finally {\n      if (ch == null) {\n        IOUtils.closeStream(raf);\n      }\n    }\n  }\n  ","sourceCodeStart":74,"sourceCodeEnd":110,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/BestEffortLongFile.java#L74-L110","documentation":"BestEffortLongFile persists one 64-bit long in raw binary form (e.g., the QJM Journal server's committed transaction id, see Journal.java:210). A missing or empty file is fine (default value is used), but on the first get()/set() the lazy loader throws this IOException when the file exists with any length other than exactly 8 bytes - a partial write cannot be interpreted as any valid long.","triggerScenarios":"First get()/set() (lazyOpen) after a JournalNode restart when the backing file (committed-txnid) exists with 1-7 or >8 bytes: torn write on crash (the class does not fsync or use atomic rename), disk corruption, full-disk partial allocation, or manual editing with a text editor.","commonSituations":"JournalNode storage directory after power loss or a killed process; storage on a filesystem that lost writes; an operator echoing text into the file; disk-full events during set().","solutions":["Confirm with 'ls -l <file>' - any size other than 0 or 8 is invalid.","Delete the corrupt file and restart the daemon: BestEffortLongFile is best-effort by design, so it re-initializes with the default and is rewritten on the next set() (verify the lost value is recoverable from edit logs / other journal nodes first).","If the value must be preserved, restore the 8-byte file from a storage-directory backup.","Fix the root cause: free disk space, move storage to a healthy local disk, and stop manual edits of journal files."],"exampleFix":"// before\nBestEffortLongFile committedTxnId = new BestEffortLongFile(file, 0);\nlong v = committedTxnId.get(); // throws 'had invalid length: 3' on a torn file\n\n// after - detect a torn file before open, quarantine it, fall back to default\nif (file.exists() && file.length() != 0 && file.length() != 8) {\n  LOG.warn(\"Quarantining corrupt {} ({} bytes)\", file, file.length());\n  Files.move(file.toPath(), file.toPath().resolveSibling(file.getName() + \".corrupt\"));\n}\nBestEffortLongFile committedTxnId = new BestEffortLongFile(file, 0);\nlong v = committedTxnId.get();","handlingStrategy":"validation","validationCode":"static void ensureValidLongFile(File f) throws IOException {\n  if (f.exists() && f.length() != 0 && f.length() != 8) {\n    throw new IOException(String.format(\n        \"Refusing to open %s: %d bytes (expected 0 or 8)\", f, f.length()));\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  return longFile.get();\n} catch (IOException e) { // 'had invalid length'\n  // value is best-effort by contract: quarantine and reinitialize with default\n  Files.move(file.toPath(), file.toPath().resolveSibling(file.getName() + \".corrupt\"));\n  longFile.close();\n  longFile = new BestEffortLongFile(file, defaultVal);\n  return longFile.get();\n}","preventionTips":["Keep journal/metadata storage on healthy local disks with journaling filesystems; avoid NFS for these files.","Monitor disk-full conditions - torn writes typically follow failed space allocation during set().","Back up storage directories so 8-byte metadata files can be restored rather than defaulted.","Never edit best-effort metadata files by hand; they are binary."],"tags":["filesystem","corruption","storage","journalnode","metadata"],"backgroundTag":"file-corruption-detected","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}