{"record":{"id":"16939c0f0d403f1c","repo":"apache/hadoop","slug":"key-length-out-of-range-klen","errorCode":null,"errorMessage":"Key length out of range: {klen}","messagePattern":"Key length out of range: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java","lineNumber":1617,"sourceCode":"        return (currentLocation.compareTo(endLocation) >= 0);\n      }\n\n      /**\n       * check whether we have already successfully obtained the key. It also\n       * initializes the valueInputStream.\n       */\n      void checkKey() throws IOException {\n        if (klen >= 0) return;\n        if (atEnd()) {\n          throw new EOFException(\"No key-value to read\");\n        }\n        klen = -1;\n        vlen = -1;\n        valueChecked = false;\n\n        klen = Utils.readVInt(blkReader);\n        if (klen < 0 || klen > MAX_KEY_SIZE) {\n          throw new IOException(\"Key length out of range: \" + klen);\n        }\n        blkReader.readFully(keyBuffer, 0, klen);\n        valueBufferInputStream.reset(blkReader);\n        if (valueBufferInputStream.isLastChunk()) {\n          vlen = valueBufferInputStream.getRemain();\n        }\n      }\n\n      /**\n       * Get an entry to access the key and value.\n       * \n       * @return The Entry object to access the key and value.\n       * @throws IOException raised on errors performing I/O.\n       */\n      public Entry entry() throws IOException {\n        checkKey();\n        return new Entry();\n      }","sourceCodeStart":1599,"sourceCodeEnd":1635,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java#L1599-L1635","documentation":"Thrown by TFile.Reader.Scanner.Entry.checkKey() while decoding the current record from a data block. Every TFile record starts with a VInt-encoded key length; the reader validates it against MAX_KEY_SIZE (64KB, TFile.java:154) and throws IOException when klen < 0 or klen > 65536. In practice it means the bytes at the reader's current block position are not a well-formed TFile record stream.","triggerScenarios":"Scanner iteration (atEnd(), advance(), any getKey*/getValue* call that forces checkKey()) where Utils.readVInt(blkReader) at the current block offset decodes to a negative value or a value above 65536. Typical causes: truncated or bit-rotted data block, a data block desynchronized from record boundaries, or opening a file that is not a TFile at all.","commonSituations":"Bad HDFS disks / corrupted replicas, jobs killed before TFile.Writer.close() so the file was never finalized, copying a SequenceFile or text file into a path then opening it as TFile, or reading through an unchecksummed transfer. Also seen when hand-rolling code that seeks into the middle of a TFile.","solutions":["Verify file integrity: run 'hdfs fsck -blocks' on the path and re-fetch a healthy replica, or compare against the original copy.","Regenerate the TFile from source data, ensuring Writer.close() (and the underlying stream close) completes before any reader opens it.","Confirm the file is really a TFile: BCFile magic check happens earlier, so a version/magic failure points elsewhere; a klen failure here points to data-block corruption specifically.","Write future files with BCFile checksums enabled so corruption is detected at the chunk level instead of surfacing as bogus lengths."],"exampleFix":"// before\nwhile (!scanner.atEnd()) { scanner.advance(); consume(scanner.entry()); }\n// after\ntry {\n  while (!scanner.atEnd()) { scanner.advance(); consume(scanner.entry()); }\n} catch (IOException e) {\n  // record stream is corrupt: stop, quarantine the file, rebuild from source\n  quarantine(path); throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  while (!scanner.atEnd()) { scanner.advance(); consume(scanner.entry()); }\n} catch (IOException e) {\n  // bogus klen = data-block corruption: quarantine, do not retry same bytes\n  quarantine(path); throw e;\n}","preventionTips":["Always close TFile.Writer before readers can see the file (temp name + atomic rename).","Enable BCFile checksums so corruption is detected as checksum failure, not bogus lengths.","Run hdfs fsck on paths before bulk-read jobs.","Keep keys within the documented 64KB MAX_KEY_SIZE when writing."],"tags":["tfile","hadoop-common","data-corruption","io","key-length"],"backgroundTag":"malformed-file-format","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}