{"record":{"id":"e01b6e72da29701a","repo":"apache/hadoop","slug":"value-length-unknown","errorCode":null,"errorMessage":"Value length unknown.","messagePattern":"Value length unknown\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java","lineNumber":1835,"sourceCode":"         * @return The input stream.\n         */\n        public DataInputStream getKeyStream() {\n          keyDataInputStream.reset(keyBuffer, klen);\n          return keyDataInputStream;\n        }\n\n        /**\n         * Get the length of the value. isValueLengthKnown() must be tested\n         * true.\n         * \n         * @return the length of the value.\n         */\n        public int getValueLength() {\n          if (vlen >= 0) {\n            return vlen;\n          }\n\n          throw new RuntimeException(\"Value length unknown.\");\n        }\n\n        /**\n         * Copy value into user-supplied buffer. User supplied buffer must be\n         * large enough to hold the whole value. The value part of the key-value\n         * pair pointed by the current cursor is not cached and can only be\n         * examined once. Calling any of the following functions more than once\n         * without moving the cursor will result in exception:\n         * {@link #getValue(byte[])}, {@link #getValue(byte[], int)},\n         * {@link #getValueStream}.\n         *\n         * @param buf buf.\n         * @return the length of the value. Does not require\n         *         isValueLengthKnown() to be true.\n         * @throws IOException raised on errors performing I/O.\n         * \n         */\n        public int getValue(byte[] buf) throws IOException {","sourceCodeStart":1817,"sourceCodeEnd":1853,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java#L1817-L1853","documentation":"RuntimeException from Scanner.Entry.getValueLength() when vlen < 0. TFile stores large values in compressed chunks; when the total value length was not recorded at write time it is unknown until the whole value has been consumed. The Javadoc contract is explicit: isValueLengthKnown() must be tested true before calling getValueLength().","triggerScenarios":"Calling entry.getValueLength() on an entry whose value was chunked, i.e. where checkKey() found valueBufferInputStream.isLastChunk() false and left vlen = -1. Values larger than the chunk threshold or written through the streaming append API hit this.","commonSituations":"Mixed-size payloads: small values work (length known) so code passes testing, then the first multi-chunk value (typically > 64KB before compression) blows up in production. Common in map-side join files and pre-HFile TFile users.","solutions":["Branch on entry.isValueLengthKnown(); only call getValueLength() in the true branch.","When unknown, get the actual length from the return value of entry.getValue(buf) (it returns the number of bytes read) or stream via getValueStream().","If you control the writer and always need lengths, write values with the whole-value append (Writer.append(key, value)) so lengths are advertised."],"exampleFix":"// before\nint vlen = entry.getValueLength(); // RuntimeException on chunked values\n// after\nif (entry.isValueLengthKnown()) {\n  int vlen = entry.getValueLength();\n} else {\n  int vlen = entry.getValue(buf, 0); // returns actual bytes read\n}","handlingStrategy":"validation","validationCode":"int vlen;\nif (entry.isValueLengthKnown()) {\n  vlen = entry.getValueLength();\n} else {\n  vlen = -1; // get actual size from getValue(buf) return or streaming\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never call getValueLength() without testing isValueLengthKnown() first.","Treat isValueLengthKnown() == false as a first-class case in your reader design, not an error path.","Write values whole via Writer.append(key, value) when readers need lengths up front."],"tags":["tfile","hadoop-common","precondition","value-length","runtime-exception"],"backgroundTag":"api-precondition-violated","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}