{"record":{"id":"c8d68d0a5c248a11","repo":"apache/hadoop","slug":"incorrect-key-length-expected-expectedlength-ac","errorCode":null,"errorMessage":"Incorrect key length: expected={expectedLength} actual={len}","messagePattern":"Incorrect key length: expected=(.+?) actual=(.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java","lineNumber":459,"sourceCode":"        }\n        expectedLength = len;\n      }\n\n      @Override\n      public void close() throws IOException {\n        if (closed == true) {\n          return;\n        }\n\n        try {\n          ++errorCount;\n          byte[] key = currentKeyBufferOS.getBuffer();\n          int len = currentKeyBufferOS.size();\n          /**\n           * verify length.\n           */\n          if (expectedLength >= 0 && expectedLength != len) {\n            throw new IOException(\"Incorrect key length: expected=\"\n                + expectedLength + \" actual=\" + len);\n          }\n\n          Utils.writeVInt(blkAppender, len);\n          blkAppender.write(key, 0, len);\n          if (tfileIndex.getFirstKey() == null) {\n            tfileIndex.setFirstKey(key, 0, len);\n          }\n\n          if (tfileMeta.isSorted() && tfileMeta.getRecordCount()>0) {\n            byte[] lastKey = lastKeyBufferOS.getBuffer();\n            int lastLen = lastKeyBufferOS.size();\n            if (tfileMeta.getComparator().compare(key, 0, len, lastKey, 0,\n                lastLen) < 0) {\n              throw new IOException(\"Keys are not added in sorted order\");\n            }\n          }\n","sourceCodeStart":441,"sourceCodeEnd":477,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/TFile.java#L441-L477","documentation":"Thrown when the stream returned by TFile.Writer.prepareAppendKey(expectedLength) is closed but the number of bytes actually written differs from expectedLength (which must be -1 for unknown length). The KeyRegister verifies the count on close and raises IOException, after which the writer is in an inconsistent state and only close() is legitimate. This is a contract check: pre-declaring the key size lets TFile write the length prefix without buffering.","triggerScenarios":"Calling prepareAppendKey(n) with n >= 0 and writing fewer or more than n bytes to the returned DataOutputStream before closing it; serializing an object whose advertised size does not match the bytes emitted (e.g. Writable.getLength() inconsistent with write()).","commonSituations":"Copying an expected-length value from another record type; changing key serialization without updating the declared length; mixing up key and value lengths; defensive code that guesses a length instead of passing -1.","solutions":["Pass -1 to prepareAppendKey when the key length is not known exactly","When pre-declaring a length, write exactly that many bytes (usually by serializing into a ByteArrayOutputStream first and using its size())","After this IOException, only call writer.close(); do not attempt further appends on the corrupted writer"],"exampleFix":"// before\nDataOutputStream kout = writer.prepareAppendKey(8);\nkout.write(keyBytes); // keyBytes.length != 8\nkout.close();\n\n// after\nDataOutputStream kout = writer.prepareAppendKey(keyBytes.length);\nkout.write(keyBytes, 0, keyBytes.length);\nkout.close();","handlingStrategy":"validation","validationCode":"// Serialize first so the declared length always matches the bytes\nbyte[] keyBytes = serializeKey(record);\ntry (DataOutputStream kout = writer.prepareAppendKey(keyBytes.length)) {\n  kout.write(keyBytes);\n}\n// Or, when length is unknown up front:\ntry (DataOutputStream kout = writer.prepareAppendKey(-1)) {\n  kout.write(keyBytes);\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Incorrect key length\")) {\n    // writer is now inconsistent: stop appending and close the writer\n  }\n}","preventionTips":["Default to prepareAppendKey(-1) unless pre-declaring the length buys measurable throughput","When pre-declaring, derive the length from the exact byte array you will write, not from a separate size estimate","Unit-test key serialization so getLength()-style sizes always equal written bytes"],"tags":["hadoop","tfile","length-mismatch","writer"],"backgroundTag":"length-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}