{"record":{"id":"7e96b5d75734975c","repo":"apache/hadoop","slug":"corrupted-data-negative-string-length-length","errorCode":null,"errorMessage":"Corrupted data: negative string length {length}","messagePattern":"Corrupted data: negative string length (.+?)","errorType":"exception","errorClass":"NegativeArraySizeException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Utils.java","lineNumber":301,"sourceCode":"   * lengths that are -2 or less. or larger than the supplied bound before any\n   * buffer is allocated.\n   * A length of -1 means \"no data\" and mapped to a null string.\n   *\n   * @param in The input stream.\n   * @param maxLength The largest permitted encoded length in bytes, negative for no limit.\n   * @return The string or null.\n   * @throws EOFException input data length exceeds {@code maxLength}.\n   * @throws IOException IO failure.\n   * @throws NegativeArraySizeException string length was minus two or less.\n   */\n  public static String readString(DataInput in, int maxLength)\n      throws IOException {\n    int length = readVInt(in);\n    if (length == -1) {\n      return null;\n    }\n    if (length < 0) {\n      throw new NegativeArraySizeException(\"Corrupted data: negative string length \"\n          + length);\n    }\n    if (maxLength >= 0 && length > maxLength) {\n      throw new EOFException(\"String length \" + length\n          + \" exceeds the limit of \" + maxLength);\n    }\n    byte[] buffer = new byte[length];\n    in.readFully(buffer);\n    return Text.decode(buffer);\n  }\n\n  /**\n   * A generic Version class. We suggest applications built on top of TFile use\n   * this class to maintain version information in their meta blocks.\n   * \n   * A version number consists of a major version and a minor version. The\n   * suggested usage of major and minor version number is to increment major\n   * version number when the new storage format is not backward compatible, and","sourceCodeStart":283,"sourceCodeEnd":319,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Utils.java#L283-L319","documentation":"Thrown by TFile's Utils.readString(DataInput, int): the method reads a VInt length prefix where -1 encodes null, so any other negative value is impossible for a real string and means the underlying bytes are corrupt. Hadoop deliberately throws NegativeArraySizeException (documented on the method) instead of attempting new byte[negativeLength], which would throw the same exception with no context. It signals stream corruption or a mispositioned reader, never a transient condition.","triggerScenarios":"Calling Utils.readString — directly or via TFile Reader meta-block parsing (comparator names, version strings written with Utils.writeString) — on a DataInput positioned at bytes that are not a length-prefixed string; reading a TFile that was truncated, partially written, mis-transferred, or written by an incompatible format/version.","commonSituations":"TFile/sequence output from a killed or crashed job read as if complete; block/key offset math off by a few bytes so the reader lands mid-record; files copied without checksum verification picking up bit corruption; mixing TFile format versions between writer and reader.","solutions":["Treat the input as corrupt: re-transfer or regenerate the file from source data, verifying checksums (HDFS CRC / distcp verification) along the way.","Confirm reader positioning: the DataInput must sit exactly where the writer wrote the string; audit offset/length arithmetic that advanced the stream before readString.","Reproduce with a minimal reader to find the first bad offset; if the file came from an interrupted write, it is simply incomplete and must be re-produced.","Check that writer and reader use the same TFile format version and the same string encoding (Utils.writeString vs raw Text)."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  String s = Utils.readString(in, maxLength);\n} catch (NegativeArraySizeException e) {\n  // stream is corrupt: quarantine the file; do NOT retry the same bytes\n  throw new IOException(\"Corrupt TFile input near offset \" + inOffset, e);\n}","preventionTips":["Write TFiles atomically (temp name + rename on success) so killed jobs never leave half-written files.","Verify checksums whenever TFiles move between systems.","Never share a DataInput across threads without synchronization — misaligned reads produce exactly this error."],"tags":["tfile","data-corruption","hadoop-common","stream-deserialization"],"backgroundTag":"corrupt-stream-data","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}