{"record":{"id":"1316d1a67ebe101e","repo":"apache/hadoop","slug":"tried-to-deserialize-bytes-of-data-newlength","errorCode":null,"errorMessage":"tried to deserialize {} bytes of data!  newLength must be non-negative.","messagePattern":"tried to deserialize (.+?) bytes of data!  newLength must be non-negative\\.","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java","lineNumber":355,"sourceCode":"  @Override\n  public String toString() {\n    try {\n      return decode(bytes, 0, length);\n    } catch (CharacterCodingException e) { \n      throw new RuntimeException(\"Should not have happened\", e);\n    }\n  }\n\n  @Override\n  public void readFields(DataInput in) throws IOException {\n    int newLength = WritableUtils.readVInt(in);\n    readWithKnownLength(in, newLength);\n  }\n\n  public void readFields(DataInput in, int maxLength) throws IOException {\n    int newLength = WritableUtils.readVInt(in);\n    if (newLength < 0) {\n      throw new IOException(\"tried to deserialize \" + newLength +\n          \" bytes of data!  newLength must be non-negative.\");\n    } else if (newLength >= maxLength) {\n      throw new IOException(\"tried to deserialize \" + newLength +\n          \" bytes of data, but maxLength = \" + maxLength);\n    }\n    readWithKnownLength(in, newLength);\n  }\n\n  /**\n   * Skips over one Text in the input.\n   * @param in input in.\n   * @throws IOException raised on errors performing I/O.\n   */\n  public static void skip(DataInput in) throws IOException {\n    int length = WritableUtils.readVInt(in);\n    WritableUtils.skipFully(in, length);\n  }\n","sourceCodeStart":337,"sourceCodeEnd":373,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/Text.java#L337-L373","documentation":"Text.readFields(DataInput, int maxLength) first reads the VInt-encoded length of the encoded bytes. Text.write never emits a negative length, so a negative VInt can only come from a damaged stream or from reading bytes that were not written by Text's layout; the guard fails fast with IOException(\"tried to deserialize N bytes of data! newLength must be non-negative.\") instead of attempting a huge allocation.","triggerScenarios":"Decoding a Text from a corrupt or truncated stream; reading at a wrong stream position (e.g., treating a value region as a Text, or field order drift between a custom write() and readFields()); a hostile/corrupt VInt from untrusted input data.","commonSituations":"Custom Writable record layouts where write/read field order diverged; corrupted shuffle or HDFS data; parsing bytes that were never Text-serialized; partially written files from killed writers.","solutions":["Check the producer's write() order mirrors the reader's readFields() order exactly.","Verify the stream/file integrity (checksums, `hadoop fs -text`) and re-run the producer if corrupt.","Use the maxLength overload on untrusted input so bad lengths fail with a clear message instead of OOM.","Log the stream position when it fires to locate which record is damaged."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  text.readFields(in, maxLength);\n} catch (IOException e) {\n  if (e.getMessage() != null\n      && e.getMessage().contains(\"newLength must be non-negative\")) {\n    throw new IOException(\"Corrupt stream at \" + in + \": negative Text length\", e);\n  }\n  throw e;\n}","preventionTips":["Mirror Text.write/readFields field order exactly in custom record layouts","Use the maxLength overload on any untrusted or network-sourced stream","Validate checksums after transfers; re-generate corrupt files instead of retrying reads"],"tags":["text","writable","deserialization","data-corruption","hadoop"],"backgroundTag":"data-corruption-detected","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}