{"record":{"id":"08159040f7fd9ed5","repo":"apache/hadoop","slug":"stream-corrupted","errorCode":null,"errorMessage":"stream corrupted","messagePattern":"stream corrupted","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/CBZip2InputStream.java","lineNumber":1060,"sourceCode":"    }\n\n    final int[] cftab = this.data.cftab;\n    final int[] tt = this.data.initTT(this.last + 1);\n    final byte[] ll8 = this.data.ll8;\n    cftab[0] = 0;\n    System.arraycopy(this.data.unzftab, 0, cftab, 1, 256);\n\n    for (int i = 1, c = cftab[0]; i <= 256; i++) {\n      c += cftab[i];\n      cftab[i] = c;\n    }\n\n    for (int i = 0, lastShadow = this.last; i <= lastShadow; i++) {\n      tt[cftab[ll8[i] & 0xff]++] = i;\n    }\n\n    if ((this.origPtr < 0) || (this.origPtr >= tt.length)) {\n      throw new IOException(\"stream corrupted\");\n    }\n\n    this.su_tPos = tt[this.origPtr];\n    this.su_count = 0;\n    this.su_i2 = 0;\n    this.su_ch2 = 256; /* not a char and not EOF */\n\n    if (this.blockRandomised) {\n      this.su_rNToGo = 0;\n      this.su_rTPos = 0;\n      setupRandPartA();\n    } else {\n      setupNoRandPartA();\n    }\n  }\n\n  private void setupRandPartA() throws IOException {\n    if (this.su_i2 <= this.last) {","sourceCodeStart":1042,"sourceCodeEnd":1078,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/bzip2/CBZip2InputStream.java#L1042-L1078","documentation":"setupBlock() runs after a whole block is decoded: it builds the inverse Burrows-Wheeler transformation vector tt (size last+1) and then indexes it with origPtr, a 24-bit value read from the block header (this.origPtr = bsR(24) at the start of getAndMoveToFrontDecode). origPtr selects which rotation was the original input; it must lie in [0, tt.length). If it is negative or out of range, the stream's metadata contradicts its payload and 'stream corrupted' is thrown — an internally inconsistent (corrupt) bzip2 stream, as no valid encoder emits such a pointer.","triggerScenarios":"Any CBZip2InputStream/BZip2Codec read where, for a fully-decoded block, origPtr < 0 or origPtr >= tt.length (tt sized last+1): flipped bits in the 24-bit origPtr field, a block whose decoded length (last) disagrees with its header, or hand-crafted/modified bzip2 data.","commonSituations":"Silent data corruption on disk/network; archives patched or truncated-then-recombined incorrectly; fuzzed input reaching a production path; files that pass the magic-byte check but are otherwise random bytes.","solutions":["Verify the archive with bzip2 -t (or decompress it with the CLI) — if that fails too, restore the file from a good replica/backup.","Run checksum validation (CRC32/MD5 end-to-end) on transfer so corrupted files never reach decompression.","Investigate the storage path (failing disk, bad cable, checksum-less transfer mode like text-mode FTP) if multiple files show the same corruption.","Quarantine and report the exact path and byte offset; do not retry the same bytes."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (\"stream corrupted\".equals(e.getMessage())) {\n    quarantineAndAlert(path); // metadata/payload inconsistency, restore from backup\n  } else {\n    throw e;\n  }\n}","preventionTips":["Checksum before decompress; corrupt origPtr is silent until this late check.","Avoid byte-patching bzip2 files; regenerate them instead.","Keep good replicas/backups for archival compressed data."],"tags":["bzip2","compression","data-corruption","bwt","hadoop"],"backgroundTag":"compressed-data-corruption","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}