apache/hadoop · error · IOException

stored gzip size doesn't match decompressed size

Error message

stored gzip size doesn't match decompressed size

What it means

Error "stored gzip size doesn't match decompressed size" thrown in apache/hadoop.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/zlib/BuiltInGzipDecompressor.java:405

        crc.reset();
        state = GzipStateLabel.TRAILER_SIZE;
      }
    }

    if (userBufLen <= 0) {
      return;
    }

    // verify that the mod-2^32 decompressed stream size matches the value
    // stored in the gzip trailer
    if (state == GzipStateLabel.TRAILER_SIZE) {
      assert (localBufOff < 4);  // initially 0, but may need multiple calls
      int n = Math.min(userBufLen, 4-localBufOff);
      copyBytesToLocal(n);       // modifies userBufLen, etc.
      if (localBufOff >= 4) {    // should be strictly ==
        long inputSize = readUIntLE(localBuf, 0);
        if (inputSize != (inflater.getBytesWritten() & 0xffffffffL)) {
          throw new IOException(
            "stored gzip size doesn't match decompressed size");
        }
        localBufOff = 0;
        state = GzipStateLabel.FINISHED;
      }
    }

    if (state == GzipStateLabel.FINISHED) {
      return;
    }
  }

  /**
   * Returns the total number of compressed bytes input so far, including
   * gzip header/trailer bytes.
   *
   * @return the total (non-negative) number of compressed bytes read so far
   */

View on GitHub (pinned to 2add963021)

Solutions

1. The ISIZE trailer does not match the decompressed byte count; the file is truncated or corrupt. Re-obtain the file.
2. Make sure the writer closed the GZIPOutputStream so the trailer was written correctly.

When it happens

Trigger: Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/zlib/BuiltInGzipDecompressor.java:405 when the library encounters an invalid state.

Common situations: Occurs when the size stored in the gzip trailer does not match the bytes actually decompressed, usually due to corruption or truncation. Validate the source file and treat as unrecoverable corrupt input.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c2aa173b06b7802b. Report an issue: GitHub.