apache/hadoop · error · IOException

Cannot seek in {codecName} compressed stream

Error message

Cannot seek in {codecName} compressed stream

What it means

Error "Cannot seek in {codecName} compressed stream" thrown in apache/hadoop.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/LineRecordReader.java:121

      CompressionCodec codec = new CompressionCodecFactory(job).getCodec(file);
      if (null!=codec) {
        isCompressedInput = true;
        decompressor = CodecPool.getDecompressor(codec);
        if (codec instanceof SplittableCompressionCodec) {
          final SplitCompressionInputStream cIn =
                  ((SplittableCompressionCodec)codec).createInputStream(
                          fileIn, decompressor, start, end,
                          SplittableCompressionCodec.READ_MODE.BYBLOCK);
          in = new CompressedSplitLineReader(cIn, job,
                  this.recordDelimiterBytes);
          start = cIn.getAdjustedStart();
          end = cIn.getAdjustedEnd();
          filePosition = cIn;
        } else {
          if (start != 0) {
            // So we have a split that is only part of a file stored using
            // a Compression codec that cannot be split.
            throw new IOException("Cannot seek in " +
                    codec.getClass().getSimpleName() + " compressed stream");
          }

          in = new SplitLineReader(codec.createInputStream(fileIn,
                  decompressor), job, this.recordDelimiterBytes);
          filePosition = fileIn;
        }
      } else {
        fileIn.seek(start);
        in = new UncompressedSplitLineReader(
                fileIn, job, this.recordDelimiterBytes, split.getLength());
        filePosition = fileIn;
      }
      // If this is not the first split, we always throw away first record
      // because we always (except the last split) read one extra line in
      // next() method.
      if (start != 0) {
        start += in.readLine(new Text(), 0, maxBytesToConsume(start));

View on GitHub (pinned to 2add963021)

Solutions

1. Use a splittable codec (e.g. bzip2) for compressed input that must be seeked, or process the stream without seeking.

When it happens

Trigger: Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/LineRecordReader.java:121 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0fcafe20bf56b0a8. Report an issue: GitHub.