quickwit-oss/tantivy · error · io::Error

failed to read block content

Error message

failed to read block content

What it means

Thrown by read_block() after parsing the compression flag when the remaining reader length is less than the declared block_len (block length minus the 1-byte compress flag). The block claims more payload bytes than the file actually contains, so the read is aborted with UnexpectedEof — a truncated or misaligned SSTable.

Source

Thrown at sstable/src/block_reader.rs:102

                        }
                    }
                }
                1..=3 => {
                    return Err(io::Error::new(
                        io::ErrorKind::UnexpectedEof,
                        "failed to read block_len",
                    ));
                }
                _ => self.reader.read_u32() as usize,
            };
            if block_len <= 1 {
                return Ok(false);
            }
            let compress = self.reader.read_u8();
            let block_len = block_len - 1;

            if self.reader.len() < block_len {
                return Err(io::Error::new(
                    io::ErrorKind::UnexpectedEof,
                    "failed to read block content",
                ));
            }
            if compress == 1 {
                #[cfg(feature = "zstd-compression")]
                {
                    let required_capacity =
                        Decompressor::upper_bound(&self.reader[..block_len]).unwrap_or(1024 * 1024);
                    self.buffer.reserve(required_capacity);
                    Decompressor::new()?
                        .decompress_to_buffer(&self.reader[..block_len], &mut self.buffer)?;

                    self.reader.advance(block_len);
                }

                if cfg!(not(feature = "zstd-compression")) {
                    return Err(io::Error::new(

View on GitHub (pinned to b5d8deb80c)

Solutions

  1. Validate the SSTable file size against expected/manifest size and re-copy if short
  2. Restore the file from backup or rebuild the index
  3. Ensure block reads start at valid offsets from the block index, not arbitrary positions
  4. Confirm writer/reader format version compatibility

Example fix

// before
if self.reader.len() < block_len {
    return Err(io::Error::new(io::ErrorKind::UnexpectedEof, "failed to read block content"));
}
// after
if self.reader.len() < block_len {
    return Err(io::Error::new(io::ErrorKind::UnexpectedEof, format!("failed to read block content: need {block_len} bytes, have {}", self.reader.len())));
}
Defensive patterns

Strategy: validation

Validate before calling

fn block_fits_in_reader(reader: &BlockReader, block_start: usize) -> bool {
    // header: 1 byte count + 1 byte compress flag + up to 4 bytes len
    if block_start + 2 > reader.len() { return false; }
    let len_byte_count = reader.peek_u8_at(block_start) as usize;
    if len_byte_count != 0 && len_byte_count != 4 { return false; }
    let block_len = reader.peek_u32_at(block_start + 2) as usize;
    block_start + 2 + block_len <= reader.len()
}

Type guard

fn declared_block_fits(reader_len: usize, offset: usize, block_len: usize) -> bool {
    offset + block_len <= reader_len
}

Try / catch

match block_reader.read_block() {
    Err(e) if e.kind() == io::ErrorKind::UnexpectedEof
        && e.to_string() == "failed to read block content" => {
        eprintln!("block extends past EOF; SSTable is truncated");
        // restore file or abort scan
    }
    Err(e) => return Err(e.into()),
    Ok(done) => if !done { /* process block */ },
}

Prevention

When it happens

Trigger: Reading an SSTable block whose declared block_len exceeds the bytes remaining in the reader; happens on truncated files, wrong offsets into the file, or blocks from an incompatible format.

Common situations: Incomplete index file transfers; disk corruption cutting segment files short; seeking into the middle of a block and misinterpreting payload bytes as a header.

Related errors

AI-assisted analysis of quickwit-oss/tantivy@b5d8deb80c (2026-09-05). Data as JSON: /api/errors/fa03fbfecd3a5778. Report an issue: GitHub.