ducaale/xh · error · io::Error

truncated zstd skippable frame

Error message

truncated zstd skippable frame

What it means

The zstd decoder encountered a skippable frame (a non-data frame decoders should pass over) whose declared payload length extends past the end of the input stream. The reader copied fewer bytes than the frame header promised, which can only mean the compressed data was cut short mid-frame. The library surfaces this as UnexpectedEof so callers can distinguish 'input ended inside a skippable frame' from a clean end-of-stream.

Solutions

1. Verify the source .zst file/stream is complete and re-download or re-fetch it
2. Check the producer: ensure the compressor finished writing and the full stream was flushed/closed
3. Validate file size against the expected size (e.g. Content-Length or checksum) before decoding
4. If the input is intentionally truncated, treat UnexpectedEof from read as expected and handle it explicitly

Example fix

// before
let mut out = Vec::new();
decoder.read_to_end(&mut out)?; // panics/errors on truncated file

// after
match decoder.read_to_end(&mut out) {
    Ok(_) => {},
    Err(e) if e.kind() == io::ErrorKind::UnexpectedEof => {
        eprintln!("compressed input is truncated; refetching");
        // refetch or recover
    }
    Err(e) => return Err(e),
}

// Check input size against expected before decoding
let meta = std::fs::metadata(&path)?;
if meta.len() < expected_min_size {
    return Err(format!("{} is truncated ({} bytes, expected >= {})", path.display(), meta.len(), expected_min_size));
}

fn looks_truncated(e: &std::io::Error) -> bool {
    e.kind() == std::io::ErrorKind::UnexpectedEof
}

match decoder.read_to_end(&mut out) {
    Ok(_) => Ok(out),
    Err(e) if e.kind() == std::io::ErrorKind::UnexpectedEof => {
        // recover: refetch source or use last-good cached copy
        Err(DecodeError::Truncated(e))
    }
    Err(e) => Err(e.into()),
}

When it happens

Trigger: Calling Decoder::read (e.g. streaming decode of a .zst file or socket) on a truncated input: the file was cut off, a download was interrupted, or the writer died before flushing the full skippable frame declared in its header.

Common situations: Partially downloaded or interrupted .zst files; piping from a producer that crashed or was SIGPIPE'd; storing compressed data in a store that truncates trailing bytes; concatenating streams where the tail frame is incomplete.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• Could not create file after unreasonable number of attempts
• unknown compression type

AI-assisted analysis of ducaale/xh@2404aceecc (2026-09-13). Data as JSON: /api/errors/dd3f9f28b29ce50c. Report an issue: GitHub.

Appendix: source

Thrown at src/decoder.rs:262

            match self.state {
                ZstdDecoderState::NeedFrame => {
                    if self.reader.fill_buf()?.is_empty() {
                        self.state = ZstdDecoderState::Finished;
                        return Ok(0);
                    }

                    match self.decoder.reset(&mut self.reader) {
                        Ok(()) => self.state = ZstdDecoderState::Decoding,
                        Err(FrameDecoderError::ReadFrameHeaderError(
                            ReadFrameHeaderError::SkipFrame { length, .. },
                        )) => {
                            let length = u64::from(length);
                            let copied = {
                                let mut payload = self.reader.by_ref().take(length);
                                io::copy(&mut payload, &mut io::sink())?
                            };
                            if copied != length {
                                return Err(io::Error::new(
                                    io::ErrorKind::UnexpectedEof,
                                    "truncated zstd skippable frame",
                                ));
                            }
                        }
                        Err(err) => return Err(io::Error::other(err)),
                    }
                }
                ZstdDecoderState::Decoding => {
                    while self.decoder.can_collect() < buf.len() && !self.decoder.is_finished() {
                        let additional_bytes = buf.len() - self.decoder.can_collect();
                        self.decoder
                            .decode_blocks(
                                &mut self.reader,
                                BlockDecodingStrategy::UptoBytes(additional_bytes),
                            )
                            .map_err(io::Error::other)?;
                    }

View on GitHub (pinned to 2404aceecc)