rustfs/rustfs · warning · io::Error

invalid index chunk type

Error message

invalid index chunk type

What it means

The first byte of the index buffer must be 0x99, the s2 skippable-index chunk type that starts the 4-byte frame header; load fails with InvalidData on anything else. This check intentionally runs on both storage forms: MinIO stores the frame header-stripped, so decode_minio_index_bytes first tries the raw bytes (failing here) and then retries with restore_index_headers re-adding the frame — an expected control-flow failure internally.

Source

Thrown at crates/rio-v2/src/s2_index.rs:173

            if idx == 0 {
                if info.uncompressed_offset != 0 {
                    return true;
                }
                continue;
            }
            if info.uncompressed_offset != self.info[idx - 1].uncompressed_offset + self.est_block_uncompressed {
                return true;
            }
        }
        false
    }

    fn load(mut bytes: &[u8]) -> io::Result<Self> {
        if bytes.len() <= SKIPPABLE_FRAME_HEADER + S2_INDEX_HEADER.len() + S2_INDEX_TRAILER.len() {
            return Err(io::Error::new(io::ErrorKind::UnexpectedEof, "buffer too small"));
        }
        if bytes[0] != CHUNK_TYPE_INDEX {
            return Err(io::Error::new(io::ErrorKind::InvalidData, "invalid index chunk type"));
        }

        let chunk_len = (bytes[1] as usize) | ((bytes[2] as usize) << 8) | ((bytes[3] as usize) << 16);
        bytes = &bytes[SKIPPABLE_FRAME_HEADER..];
        if bytes.len() < chunk_len {
            return Err(io::Error::new(io::ErrorKind::UnexpectedEof, "buffer too small"));
        }
        bytes = &bytes[..chunk_len];

        if !bytes.starts_with(S2_INDEX_HEADER) {
            return Err(io::Error::new(io::ErrorKind::InvalidData, "invalid index header"));
        }
        bytes = &bytes[S2_INDEX_HEADER.len()..];

        let (total_uncompressed, used) = read_varint(bytes)?;
        if total_uncompressed < 0 {
            return Err(io::Error::new(io::ErrorKind::InvalidData, "invalid uncompressed size"));
        }

View on GitHub (pinned to 9e6e02ea09)

Solutions

1. Use decode_minio_index_bytes, which transparently retries with restored headers, instead of parsing raw stored bytes yourself.
2. When validating manually, accept either form: byte 0 == 0x99 (full frame) or the stripped form that starts directly with the 's2idx' magic/varint payload.
3. Confirm the buffer really is the seek-index metadata value, not object data or another header.

fn is_full_s2_frame(b: &[u8]) -> bool { b.first() == Some(&0x99) }
fn is_stripped_s2_form(b: &[u8]) -> bool { b.starts_with(b"s2idx\x00") }

fn is_s2_index_payload(b: &[u8]) -> bool {
    is_full_s2_frame(b) || is_stripped_s2_form(b)
}

match decode_index(bytes) {
    Err(e) if e.kind() == io::ErrorKind::InvalidData => {
        // try the alternate storage form, then degrade to sequential reads
        decode_index(&restore_index_headers(bytes)).unwrap_or(None)
    }
    other => other.ok(),
}

When it happens

Trigger: Feeding the header-stripped MinIO storage form directly to the frame parser (expected on the first internal attempt; the restored-header retry succeeds); arbitrary or JSON bytes passed as an index; the wrong byte range sliced out of the object or metadata.

Common situations: Interop with MinIO-written metadata whose stored form lacks the 0x99 frame header; passing a legacy JSON index; unit tests with synthetic buffers that omit the chunk-type byte.

Related errors

• buffer too small
• invalid index header
• invalid uncompressed size
• invalid block size
• invalid number of entries

AI-assisted analysis of rustfs/rustfs@9e6e02ea09 (2026-08-16). Data as JSON: /api/errors/f4d6864fa626db40. Report an issue: GitHub.