rustfs/rustfs · error · io::Error

data block count exceeds available shards

Error message

data block count exceeds available shards

What it means

Write-stage guard in write_data_blocks: the stripe's decoded/verified block vector (en_blocks) has fewer entries than the erasure code's data_blocks, so the requested object range cannot be emitted and the GET fails as InvalidInput at the range stage (RangeOrLengthInvalid reason). In practice the caller's Erasure metadata and the assembled block set disagree — corrupt or mismatched object metadata, or an internal pipeline invariant break.

Source

Thrown at crates/ecstore/src/erasure/coding/decode.rs:1410

    en_blocks: &[Option<Vec<u8>>],
    data_blocks: usize,
    mut offset: usize,
    length: usize,
) -> std::io::Result<usize>
where
    W: tokio::io::AsyncWrite + Send + Sync + Unpin,
{
    if en_blocks.len() < data_blocks {
        let reason = GetObjectFailureReason::RangeOrLengthInvalid;
        record_get_object_pipeline_failure(GET_STAGE_RANGE, reason);
        error!(
            data_blocks,
            available_shards = en_blocks.len(),
            stage = GET_STAGE_RANGE,
            reason = reason.as_str(),
            "Write data blocks received fewer shards than data blocks"
        );
        return Err(io::Error::new(ErrorKind::InvalidInput, "data block count exceeds available shards"));
    }

    if length == 0 {
        return Ok(0);
    }

    let Some(required_len) = offset.checked_add(length) else {
        let reason = GetObjectFailureReason::RangeOrLengthInvalid;
        record_get_object_pipeline_failure(GET_STAGE_RANGE, reason);
        error!(
            offset,
            length,
            stage = GET_STAGE_RANGE,
            reason = reason.as_str(),
            "Write data blocks offset and length overflow"
        );
        return Err(io::Error::new(ErrorKind::InvalidInput, "offset + length overflows"));
    };

View on GitHub (pinned to 9e6e02ea09)

Solutions

1. Dump the object's xl.meta (cargo run -p rustfs-filemeta --example dump_fileinfo) and compare data_shards across shards
2. Heal the object or re-upload it so metadata is rewritten consistently
3. If it reproduces on new objects, verify callers pass a block set built from the same ErasureInfo as the Erasure instance
4. Confirm pool/set EC configuration has not changed underneath existing objects

// Caller-side pre-check mirroring the guard
if en_blocks.len() < data_blocks {
    return Err(io::Error::new(
        io::ErrorKind::InvalidInput,
        format!("expected at least {} blocks, got {}", data_blocks, en_blocks.len()),
    ));
}

match erasure.decode(...).await {
    (n, Some(e)) if e.kind() == std::io::ErrorKind::InvalidInput
        && e.to_string().contains("data block count") =>
    {
        // metadata/assembly mismatch: heal or re-upload; retrying unchanged will fail again
    }
    other => other,
}

When it happens

Trigger: xl.meta ErasureInfo with data_shards larger than the number of blocks the decode pipeline assembled; objects read cross-pool after EC settings changed; truncated xl.meta after a crash; a caller passing a short block vector.

Common situations: Corrupt xl.meta on some shards; mixed-version clusters mid-upgrade; heal writing inconsistent metadata; custom callers of Erasure::decode with mismatched inputs.

Related errors

• Invalid number of readers
• offset + length overflows
• Missing data block
• Invalid erasure coding parameters
• offset + length exceeds total length

AI-assisted analysis of rustfs/rustfs@9e6e02ea09 (2026-08-16). Data as JSON: /api/errors/d356b904d6b20baa. Report an issue: GitHub.