influxdata/influxdb · critical · FormatError

buffer too short: expected at least

Error message

buffer too short: expected at least {expected} bytes, got {actual}

What it means

`FormatError::BufferTooShort` is returned when a buffer read from a catalog format file is shorter than the minimum bytes required for the expected structure (header, record, etc.). It indicates a truncated or malformed file/stream rather than a caller mistake.

Solutions

1. Restore the catalog file from a complete backup
2. Check storage integrity — re-copy or re-download the file and compare checksums/sizes
3. Investigate the writer/crash that truncated the file; ensure durable writes (fsync) are functioning

let meta = fs::metadata(&path)?;
if meta.len() < MIN_CATALOG_SIZE {
    return Err(format!("catalog file {path:?} looks truncated ({} bytes)", meta.len()));
}

match result {
    Err(FormatError::BufferTooShort { expected, actual }) => {
        log::error!("truncated catalog buffer: need {expected}, got {actual}");
        // restore from backup
    }
    r => r?,
}

When it happens

Trigger: Decoding a catalog file or record whose byte buffer has fewer bytes than the parser requires (expected/actual counts in the error).

Common situations: Truncated catalog files after a crash, interrupted write, or failed sync; partial file copies during backup/restore; network/object-store reads returning incomplete data.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• invalid magic bytes: expected
• record data exceeds file bounds: offset
• {0}
• catalog format error
• header CRC32 mismatch: expected

AI-assisted analysis of influxdata/influxdb@06200ef96b (2026-09-19). Data as JSON: /api/errors/303b902c302b6e80. Report an issue: GitHub.

Appendix: source

Thrown at influxdb3_catalog/src/format/mod.rs:179

}

/// Errors that can occur when working with the binary format.
#[derive(Debug, Clone, thiserror::Error)]
pub enum FormatError {
    /// A decoded record failed to apply to the catalog.
    #[error(transparent)]
    Apply(#[from] apply::ApplyError),

    /// Invalid magic bytes at start of file.
    #[error("invalid magic bytes: expected {expected:?}, got {actual:?}")]
    InvalidMagic { expected: [u8; 4], actual: [u8; 4] },

    /// Unsupported format version.
    #[error("unsupported format version: {version}")]
    UnsupportedVersion { version: u32 },

    /// Buffer too short for required data.
    #[error("buffer too short: expected at least {expected} bytes, got {actual}")]
    BufferTooShort { expected: usize, actual: usize },

    /// Header CRC32 checksum mismatch.
    #[error("header CRC32 mismatch: expected {expected:#010x}, actual {actual:#010x}")]
    HeaderCrc32Mismatch { expected: u32, actual: u32 },

    /// Payload CRC32 checksum mismatch.
    #[error("payload CRC32 mismatch: expected {expected:#010x}, computed {computed:#010x}")]
    Crc32Mismatch { expected: u32, computed: u32 },

    /// Unknown record type without UPGRADE_SAFE flag — hard error.
    #[error("unknown record id {record_id} without UPGRADE_SAFE flag")]
    UnknownNonUpgradeSafeRecord { record_id: u16 },

    /// Invalid record length.
    #[error("invalid record length: {length}")]
    InvalidRecordLength { length: u32 },

View on GitHub (pinned to 06200ef96b)