vectordotdev/vector · error · io::Error

InvalidInput

InvalidInput

Error message

cannot extend a file through the truncation API

What it means

The disk-buffer v2 `AsyncFile` trait's `truncate` is intentionally shrink-only: it compares the requested size against the file's current length and returns `InvalidInput` ("cannot extend a file through the truncation API") if `size > current_size`, before calling `set_len`. This mirrors `ftruncate(2)` semantics the codebase chose to enforce explicitly, preventing accidental file growth that would corrupt the data-file layout.

Source

Thrown at lib/vector-buffers/src/variants/disk_v2/io.rs:298

/// Builds a set of `OpenOptions` for opening a file as readable.
fn open_readable_file_options() -> OpenOptions {
    let mut open_options = OpenOptions::new();
    open_options.read(true);
    open_options
}

impl AsyncFile for tokio::fs::File {
    async fn metadata(&self) -> io::Result<Metadata> {
        let metadata = self.metadata().await?;
        Ok(Metadata {
            len: metadata.len(),
        })
    }

    async fn truncate(&self, size: u64) -> io::Result<()> {
        let current_size = self.metadata().await?.len();
        if size > current_size {
            return Err(io::Error::new(
                io::ErrorKind::InvalidInput,
                "cannot extend a file through the truncation API",
            ));
        }

        self.set_len(size).await
    }

    async fn sync_all(&self) -> io::Result<()> {
        self.sync_all().await
    }
}

impl ReadableMemoryMap for memmap2::Mmap {}

impl ReadableMemoryMap for memmap2::MmapMut {}

impl WritableMemoryMap for memmap2::MmapMut {

View on GitHub (pinned to 3708c39b12)

Solutions

1. Check `metadata().await?.len` first and clamp: only call `truncate(size)` when `size <= current_len`; grow with `set_len` if growth is truly intended.
2. If the intent is to extend, call `File::set_len` directly instead of the truncation API.
3. Audit why the requested size exceeds the file — a stale checkpoint or external truncation is usually the real bug.

Example fix

// before
file.truncate(size).await?; // size may exceed len → InvalidInput

// after
let len = file.metadata().await?.len;
if size <= len {
    file.truncate(size).await?;
} else {
    file.set_len(size).await?; // explicit growth path
}

let len = file.metadata().await?.len;
if size <= len {
    file.truncate(size).await?;
} else {
    return Err(io::Error::new(
        io::ErrorKind::InvalidInput,
        format!("refusing to grow file (len {len}) to {size} via truncate"),
    ));
}

match file.truncate(size).await {
    Ok(()) => {}
    Err(e) if e.kind() == io::ErrorKind::InvalidInput => {
        // requested size exceeded current length — treat as stale checkpoint
        warn!(size, error = %e, "truncation skipped: would extend file");
    }
    Err(e) => return Err(e),
}

When it happens

Trigger: Library code (or a custom `AsyncFile` implementation) calling `truncate(n)` where `n` exceeds the file's current length — e.g. computing a truncation offset from a checkpoint that is ahead of the actual file size, or mixing up `truncate` with `resize`/`set_len`.

Common situations: Writing a new `AsyncFile`/filesystem backend for disk_v2 (tests, in-memory FS, alternate storage) and reusing the truncation path; checkpoint/replay logic drifting from the physical file length after an external process truncated the data file.

Related errors

• InvalidInput
• Reader encountered unrecoverable error: {e:?}
• Failed to bind to listener socket at path: {}. Err: {}
• effective reader file ID must be in the checkpoint window
• record was already validated

AI-assisted analysis of vectordotdev/vector@3708c39b12 (2026-08-20). Data as JSON: /api/errors/d67c1fabf7ffd5d3. Report an issue: GitHub.