BoundaryML/baml · error · io::Error

corrupt tail segment in profiling stream

Error message

corrupt tail segment in profiling stream

What it means

The profiling store's tail-segment reader finds the highest-numbered segment file on disk and validates its SHA-based checksum before reporting its sequence number. If validate_checksum rejects the bytes, the store raises InvalidData 'corrupt tail segment in profiling stream' because the last segment can no longer be trusted for sequencing.

Source

Thrown at baml_language/crates/bex_prof_store/src/prof/backend/store.rs:1030

        let Some(stem) = name.strip_suffix(&suffix) else {
            continue;
        };
        if stem.len() != 20 || !stem.bytes().all(|byte| byte.is_ascii_digit()) {
            continue;
        }
        let Ok(sequence) = stem.parse::<u64>() else {
            continue;
        };
        if highest.as_ref().is_none_or(|(high, _)| sequence > *high) {
            highest = Some((sequence, entry.path()));
        }
    }
    let Some((sequence, path)) = highest else {
        return Ok(0);
    };
    let bytes = fs::read(&path)?;
    validate_checksum(&bytes).map_err(|_| {
        io::Error::new(
            io::ErrorKind::InvalidData,
            "corrupt tail segment in profiling stream",
        )
    })?;
    Ok(sequence)
}

pub(crate) fn thread_ref_bytes(thread: ThreadRef) -> [u8; 32] {
    let mut bytes = [0u8; 32];
    bytes[..16].copy_from_slice(&thread.process_euid.0);
    bytes[16..24].copy_from_slice(&thread.engine_id.0.to_be_bytes());
    bytes[24..32].copy_from_slice(&thread.thread_id.0.to_be_bytes());
    bytes
}

#[derive(Clone, Copy, Debug, PartialEq, Eq)]
enum EncodeError {
    StringTooLong,

View on GitHub (pinned to bd85ce9dee)

Solutions

1. Delete or archive the corrupt tail segment file so recovery can fall back to the previous intact segment, then reopen the store.
2. Restore the profiling store directory from a known-good backup.
3. Check disk health/free space and rerun the workload after removing the damaged segment.
4. Avoid killing the process or copying the store directory while profiling writes are active.

Example fix

// before
let seq = store.highest_sequence()?; // fails: tail segment corrupt
// after
let seq = match store.highest_sequence() {
    Ok(seq) => seq,
    Err(e) if e.kind() == io::ErrorKind::InvalidData => {
        fs::remove_file(tail_path)?; // drop corrupt tail, fall back to prior segment
        store.highest_sequence()?
    }
};

let tail = find_highest_segment(&store_dir)?;
if let Some(bytes) = tail.and_then(|p| fs::read(p).ok()) {
    if validate_checksum(&bytes).is_err() {
        // quarantine the corrupt tail before opening the store
    }
}

match store.highest_sequence() {
    Ok(seq) => seq,
    Err(e) if e.kind() == io::ErrorKind::InvalidData && e.to_string().contains("corrupt tail segment") => {
        remove_tail_segment(&store_dir)?;
        store.highest_sequence()?
    }
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling the API that recovers the latest sequence number (fs::read of the highest segment path followed by validate_checksum) when the tail segment file is truncated, partially written after a crash, or otherwise fails its checksum.

Common situations: Process killed mid-write leaving a partial tail segment; disk corruption; copying or rsyncing the profiling store directory while writes were in flight; running out of disk space during segment flush.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• invalid profiling usage ledger
• profiling usage ledger checksum mismatch
• profiling store contains an unsupported file type
• no filesystem is attached to this server ({})
• no profile store at {0}

AI-assisted analysis of BoundaryML/baml@bd85ce9dee (2026-09-12). Data as JSON: /api/errors/88eacdb371966698. Report an issue: GitHub.