xai-org/grok-build · warning · io::Error

journal changed during open

Error message

journal changed during open

What it means

read_journal_bounded opens the journal file with O_NOFOLLOW, then re-statfs the opened fd and rejects it if it is not a regular file or its size already exceeds MAX_JOURNAL_BYTES. This detects a TOCTOU race: the file was replaced, truncated, or swapped between path resolution and metadata read, so the content about to be loaded cannot be trusted. The library throws std::io::Error with ErrorKind::InvalidData.

Source

Thrown at crates/codegen/xai-workflow/src/journal.rs:279

        ));
    }
    if metadata.len() > MAX_JOURNAL_BYTES {
        return Err(std::io::Error::new(
            std::io::ErrorKind::InvalidData,
            format!("journal exceeds {MAX_JOURNAL_BYTES} bytes"),
        ));
    }
    let mut options = std::fs::OpenOptions::new();
    options.read(true);
    #[cfg(unix)]
    {
        use std::os::unix::fs::OpenOptionsExt;
        options.custom_flags(libc::O_NOFOLLOW);
    }
    let file = options.open(path)?;
    let opened = file.metadata()?;
    if !opened.is_file() || opened.len() > MAX_JOURNAL_BYTES {
        return Err(std::io::Error::new(
            std::io::ErrorKind::InvalidData,
            "journal changed during open",
        ));
    }
    let mut content = Vec::with_capacity(opened.len() as usize);
    file.take(MAX_JOURNAL_BYTES.saturating_add(1))
        .read_to_end(&mut content)?;
    if content.len() as u64 > MAX_JOURNAL_BYTES {
        return Err(std::io::Error::new(
            std::io::ErrorKind::InvalidData,
            format!("journal exceeds {MAX_JOURNAL_BYTES} bytes"),
        ));
    }
    Ok(content)
}

fn validate_sequence(entries: &[JournalEntry], entry: &JournalEntry) -> Result<(), JournalError> {
    let expected = entries.len() as u64;

View on GitHub (pinned to bc7f02eddd)

Solutions

1. Retry load() after a short backoff - the race is transient and a fresh open usually sees a stable file
2. Ensure only one process owns the journal (use a lockfile or flock before load)
3. Exclude the journal path from rotators/sync tools, or point it at a private directory
4. If it reproduces deterministically, inspect the path for symlinks or non-regular files (ls -la, stat)

Example fix

// before
let journal = Journal::load(path)?;
// after
let journal = match Journal::load(path) {
    Ok(j) => j,
    Err(e) if e.kind() == std::io::ErrorKind::InvalidData
        && e.to_string().contains("changed during open") =>
    {
        std::thread::sleep(std::time::Duration::from_millis(50));
        Journal::load(path)?
    }
    Err(e) => return Err(e),
};

let md = std::fs::metadata(path)?;
if !md.is_file() { return Err(anyhow!("journal path is not a regular file")); }
// note: size check alone cannot close the race; retry handles it

fn is_transient_journal_race(e: &std::io::Error) -> bool {
    e.kind() == std::io::ErrorKind::InvalidData
        && e.to_string().contains("changed during open")
}

match Journal::load(path) {
    Ok(j) => j,
    Err(e) if is_transient_journal_race(&e) => retry_with_backoff(3, || Journal::load(path))?,
    Err(e) => return Err(e.into()),
}

When it happens

Trigger: Calling load() (which calls read_journal_bounded) while another process/thread concurrently writes, truncates, replaces, or rotates the journal file; the path being swapped (e.g. mv/rename over it) during the open window; a non-regular file (device, fifo) substituted at the path.

Common situations: Two agent processes sharing the same journal directory; a log-rotation or cleanup job racing with startup; editors or sync tools (Dropbox/rsync) replacing the file; running against a stale symlinked path.

Related errors

• failed to read {}: {e}
• failed to open {}: {e}
• failed to replace {}: {e}
• destination already exists
• workflow artifact changed during open: {}

AI-assisted analysis of xai-org/grok-build@bc7f02eddd (2026-08-31). Data as JSON: /api/errors/46d21fc569358ded. Report an issue: GitHub.