xai-org/grok-build · error · std::io::Error

workflow artifact changed during open: {}

Error message

workflow artifact changed during open: {}

What it means

After opening with O_NOFOLLOW, read_bounded_nofollow re-checks the opened file's metadata. If the opened target is no longer a regular file or already exceeds `limit`, the file changed between the pre-open check and open — it returns InvalidData with this message rather than reading a swapped-in object.

Source

Thrown at crates/codegen/xai-grok-shell/src/session/workflow/store.rs:337

        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            format!(
                "workflow artifact exceeds {limit} bytes: {}",
                path.display()
            ),
        ));
    }
    let mut options = std::fs::OpenOptions::new();
    options.read(true);
    #[cfg(unix)]
    {
        use std::os::unix::fs::OpenOptionsExt;
        options.custom_flags(libc::O_NOFOLLOW);
    }
    let file = options.open(path)?;
    let opened = file.metadata()?;
    if !opened.is_file() || opened.len() > limit {
        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            format!("workflow artifact changed during open: {}", path.display()),
        ));
    }
    let mut bytes = Vec::with_capacity(opened.len() as usize);
    file.take(limit.saturating_add(1)).read_to_end(&mut bytes)?;
    if bytes.len() as u64 > limit {
        return Err(io::Error::new(
            io::ErrorKind::InvalidData,
            format!(
                "workflow artifact exceeds {limit} bytes: {}",
                path.display()
            ),
        ));
    }
    Ok(bytes)
}

View on GitHub (pinned to bc7f02eddd)

Solutions

1. Ensure only one process writes run state (file lock or single-owner actor)
2. Retry the load; transient races usually clear on a second attempt
3. Restore a stable artifact via atomic_write (rename is atomic so readers never see partial files)
4. Investigate the concurrent writer or tampering if it recurs

Example fix

// before
// direct writes to runs.json while another reader opens it -> race
// after
let bytes = (|| store.read_bounded_nofollow(&path, LIMIT))( ).or_else(|e| if e.kind()==BrokenPipe||e.kind()==InvalidData { retry_once() } else { Err(e) })?; // and all writers use atomic_write_replace

fn stable_snapshot<F>(read: F) -> std::io::Result<Vec<u8>> where F: Fn() -> std::io::Result<Vec<u8>> {
    const TRIES: usize = 3;
    (0..TRIES).find_map(|_| read().ok()).ok_or_else(|| {
        std::io::Error::new(std::io::ErrorKind::InvalidData, "artifact unstable across reads")
    })
}

let bytes = loop {
    match store.read_bounded_nofollow(&path, LIMIT) {
        Ok(b) => break b,
        Err(e) if e.kind() == std::io::ErrorKind::InvalidData
            && e.to_string().contains("changed during open") && retries < 3 => {
            retries += 1;
            tokio::time::sleep(Duration::from_millis(50)).await;
        }
        Err(e) => return Err(e.into()),
    }
};

When it happens

Trigger: Concurrent modification of the artifact during load_workflow_runs_sync: another process replaces the file (e.g. with a symlink or fifo) between symlink_metadata and open, or a writer truncates/extends it past the limit in that window.

Common situations: Two shell instances persisting the same run concurrently; an attacker racing the reader (TOCTOU exploit); a background compaction job rewriting artifacts while runs are being loaded.

Related errors

• destination already exists
• workflow artifact is not a regular file: {}
• workflow artifact exceeds {limit} bytes: {}
• journal changed during open
• process scope already closed; fetch killed{}

AI-assisted analysis of xai-org/grok-build@bc7f02eddd (2026-08-31). Data as JSON: /api/errors/0fb01f9caeac3e25. Report an issue: GitHub.