xai-org/grok-build · error

reference snapshot did not match expected sha256; upload ski

Error message

reference snapshot did not match expected sha256; upload skipped

What it means

This error is produced by the background upload queue when a reference snapshot's SHA-256 no longer matches the expected digest (SnapshotCheck::Stale). The queue detects that the file content changed since the snapshot was taken and refuses to upload stale data, notifying the caller's completion channel with this error instead. It indicates an integrity guard, not a transport failure: the upload is intentionally skipped to avoid publishing content that no longer matches what the caller snapshotted.

Source

Thrown at crates/codegen/xai-file-utils/src/queue.rs:1220

        };
        let snapshot = self
            .queue_dir
            .join(temp_file_name(artifact_name, session_id, turn_number));
        let disk_bytes = match reflink_copy::reflink_or_copy(source_path, &snapshot) {
            Ok(copied) => copied.unwrap_or(0),
            Err(e) => {
                return Err(anyhow::Error::new(e).context(format!(
                    "Failed to snapshot {} into upload queue",
                    source_path.display()
                )));
            }
        };
        match check_snapshot(&snapshot, expected_sha256) {
            SnapshotCheck::Match => {}
            SnapshotCheck::Stale => {
                try_remove_temp(&snapshot, Some(&self.stats));
                self.stats.reference_stale.fetch_add(1, Ordering::Relaxed);
                let _ = tx.send(Err(anyhow::anyhow!(
                    "reference snapshot did not match expected sha256; upload skipped"
                )));
                return Ok(EnqueueResult {
                    completion_rx: rx,
                    original_size,
                });
            }
            SnapshotCheck::Io(e) => {
                try_remove_temp(&snapshot, Some(&self.stats));
                self.stats.failed.fetch_add(1, Ordering::Relaxed);
                let _ = tx.send(Err(e));
                return Ok(EnqueueResult {
                    completion_rx: rx,
                    original_size,
                });
            }
        }
        tracing::debug!(

View on GitHub (pinned to bc7f02eddd)

Solutions

  1. Do not modify or delete the source/temp snapshot file until the completion receiver resolves; hold it for the duration of the queue wait.
  2. Take a fresh snapshot and re-enqueue the upload after all writes to the file have completed.
  3. Check completion_rx for this error and treat it as 'upload skipped' rather than a network failure; retry the whole enqueue operation.
  4. Ensure only one writer owns the snapshot at a time (use unique temp paths per upload instead of shared reference files).

Example fix

// before
let result = queue.enqueue(path).await;
std::fs::write(path, new_content)?; // mutates snapshot before upload

// after
let result = queue.enqueue(path).await;
let outcome = result.completion_rx.await; // wait before touching the file
std::fs::write(path, new_content)?;
Defensive patterns

Strategy: validation

Validate before calling

// Verify snapshot integrity before enqueue
fn ensure_snapshot_matches(path: &Path, expected_sha256: &str) -> anyhow::Result<()> {
    let bytes = std::fs::read(path)?;
    let digest = sha2::Sha256::digest(&bytes);
    if format!("{:x}", digest) != expected_sha256 {
        anyhow::bail!("snapshot changed on disk; re-snapshot before enqueue");
    }
    Ok(())
}

Prevention

When it happens

Trigger: Enqueueing a file whose on-disk bytes (or temp snapshot) were modified or truncated between snapshot creation and the SHA-256 verification at upload time; calling to_url/builder_for/client_for flows that feed the queue with a snapshot that was rotated, re-written, or removed before upload; concurrent writes to the same temp file while a queued upload is pending.

Common situations: Two processes writing the same temp/reference file concurrently; a build or formatter regenerating the reference right after it was queued; a snapshot kept alive past its validity window while the source file changed underneath; disk-level partial writes corrupting the temp snapshot.

Related errors


AI-assisted analysis of xai-org/grok-build@bc7f02eddd (2026-08-31). Data as JSON: /api/errors/54e9994333a9136f. Report an issue: GitHub.