clockworklabs/SpacetimeDB · error · SnapshotError::ReadObject

missing object {}

Error message

missing object {}

What it means

object_file_path in the snapshot remote code decides where each object (blob/page) goes. If the object already exists locally or can be hardlinked from a parent repo, it is reused; otherwise, in dry-run mode nothing can be fetched, so the missing object is reported as SnapshotError::ReadObject with a NotFound cause naming the expected path. Non-dry-run returns the path so the object is downloaded instead.

Source

Thrown at crates/snapshot/src/remote.rs:565

            self.stats.skipped_object();
            return Ok(None);
        }

        if self.try_hardlink(hash).await? {
            if self.dry_run {
                return Ok(Some(path));
            }

            self.stats.hardlinked_object();
            return Ok(None);
        }

        if self.dry_run {
            return Err(SnapshotError::ReadObject {
                ty,
                source_repo: self.object_repo.root().to_owned(),
                cause: io::Error::new(io::ErrorKind::NotFound, format!("missing object {}", path.display())),
            });
        }

        Ok(Some(path))
    }

    async fn try_hardlink(&self, hash: blake3::Hash) -> Result<bool> {
        let Some(parent) = self.parent_repo.as_ref() else {
            return Ok(false);
        };

        let object_repo = Arc::clone(&self.object_repo);
        let parent_repo = Arc::clone(parent);
        if !self.dry_run {
            spawn_blocking(move || object_repo.try_hardlink_from(&parent_repo, hash.as_bytes()))
                .await
                .unwrap()
                .map_err(Into::into)

View on GitHub (pinned to 524b4487d9)

Solutions

  1. Run the operation without dry-run so missing objects are actually fetched.
  2. Restore or re-download the object repository from the upstream snapshot source.
  3. If objects were intentionally pruned, fetch the missing range from a peer or full replica before dry-running.
Defensive patterns

Strategy: validation

Validate before calling

use std::path::Path;

// Before a dry-run, confirm every object you expect to verify is present:
fn all_objects_present(repo_root: &Path, needed_hashes: &[blake3::Hash]) -> bool {
    needed_hashes.iter().all(|h| repo_root.join(format!("objects/{}", h)).exists())
}

Try / catch

match dry_run_result {
    Err(SnapshotError::ReadObject { cause: ref e, .. }) if e.kind() == std::io::ErrorKind::NotFound => {
        // Dry-run cannot fetch: run the operation for real so missing objects are
        // downloaded, or restore the object repo from the upstream source.
    }
    r => r,
}

Prevention

When it happens

Trigger: Running a snapshot fetch/clone/verify with dry-run enabled against an object repo that is missing objects - a partial restore, objects removed by retention/GC, or a parent repo that lacks the requested object.

Common situations: Planning/sizing runs (--dry-run) after objects were garbage-collected; verifying a supposedly complete object store; a misconfigured parent repo path.

Related errors


AI-assisted analysis of clockworklabs/SpacetimeDB@524b4487d9 (2026-08-16). Data as JSON: /api/errors/2526d3d49c86e717. Report an issue: GitHub.