rustfs/rustfs · error · BackfillError

RunnerNotInstalled

RunnerNotInstalled

Error message

backfill runner is not installed

What it means

BackfillError::RunnerNotInstalled is thrown when a backfill operation is requested but the process has no backfill runner installed — i.e. the runtime wiring (the component that actually executes migration work) was never registered in this node/binary. This is an assembly/initialization bug, not a runtime data problem.

Source

Thrown at rustfs/src/on_demand_migration/backfill.rs:352

#[derive(Debug, thiserror::Error)]
pub enum BackfillError {
    #[error("backfill checkpoint is malformed: {0}")]
    Malformed(String),
    #[error("unsupported backfill checkpoint format version {found} (this build supports {supported})")]
    UnsupportedFormatVersion { found: u32, supported: u32 },
    #[error("a backfill job is already running for bucket {bucket} (job {job_id}, owner {owner})")]
    AlreadyRunning { bucket: String, job_id: Uuid, owner: String },
    #[error("no backfill job recorded for bucket {0}")]
    NotFound(String),
    #[error("bucket {0} has no usable on-demand migration state on this node")]
    Unavailable(String),
    #[error("bucket {0} has no on-demand migration config")]
    NotConfigured(String),
    #[error("backfill lease lock for bucket {0} is busy")]
    LeaseBusy(String),
    #[error("backfill checkpoint for bucket {0} changed concurrently")]
    Conflict(String),
    #[error("backfill runner is not installed")]
    RunnerNotInstalled,
    #[error(transparent)]
    Storage(#[from] StorageError),
}

/// Current local version of a key as the skip policy sees it.
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct LocalBackfillObject {
    pub size: u64,
    /// The `odm-source-etag` provenance value when present, else the local ETag.
    pub source_etag: Option<String>,
}

/// Shared report of a new or coalesced pull; absent only when not admitted.
pub type PullReport = Option<super::pull::QueuedPullReport>;

/// Everything the job needs from its bucket, so the loop can run against a
/// mock in unit tests. Production: [`BucketBackfillContext`].

View on GitHub (pinned to 5dca076efe)

Solutions

1. Install the backfill runner during process initialization before any backfill call.
2. Confirm you are targeting the full rustfs binary that bundles the migration runner, not a stripped build.
3. In tests, register a test/mock runner before exercising backfill APIs.

Example fix

// before
backfill::start(bucket).await?; // RunnerNotInstalled
// after
backfill::install_runner(my_runner).await?;
backfill::start(bucket).await?;

if !backfill::runner_installed() { return Err(anyhow!("backfill runner not installed")); }

fn runner_ready() -> bool { backfill::runner_installed() }

When it happens

Trigger: Invoking a backfill API (start/resume/status-advance) on a node or in a test/binary where the runner was never registered via the runner-install hook before use.

Common situations: Embedding the on_demand_migration module in a custom binary and forgetting to install the runner; calling backfill endpoints against a node built without migration support; unit tests exercising the API without a mock runner.

Related errors

• LeaseBusy
• Conflict
• Telemetry initialization failed: {0}
• Target initialization failed: {0}
• Server not initialized: {0}

AI-assisted analysis of rustfs/rustfs@5dca076efe (2026-09-06). Data as JSON: /api/errors/1db979940eff1dc2. Report an issue: GitHub.