Hmbown/CodeWhale · critical

cloud agent was created but its labels could not be applied…

Error message

cloud agent {sandbox_id} was created but its labels could not be applied ({}) and teardown also failed ({}); the sandbox needs manual cleanup at the provider.

What it means

Worst-case rollback outcome: the sandbox was created, label application failed, AND the teardown DELETE also failed. The sandbox is now orphaned at the provider with no job labels, so the error tells the operator to clean it up manually. Both underlying errors are included, sanitized.

Solutions

  1. Find the sandbox id from the message in the Daytona dashboard and delete it manually
  2. Verify the API key has sandbox delete permissions
  3. Retry the job only after cleaning up to avoid quota drain
  4. Check Daytona status/incidents if 5xx
Defensive patterns

Strategy: try-catch

Validate before calling

check_permission(key, "sandbox:delete")?; // ensure rollback is possible before creating anything

Try / catch

match dispatch(job) {
    Err(e) if e.to_string().contains("manual cleanup") => {
        let id = extract_sandbox_id_from_error(&e);
        alert_orphaned_sandbox(id);
    }
    other => other,
}

Prevention

When it happens

Trigger: Labels PUT fails and the compensating DELETE returns a non-success status other than 404 (e.g. 403 no delete permission, 5xx outage).

Common situations: Daytona incident affecting both endpoints, API key scoped read/write but not delete, network partition during rollback.

Related errors


AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/44d7df637bd35ac8. Report an issue: GitHub.

Appendix: source

Thrown at crates/tui/src/cloud_dispatch.rs:1571

                &Self::control_plane_url(&format!("sandbox/{sandbox_id}"))?,
                &api_key,
                serde_json::Value::Null,
            )
            .and_then(|response| {
                let status = response.status();
                if status.is_success() || status.as_u16() == 404 {
                    Ok(())
                } else {
                    bail!("HTTP {status}")
                }
            });
            match undo {
                Ok(()) => bail!(
                    "cloud agent created but its labels could not be applied ({}); \
                     the sandbox was torn down — retry the job.",
                    sanitize_error(&label_error.to_string())
                ),
                Err(undo_error) => bail!(
                    "cloud agent {sandbox_id} was created but its labels could not be \
                     applied ({}) and teardown also failed ({}); the sandbox needs \
                     manual cleanup at the provider.",
                    sanitize_error(&label_error.to_string()),
                    sanitize_error(&undo_error.to_string())
                ),
            }
        }
        Ok(SandboxReceipt {
            sandbox_id,
            toolbox_url,
        })
    }

    fn wait_ready(&self, receipt: &SandboxReceipt) -> Result<()> {
        let api_key = Self::api_key()?;
        if !valid_sandbox_id(&receipt.sandbox_id) {
            bail!("the sandbox id is not a usable path token");

View on GitHub (pinned to 73e0f67d83)