Tencent/WeKnora · error

cleanup losing remote sandbox: %w

Error message

cleanup losing remote sandbox: %w

What it means

After losing the binding-create race, createAndBind must delete the just-created remote sandbox that it did not win. If the provider Delete (via cleanupCreated) fails, the error is wrapped so the caller knows a losing/leaked sandbox may remain in the provider account.

Source

Thrown at internal/sandbox/session_lifecycle.go:407

		)
	}
	if created {
		return handle, nil
	}

	winner, winnerErr := l.readBinding(ctx, key)
	if winnerErr != nil {
		// The authoritative winner is unknown, so deleting this sandbox could
		// destroy the resource another coordinator just bound.
		return nil, fmt.Errorf("read winning sandbox binding: %w", winnerErr)
	}
	if winner != nil &&
		winner.Provider == l.client.Provider() &&
		winner.SandboxID == handle.ID() {
		return handle, nil
	}
	if cleanupErr := l.cleanupCreated(ctx, handle); cleanupErr != nil {
		return nil, fmt.Errorf("cleanup losing remote sandbox: %w", cleanupErr)
	}
	if winner == nil {
		return nil, errors.New("sandbox binding create lost without a winner")
	}
	return l.connectKnownWinner(ctx, *winner)
}

func (l *remoteSessionLifecycle) connectWinner(
	ctx context.Context,
	key SessionSandboxKey,
) (RemoteSandboxHandle, error) {
	winner, err := l.readBinding(ctx, key)
	if err != nil {
		return nil, err
	}
	if winner == nil {
		return nil, errors.New("sandbox binding create lost without a winner")
	}

View on GitHub (pinned to 988cbb0330)

Solutions

1. Inspect the wrapped provider error and retry the session resolve — cleanup may succeed on retry
2. Check the provider dashboard for leaked/orphaned sandboxes from the failed session key and delete them manually
3. Increase cleanup timeout / reduce concurrent allocation pressure
4. Verify the client's credentials have delete permission for the sandbox

Example fix

// before
handle, err := manager.Resolve(ctx, sessionKey)
// after
if err != nil && strings.Contains(err.Error(), "cleanup losing remote sandbox") {
    // losing sandbox cleanup failed; retry, then reconcile leaked sandboxes
    handle, err = manager.Resolve(ctx, sessionKey)
}

// Check provider health before concurrent allocations
if err := client.Health(ctx); err != nil { /* defer workloads */ }

handle, err := manager.Resolve(ctx, key)
if err != nil && strings.Contains(err.Error(), "cleanup losing remote sandbox") {
    // retry resolve; leaked sandbox cleanup may succeed on retry
    handle, err = manager.Resolve(ctx, key)
}

When it happens

Trigger: Coordinator loses a concurrent binding race and its cleanupCreated -> client.Delete call against the remote provider fails (network error, provider 5xx, timeout, sandbox already terminating in a way not recognized as replaceable).

Common situations: Provider API instability or rate limits during concurrent allocation storms; short cleanup context timeouts; orphaned sandboxes accumulating in the provider console after repeated failures.

Related errors

• delete remote sandbox: %w
• model ID cannot be empty
• model is currently downloading
• model download failed
• abnormal model status

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/8b547ca9291905a0. Report an issue: GitHub.