Tencent/WeKnora · error

mark sandbox binding stale: %w

Error message

mark sandbox binding stale: %w

What it means

Returned by markBindingStale when the markBindingStaleIfMatch Lua script Run fails — Redis unreachable, context cancelled, server-side rejection (READONLY, OOM, NOAUTH), or script execution error. The stale marker was not written. Wrapped as 'mark sandbox binding stale: %w'. Returning (false, nil) only means the stored binding no longer matches the expected provider/sandbox ID (already rebound or deleted) — that is not an error.

Source

Thrown at internal/sandbox/session_binding_redis.go:284

	if err := validateBindingMatch(key, expected.Provider, expected.SandboxID); err != nil {
		return false, err
	}
	marked := expected
	marked.StaleAt = &staleAt
	payload, err := json.Marshal(marked)
	if err != nil {
		return false, fmt.Errorf("encode stale sandbox binding: %w", err)
	}
	wrote, err := markBindingStaleIfMatchScript.Run(
		ctx,
		s.client,
		[]string{s.bindingKey(key)},
		string(expected.Provider),
		expected.SandboxID,
		payload,
	).Int64()
	if err != nil {
		return false, fmt.Errorf("mark sandbox binding stale: %w", err)
	}
	return wrote != 0, nil
}

// escapeRedisGlob quotes the characters SCAN's MATCH treats as wildcards. The
// namespace is operator-supplied and only screened for braces and control
// characters, so a namespace containing "*" would otherwise widen the pattern
// past the workspace it is meant to anchor.
func escapeRedisGlob(literal string) string {
	var out strings.Builder
	out.Grow(len(literal))
	for _, r := range literal {
		switch r {
		case '\\', '*', '?', '[', ']', '^':
			out.WriteByte('\\')
		}
		out.WriteRune(r)
	}

View on GitHub (pinned to 988cbb0330)

Solutions

1. Retry the invalidation — the script is conditional and idempotent per key
2. Give the invalidation job a generous context timeout scaled to tenant key count
3. Check redis-server logs for READONLY/OOM/NOAUTH and fix server-side state
4. Verify the app's Redis ACL user has WRITE on the weknora:* key pattern

Example fix

// before
ok, err := store.InvalidateByConfig(ctx, tenantID, configID)
if err != nil { return err }
// after: per-run retry since the script is match-conditional and safe to re-run
for attempt := 0; attempt < 3; attempt++ {
    if _, err = store.InvalidateByConfig(ctx, tenantID, configID); err == nil {
        break
    }
    if !errors.Is(err, context.DeadlineExceeded) && !isTransientRedis(err) {
        break
    }
    time.Sleep(time.Duration(attempt+1) * 250 * time.Millisecond)
}
if err != nil { return err }

// confirm write access and headroom before a large invalidation fan-out
if err := rdb.Ping(ctx).Err(); err != nil { return err }
if info, err := rdb.Info(ctx, "memory").Result(); err == nil && strings.Contains(info, "maxmemory_human") {
    // log memory section; alert if near maxmemory
    log.Printf("redis memory section: %s", info)
}

func isMarkStaleTransportError(err error) bool {
    return err != nil && strings.Contains(err.Error(), "mark sandbox binding stale")
}
// (false, nil) means the binding changed under us — expected, not an error

wrote, err := store.InvalidateByConfig(ctx, tenantID, configID)
if err != nil {
    if isMarkStaleTransportError(err) && errors.Is(err, context.DeadlineExceeded) {
        // re-run: script is match-conditional, so re-marking is safe
        wrote, err = store.InvalidateByConfig(ctx, tenantID, configID)
    }
    if err != nil { return err }
}
log.Printf("marked %d bindings stale", wrote)

When it happens

Trigger: Calling InvalidateByConfig which fans out markBindingStale per key when Redis is degraded, ctx times out across the fan-out, or the replica rejects writes during failover.

Common situations: Failover in progress while config invalidation runs; context budget consumed by earlier keys in the invalidation loop leaving no time for later ones; Redis maxmemory blocking writes; ACL revoking write access to the app user.

Related errors

• delete sandbox binding: %w
• begin sandbox turn lease: %w
• end sandbox turn lease: %w
• WEKNORA_REDIS_NAMESPACE must not contain braces
• WEKNORA_REDIS_NAMESPACE must not contain control characters

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/01ac1d92329929b9. Report an issue: GitHub.