micro/go-micro · warning

circuit breaker half-open (probe limit reached)

Error message

circuit breaker half-open (probe limit reached)

What it means

After the breaker's timeout, Allow() moves the circuit to half-open and permits at most maxHalfOpen probe calls. Once halfOpenUsed reaches maxHalfOpen, additional calls are rejected with this error until a probe succeeds (closing the circuit) or fails (reopening it).

Source

Thrown at gateway/mcp/circuitbreaker.go:103

	cb.mu.Lock()
	defer cb.mu.Unlock()

	switch cb.state {
	case circuitClosed:
		return nil
	case circuitOpen:
		if time.Since(cb.lastFailure) > cb.timeout {
			cb.state = circuitHalfOpen
			cb.halfOpenUsed = 0
			return nil
		}
		return fmt.Errorf("circuit breaker open (consecutive failures: %d)", cb.failures)
	case circuitHalfOpen:
		if cb.halfOpenUsed < cb.maxHalfOpen {
			cb.halfOpenUsed++
			return nil
		}
		return fmt.Errorf("circuit breaker half-open (probe limit reached)")
	}
	return nil
}

// RecordSuccess records a successful call. If half-open, closes the circuit.
func (cb *circuitBreaker) RecordSuccess() {
	cb.mu.Lock()
	defer cb.mu.Unlock()

	cb.failures = 0
	cb.state = circuitClosed
}

// RecordFailure records a failed call. May trip the circuit open.
func (cb *circuitBreaker) RecordFailure() {
	cb.mu.Lock()
	defer cb.mu.Unlock()

View on GitHub (pinned to 24529f1404)

Solutions

1. Retry after an in-flight probe completes: a success closes the circuit and normal calls resume.
2. Increase maxHalfOpen if the target can safely absorb more concurrent probes.
3. Apply client-side queuing/rate limiting so excess calls wait instead of being rejected.
4. Speed up probe completion by fixing whatever makes the target slow or flaky.

Example fix

// before
if err := cb.Allow(); err != nil { call() } // ignoring rejection, hammering half-open
// after
if err := cb.Allow(); err != nil {
    <-probeDone // wait for the allowed probe to finish
    continue
}

func isBreakerHalfOpen(err error) bool {
    return err != nil && strings.Contains(err.Error(), "circuit breaker half-open")
}

if err := cb.Allow(); err != nil {
    if isBreakerHalfOpen(err) {
        select {
        case <-probeFinished: // a permitted probe completed; retry
        case <-time.After(backoff):
        }
    }
    return errRetryLater
}

When it happens

Trigger: Calling Allow() while the breaker is half-open and cb.halfOpenUsed >= cb.maxHalfOpen, i.e. more concurrent calls than the probe limit arrive during the recovery window.

Common situations: High traffic bursts right after the cool-down expires; maxHalfOpen configured to 1 and multiple goroutines probing; slow probes keeping the breaker half-open for a long stretch.

Related errors

• circuit breaker open (consecutive failures: %d)
• rate limit exceeded for tool %s

AI-assisted analysis of micro/go-micro@24529f1404 (2026-09-01). Data as JSON: /api/errors/4f7fb7b7070ae64f. Report an issue: GitHub.