t8y2/dbx · warning

ETCD_LEASE_LIST_TIMEOUT

ETCD_LEASE_LIST_TIMEOUT

Error message

ETCD_LEASE_LIST_TIMEOUT

What it means

leaseList paginates leases under a deadline represented by leaseListDeadline. Before each page fetch, remaining() checks time.Until(deadline); if the deadline has already passed (remaining <= 0), it aborts with ETCD_LEASE_LIST_TIMEOUT. The operation is a best-effort listing, so hitting the deadline yields a partial result rather than blocking indefinitely.

Source

Thrown at agents/drivers/etcd-go/lease.go:33

	"google.golang.org/grpc/status"
)

const (
	maxLeaseAttachedKeys     = 256
	defaultLeaseListLimit    = 100
	maxLeaseListLimit        = 200
	leaseListConcurrency     = 8
	leaseListDeadlineSeconds = 5
)

type leaseListDeadline struct {
	deadline time.Time
}

func (d leaseListDeadline) remaining() (time.Duration, error) {
	remaining := time.Until(d.deadline)
	if remaining <= 0 {
		return 0, errors.New("ETCD_LEASE_LIST_TIMEOUT")
	}
	return remaining, nil
}

func (s *etcdSession) leaseList(params map[string]json.RawMessage) (any, error) {
	client, err := s.activeClient()
	if err != nil {
		return nil, err
	}
	deadline := leaseListDeadline{deadline: time.Now().Add(leaseListDeadlineSeconds * time.Second)}
	limit := intOrDefault(params, "limit", defaultLeaseListLimit)
	if limit < 1 {
		limit = 1
	}
	if limit > maxLeaseListLimit {
		limit = maxLeaseListLimit
	}
	continuation := stringOrNull(params, "continuation")

View on GitHub (pinned to c0390bff16)

Solutions

1. Increase the lease-list deadline/timeout used by beginOperation or the leaseList call configuration.
2. Use the returned continuation token to fetch the remaining leases in a follow-up call instead of one pass.
3. Reduce lease churn / total lease count (revoke expired leases) so the listing completes in time.
4. Check network latency to etcd endpoints and target a healthier endpoint.

res, err := agent.call("lease_list", params)
if err != nil && strings.Contains(err.Error(), "ETCD_LEASE_LIST_TIMEOUT") {
    // retry with a longer deadline or page via continuation
    return agent.call("lease_list", map[string]any{"continuation": lastTok})
}

When it happens

Trigger: Calling leaseList on a cluster with many leases or slow endpoints such that time.Until(d.deadline) <= 0 before/during paging, i.e. the configured lease-list deadline elapsed at lease.go:33.

Common situations: Very large clusters (thousands of active leases) where paging exceeds the timeout; slow or loaded etcd endpoints; a client clock vs server clock mismatch making the deadline appear already expired; setting the per-operation timeout too low in the environment.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• invalid lease continuation: %s
• ETCD_HISTORY_TIMEOUT
• lease, ttl, and preserveLease cannot be specified together
• ttl must be a positive integer
• Cannot preserve lease: key does not exist or has no lease

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/1e7ffa725082a622. Report an issue: GitHub.