t8y2/dbx · warning

agent operation capacity is temporarily exhausted

Error message

agent operation capacity is temporarily exhausted

What it means

errOperationCapacity signals that the driver's bounded runtime is saturated: either all operation permits are in use (acquire timed out after operationPermitTimeout) or the metadata acquisition limit was hit and the context expired. It is classified by classifyRPCError as a retryable 'resource' category error.

Source

Thrown at agents/drivers/cassandra-go/runtime.go:24

	"encoding/json"
	"errors"
	"fmt"
	"os"
	"strconv"
	"strings"
	"sync"
	"time"

	gocql "github.com/apache/cassandra-gocql-driver/v2"
)

const (
	defaultRuntimePoolSize      = 32
	defaultRuntimeMetadataLimit = 8
	operationPermitTimeout      = 30 * time.Second
)

var errOperationCapacity = errors.New("agent operation capacity is temporarily exhausted")

type connectionRuntime struct {
	mu               sync.Mutex
	config           cassandraConfig
	sessions         map[string]*gocql.Session
	retiredSessions  []*gocql.Session
	permits          chan struct{}
	metadataPermits  chan struct{}
	activeOperations int
	references       int
	closed           bool
}

func newConnectionRuntime(cp connectParams) (*connectionRuntime, error) {
	config, err := parseCassandraConfig(cp)
	if err != nil {
		return nil, err
	}

View on GitHub (pinned to c0390bff16)

Solutions

1. Retry the operation with backoff — classifyRPCError marks it Retryable=true
2. Reduce client-side concurrency or serialize burst operations
3. Increase defaultRuntimePoolSize / defaultRuntimeMetadataLimit if the workload legitimately needs more
4. Investigate why permits are held so long (slow queries, stuck metadata fetches) or raise the context deadline

Example fix

// before
// result := rpc("execute_query", opts)
// after
// result := withRetry(func() error { return rpc("execute_query", opts) }, // retryable: resource category
//   backoff.Exponential(100*time.Millisecond, 5))

function isCapacityError(err) {
  return err && err.message.includes("agent operation capacity is temporarily exhausted");
}

try { result = await rpc("execute_query", opts) }
catch (e) {
  if (isCapacityError(e)) {
    await sleep(backoff(attempt++)); // retryable: resource category
    result = await rpc("execute_query", opts);
  }
}

When it happens

Trigger: acquire waits operationPermitTimeout (30s) without a free permit; allKeyspaceMetadata/metadata acquisition returns errOperationCapacity when ctx.Done fires while metadataAcquired is false; concurrent load exceeds the 32-pool-size / 8-metadata-limit defaults.

Common situations: Sudden spike of concurrent queries against one agent connection; slow/overloaded Cassandra cluster making metadata fetches hang past the context deadline; clients issuing more parallel operations than the runtime pool supports.

Related errors

• agent operation capacity is temporarily exhausted
• %w: %d
• agent request capacity is temporarily exhausted
• agent session limit reached: {MAX_AGENT_SESSIONS}
• agentSessionId is required

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/c5b2c87d466f9773. Report an issue: GitHub.