cocoindex-io/cocoindex · error · DeadlineExceededError

CocoIndex timeout deadline exceeded

Error message

CocoIndex timeout deadline exceeded

What it means

DeadlineExceededError is raised by retry_transient when the cooperative deadline has expired: no attempt may start past the deadline, and a remaining time of exactly zero or less counts as expired before each attempt. It can also arise with bound_attempt=True when asyncio.wait_for cancels an in-flight attempt at the deadline. Unlike a retryable failure, this is a policy wall — the operation as a whole has run out of time and the loop stops instead of retrying.

Source

Thrown at python/cocoindex/_internal/deadline.py:188

        raise ValueError("retry_transient requires a positive timeout")
    if backoff is None:
        backoff = exponential_backoff()

    scope = _timeout_scope(timeout) if timeout is not None else contextlib.nullcontext()
    with scope:
        # Exception, not BaseException: doubles as a type-level guard — if
        # the except clause below ever widens back to BaseException, this
        # assignment becomes a mypy error.
        last_error: Exception | None = None
        attempt_index = 0
        while True:
            # Never start an attempt past the deadline. A remaining time of
            # exactly zero counts as expired, so a sleep clipped to the
            # deadline cannot spin at the boundary.
            check_cancellation()
            remaining = remaining_seconds()
            if remaining is not None and remaining <= 0:
                raise DeadlineExceededError("CocoIndex timeout deadline exceeded")

            try:
                if bound_attempt and remaining is not None:
                    try:
                        result = await asyncio.wait_for(fn(), timeout=remaining)
                    except TimeoutError as timeout_error:
                        # Translate only a wait_for cancellation at the
                        # deadline; a TimeoutError raised by fn itself
                        # before the deadline re-raises as-is.
                        remaining_now = remaining_seconds()
                        if remaining_now is not None and remaining_now <= 0:
                            raise DeadlineExceededError(
                                "CocoIndex timeout deadline exceeded"
                            ) from timeout_error
                        raise
                else:
                    result = await fn()
            except Exception as error:

View on GitHub (pinned to e84aa99b32)

Solutions

  1. Increase the timeout= budget or raise the enclosing coco.timeout(timedelta(...)) scope to cover the realistic total time including retries.
  2. Reduce retry pressure: lower max_attempts, increase exponential_backoff initial/max_delay, or narrow retry_on to genuinely transient errors so fatal ones fail fast.
  3. Make fn() cooperative — await checkpointed calls so the deadline check runs inside attempts, and consider bound_attempt=True to hard-cancel long attempts.
  4. Catch DeadlineExceededError at the operation boundary and handle it as a timeout (report, reschedule, or propagate), not as a transient failure to retry.

Example fix

// before
with timeout(timedelta(seconds=5)):  # too small for N retries
    await retry_transient(fn, retry_on=(IOError,), max_attempts=10)

// after
with timeout(timedelta(seconds=60)):
    await retry_transient(fn, retry_on=(IOError,), max_attempts=10, bound_attempt=True)
Defensive patterns

Strategy: try-catch

Validate before calling

from cocoindex._internal import deadline
remaining = deadline.remaining_seconds()
if remaining is not None and remaining <= 0:
    raise RuntimeError("budget already exhausted before starting retries")

Type guard

def has_budget(seconds_needed: float) -> bool:
    from cocoindex._internal import deadline
    r = deadline.remaining_seconds()
    return r is None or r > seconds_needed

Try / catch

from cocoindex import DeadlineExceededError
try:
    result = await retry_transient(fn, retry_on=(IOError,), max_attempts=5, timeout=timeout)
except DeadlineExceededError:
    logger.warning("operation exceeded its time budget; giving up")
    # treat as timeout: reschedule, report, or propagate — do NOT retry

Prevention

When it happens

Trigger: The sum of attempt durations plus backoff sleeps exceeds the effective timeout (either the timeout= argument or an ambient coco.timeout() scope); calling retry_transient when an outer deadline is already nearly exhausted; bound_attempt=True with fn() that ignores deadlines and has to be cancelled at the wall.

Common situations: Long-running network/database operations whose transient errors burn the whole budget through repeated retries; nested deadlines where an outer app-level timeout leaves almost nothing for an inner retry loop; slow fn() calls (no awaits that check cancellation) that only get cut off by bound_attempt=True.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

AI-assisted analysis of cocoindex-io/cocoindex@e84aa99b32 (2026-09-08). Data as JSON: /api/errors/1c19dd2c489fddb4. Report an issue: GitHub.