dropping batch ${batch.batchId} after ${batch.attempt} attem

Error message

dropping batch ${batch.batchId} after ${batch.attempt} attempt(s); ${batch.events.length} event(s) lost

What it means

For retryable failures (429/502/503/504, network or timeout) the client re-queues the exact same events+batchId with capped, jittered backoff honoring Retry-After. Once batch.attempt reaches backoff.maxAttempts, the batch is dropped and logged. Sustained collector unavailability or aggressive rate limiting therefore ends in data loss.

Source

Thrown at packages/shared/src/telemetry/client.ts:276

   * same events + `batchId` are re-queued with capped, jittered backoff; on a
   * terminal failure or after `maxAttempts` the batch is dropped-and-logged.
   */
  private async attemptSend(batch: PendingBatch): Promise<void> {
    const body = JSON.stringify(this.buildEnvelope(batch.events, batch.batchId));
    const outcome = this.classifyOutcome(await this.postEnvelope(body));

    if (outcome.kind === "ok") return;

    if (outcome.kind === "terminal") {
      this.warn(
        `dropping batch ${batch.batchId} on terminal response (HTTP ${outcome.status}); ${batch.events.length} event(s) lost`,
      );
      return;
    }

    // Retryable (429/502/503/504 or network/timeout).
    if (batch.attempt >= this.caps.backoff.maxAttempts) {
      this.warn(
        `dropping batch ${batch.batchId} after ${batch.attempt} attempt(s); ${batch.events.length} event(s) lost`,
      );
      return;
    }
    // Cap the delay at maxDelayMs. `computeBackoffMs` is already capped, but a
    // server `Retry-After` hint is not — an out-of-range value could otherwise
    // overflow the runtime timer range and be clamped by Node to a near-immediate
    // timeout, causing rapid retries. The cap keeps every retry within the
    // configured backoff ceiling.
    const delayMs = Math.min(
      outcome.retryAfterMs ?? this.computeBackoffMs(batch.attempt),
      this.caps.backoff.maxDelayMs,
    );
    this.enqueuePending({
      events: batch.events,
      batchId: batch.batchId,
      attempt: batch.attempt + 1,
      nextAttemptAt: Date.now() + delayMs,

View on GitHub (pinned to 120ae5428f)

Solutions

Restore collector reachability: check URL, DNS, and egress/firewall rules from the server host.
If 429-driven, raise rate limits at the collector or reduce emit volume/flush frequency.
Increase backoff.maxAttempts and maxDelayMs so retries span realistic outage windows.
Raise maxPendingRetryBatches to queue more batches while the collector recovers.

Example fix

// before
telemetry: { backoff: { maxAttempts: 3, maxDelayMs: 10_000 } }

// after
telemetry: { backoff: { maxAttempts: 8, maxDelayMs: 60_000 } }

Defensive patterns

Strategy: retry

Validate before calling

// Probe collector reachability before sending real traffic.
const ok = await fetch(`${collectorUrl.replace(/\/$/, '')}/healthz`) // or known 2xx route
  .then((r) => r.ok)
  .catch(() => false);
if (!ok) logger.warn('telemetry collector unreachable; batches will burn retry budget');

Prevention

Size backoff.maxAttempts × maxDelayMs to exceed your realistic collector outage window.
Alert once drop-after-max-attempts warns appear — that is the end of the retry budget.
Ensure sandbox/host egress rules allow the collector host (FQDN egress limitations bite here).
Keep 429 rate limits aligned with client emit volume.

When it happens

Trigger: The collector (or the network path to it) stays failed for longer than maxAttempts × backoff window; prolonged 429 rate limiting; DNS failure; sandbox egress policy blocking the collector host.

Common situations: Collector outage during business hours; egress firewalls (see the kubernetes FQDN warnings) newly blocking telemetry; quota exhausted on the telemetry SaaS; retry budget too small for real outage durations.

Related errors

AI-assisted analysis of paperclipai/paperclip@120ae5428f (2026-08-18). Data as JSON: /api/errors/534e6a8d6567607b. Report an issue: GitHub.