{"record":{"id":"534e6a8d6567607b","repo":"paperclipai/paperclip","slug":"dropping-batch-batch-batchid-after-batch-atte","errorCode":null,"errorMessage":"dropping batch ${batch.batchId} after ${batch.attempt} attempt(s); ${batch.events.length} event(s) lost","messagePattern":"dropping batch (.+?) after (.+?) attempt\\(s\\); (.+?) event\\(s\\) lost","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"packages/shared/src/telemetry/client.ts","lineNumber":276,"sourceCode":"   * same events + `batchId` are re-queued with capped, jittered backoff; on a\n   * terminal failure or after `maxAttempts` the batch is dropped-and-logged.\n   */\n  private async attemptSend(batch: PendingBatch): Promise<void> {\n    const body = JSON.stringify(this.buildEnvelope(batch.events, batch.batchId));\n    const outcome = this.classifyOutcome(await this.postEnvelope(body));\n\n    if (outcome.kind === \"ok\") return;\n\n    if (outcome.kind === \"terminal\") {\n      this.warn(\n        `dropping batch ${batch.batchId} on terminal response (HTTP ${outcome.status}); ${batch.events.length} event(s) lost`,\n      );\n      return;\n    }\n\n    // Retryable (429/502/503/504 or network/timeout).\n    if (batch.attempt >= this.caps.backoff.maxAttempts) {\n      this.warn(\n        `dropping batch ${batch.batchId} after ${batch.attempt} attempt(s); ${batch.events.length} event(s) lost`,\n      );\n      return;\n    }\n    // Cap the delay at maxDelayMs. `computeBackoffMs` is already capped, but a\n    // server `Retry-After` hint is not — an out-of-range value could otherwise\n    // overflow the runtime timer range and be clamped by Node to a near-immediate\n    // timeout, causing rapid retries. The cap keeps every retry within the\n    // configured backoff ceiling.\n    const delayMs = Math.min(\n      outcome.retryAfterMs ?? this.computeBackoffMs(batch.attempt),\n      this.caps.backoff.maxDelayMs,\n    );\n    this.enqueuePending({\n      events: batch.events,\n      batchId: batch.batchId,\n      attempt: batch.attempt + 1,\n      nextAttemptAt: Date.now() + delayMs,","sourceCodeStart":258,"sourceCodeEnd":294,"githubUrl":"https://github.com/paperclipai/paperclip/blob/120ae5428fa29bee300bcf806491cd4d965fbb7c/packages/shared/src/telemetry/client.ts#L258-L294","documentation":"For retryable failures (429/502/503/504, network or timeout) the client re-queues the exact same events+batchId with capped, jittered backoff honoring Retry-After. Once batch.attempt reaches backoff.maxAttempts, the batch is dropped and logged. Sustained collector unavailability or aggressive rate limiting therefore ends in data loss.","triggerScenarios":"The collector (or the network path to it) stays failed for longer than maxAttempts × backoff window; prolonged 429 rate limiting; DNS failure; sandbox egress policy blocking the collector host.","commonSituations":"Collector outage during business hours; egress firewalls (see the kubernetes FQDN warnings) newly blocking telemetry; quota exhausted on the telemetry SaaS; retry budget too small for real outage durations.","solutions":["Restore collector reachability: check URL, DNS, and egress/firewall rules from the server host.","If 429-driven, raise rate limits at the collector or reduce emit volume/flush frequency.","Increase backoff.maxAttempts and maxDelayMs so retries span realistic outage windows.","Raise maxPendingRetryBatches to queue more batches while the collector recovers."],"exampleFix":"// before\ntelemetry: { backoff: { maxAttempts: 3, maxDelayMs: 10_000 } }\n\n// after\ntelemetry: { backoff: { maxAttempts: 8, maxDelayMs: 60_000 } }","handlingStrategy":"retry","validationCode":"// Probe collector reachability before sending real traffic.\nconst ok = await fetch(`${collectorUrl.replace(/\\/$/, '')}/healthz`) // or known 2xx route\n  .then((r) => r.ok)\n  .catch(() => false);\nif (!ok) logger.warn('telemetry collector unreachable; batches will burn retry budget');","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Size backoff.maxAttempts × maxDelayMs to exceed your realistic collector outage window.","Alert once drop-after-max-attempts warns appear — that is the end of the retry budget.","Ensure sandbox/host egress rules allow the collector host (FQDN egress limitations bite here).","Keep 429 rate limits aligned with client emit volume."],"tags":["telemetry","retry","backoff","rate-limit","batch-drop"],"backgroundTag":"retry-limit-exceeded","analyzedSha":"120ae5428fa29bee300bcf806491cd4d965fbb7c","analyzedAt":"2026-08-18T22:49:45.177Z","schemaVersion":2},"datasetVersion":"2026-08-22T09:17:25.309Z"}