{"record":{"id":"563b4eb75597b96a","repo":"paperclipai/paperclip","slug":"provider-notification-window-exceeded","errorCode":"provider_notification_window_exceeded","errorMessage":"provider_notification_window_exceeded: expected source sequence ${expectedSourceSeq}, received ${event.sourceSeq}","messagePattern":"provider_notification_window_exceeded: expected source sequence (.+?), received (.+?)","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"packages/paperclip-runner/src/live/runnerd-codex-transport.ts","lineNumber":1302,"sourceCode":"    if (\n      event.eventType !== \"harness.ready\" &&\n      event.eventType !== \"session.started\" &&\n      event.eventType !== \"session.resumed\"\n    ) continue;\n    return record(record(record(event.envelope).payload).payload);\n  }\n  return null;\n}\n\nexport function unseenRunnerdCommittedEvents<\n  T extends { sourceSeq: number },\n>(events: readonly T[], lastSourceSeq: number): T[] {\n  const unseen = events.filter((event) => event.sourceSeq > lastSourceSeq);\n  if (unseen.length === 0) return [];\n  let expectedSourceSeq = lastSourceSeq + 1;\n  for (const event of unseen) {\n    if (event.sourceSeq !== expectedSourceSeq) {\n      throw new Error(\n        `provider_notification_window_exceeded: expected source sequence ${expectedSourceSeq}, received ${event.sourceSeq}`,\n      );\n    }\n    expectedSourceSeq += 1;\n  }\n  return unseen;\n}\n\nexport function expandRunnerdCanonicalNotifications(\n  method: string,\n  input: unknown,\n): Array<{ method: string; params: Record<string, unknown> }> {\n  const payload = record(input);\n  if (!Array.isArray(payload.events)) return [{ method, params: payload }];\n  return payload.events.map((event) => ({ method, params: record(event) }));\n}\n\nexport function runnerdCanonicalNotificationMethod(","sourceCodeStart":1284,"sourceCodeEnd":1320,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/packages/paperclip-runner/src/live/runnerd-codex-transport.ts#L1284-L1320","documentation":"This helper filters an event batch down to events the consumer has not yet seen, based on monotonically increasing sourceSeq values. It requires the unseen events to form a contiguous sequence starting at lastSourceSeq + 1; if there is a gap or overlap (the provider's notification window was exceeded and events were dropped or reordered), it throws this error naming the expected and received sequence numbers.","triggerScenarios":"Calling this filter with events where the first unseen event's sourceSeq is greater than lastSourceSeq + 1, or where consecutive unseen events skip a sequence number — indicating lost or dropped provider notifications between polls.","commonSituations":"Polling intervals too long so the provider's notification retention window expired and dropped events; a provider restart resetting or skipping sequence numbers; network interruption losing events; concurrent consumers draining the same window so events are consumed out of band.","solutions":["Resync by fetching a full state snapshot from the provider and resetting lastSourceSeq, instead of replaying the gapped event stream","Reduce polling interval or enable a provider-side replay/catch-up mechanism so events are not dropped","Log the expected vs received sequence numbers and re-establish the session/window with the provider","If the provider restarted, re-baseline the sequence tracker from the provider's current sequence"],"exampleFix":"// before\nconst unseen = filterUnseen(events, lastSourceSeq); // throws on sequence gap\n// after\nlet unseen: T[];\ntry {\n  unseen = filterUnseen(events, lastSourceSeq);\n} catch (err) {\n  if (!(err as Error).message.startsWith(\"provider_notification_window_exceeded\")) throw err;\n  await resyncFromProviderSnapshot(); // full snapshot re-baselines lastSourceSeq\n  unseen = [];\n}","handlingStrategy":"try-catch","validationCode":"const unseen = events.filter(e => e.sourceSeq > lastSourceSeq);\nconst contiguous = unseen.every((e, i) => e.sourceSeq === lastSourceSeq + 1 + i);\nif (!contiguous) throw new Error(\"sequence gap detected; resync from snapshot before draining\");","typeGuard":"const isContiguousWindow = <T extends { sourceSeq: number }>(events: readonly T[], last: number): boolean =>\n  events.filter(e => e.sourceSeq > last)\n    .every((e, i) => e.sourceSeq === last + 1 + i);","tryCatchPattern":"try {\n  unseen = filterUnseenWindow(events, lastSourceSeq);\n} catch (err) {\n  if (!(err as Error).message.startsWith(\"provider_notification_window_exceeded\")) throw err;\n  await resyncFromProviderSnapshot();\n  unseen = [];\n}","preventionTips":["Poll frequently enough to stay inside the provider's notification retention window","Track and persist lastSourceSeq durably across restarts","On provider restart, re-baseline the sequence tracker from a full snapshot"],"tags":["sequence","events","window"],"backgroundTag":"sequence-gap-detected","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-09-10T03:14:50.855Z","contentChangedAt":"2026-09-10T03:14:50.855Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}