{"record":{"id":"eb84ae8334d95a79","repo":"thedotmack/claude-mem","slug":"sync-hub-push-sent-operation-is-not-covered-by-pr","errorCode":null,"errorMessage":"sync hub push: sent operation is not covered by projected_seq","messagePattern":"sync hub push: sent operation is not covered by projected_seq","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/services/sync/CloudSync.ts","lineNumber":1077,"sourceCode":"        throw new Error(\n          `sync hub push: 200 response acknowledgment multiplicity mismatch (expected ${expected}, received ${actual})`\n        );\n      }\n    }\n    if (ackCounts.size !== sentCounts.size) {\n      // Defensive: the unknown-tuple branch above should make this impossible.\n      throw new Error('sync hub push: 200 response acknowledgment multiset mismatch');\n    }\n\n    if (compareCanonicalDecimals(response.head_seq, response.projected_seq) > 0) {\n      throw new Error('sync hub push: checkpoint order requires head_seq <= projected_seq');\n    }\n    for (const ack of response.acked) {\n      if (compareCanonicalDecimals(ack.seq, response.head_seq) > 0) {\n        throw new Error('sync hub push: acknowledgment seq exceeds head_seq');\n      }\n      if (compareCanonicalDecimals(ack.seq, response.projected_seq) > 0) {\n        throw new Error('sync hub push: sent operation is not covered by projected_seq');\n      }\n    }\n  }\n\n  /**\n   * Stamp rows / delete outbox entries for a fully validated acknowledgment\n   * multiset. The hub may return entries in any order.\n   */\n  private stampAcked(acked: AckedOp[], pushed: WireOp[]): void {\n    const now = Date.now();\n    const bodies = new Map(pushed.map(op => {\n      const body = parseCanonicalOperation(op);\n      return [operationTupleKey({\n        id: body.id,\n        kind: body.kind,\n        entity_rev: body.entity_rev,\n        operation_sha256: op.operation_sha256,\n      }), { body, operationSha256: op.operation_sha256 }] as const;","sourceCodeStart":1059,"sourceCodeEnd":1095,"githubUrl":"https://github.com/thedotmack/claude-mem/blob/e2d1df569a8f04075d40e92461128ece7cf04c82/src/services/sync/CloudSync.ts#L1059-L1095","documentation":"An acked operation's seq exceeds the response's projected_seq — the acknowledgment is not covered by the projection checkpoint, breaking the core promise that 200 is returned only when the checkpoint covers every acknowledged commit. Caught in validatePushResponse before any local stamping; flush() backs off and retries the same ops (hub dedupe keeps this safe).","triggerScenarios":"Hub emits acks from the append path but projected_seq from a lagging projection worker; projection advance fails silently and the response path doesn't wait on it; field computation order bug in the response builder.","commonSituations":"Hub under heavy load where the projection falls behind and the 200 gate was relaxed; a hub change that made projection fire-and-forget; partial deployment mixing gated and ungated response paths.","solutions":["Check hub metrics/logs for projection lag at the time of the failure","Restore the gate: the hub must not answer 200 until projected_seq >= every ack.seq it returns","If projection advance is failing (worker crash, Pro errors), fix that root cause first","Clients self-heal via retry once the hub honors the checkpoint gate again"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// Hub-side gate: 200 only when the checkpoint covers every ack:\nfunction checkpointCoversAcks(acked: { seq: string }[], projected_seq: string): boolean {\n  return acked.every(a => BigInt(a.seq) <= BigInt(projected_seq));\n}\n// if (!checkpointCoversAcks(...)) await waitForProjection(minAckSeq) before responding","typeGuard":"function acksCoveredByCheckpoint(acked: { seq: string }[], projected_seq: string): boolean {\n  return acked.every(a => BigInt(a.seq) <= BigInt(projected_seq));\n}","tryCatchPattern":"if (/not covered by projected_seq/.test(sync.status().lastError ?? '')) {\n  // hub returned 200 too early: projection gate broken or projection worker failing; check hub logs\n}","preventionTips":["Make projection advancement part of the push commit path, not fire-and-forget","Alert on hub projection lag; a lagging projection turns this into a hard sync stall","Verify the gate survives hub restarts and partial projection replays"],"tags":["sync","hub","integrity","sequence","checkpoint","projection"],"backgroundTag":"sequence-invariant-violation","analyzedSha":"e2d1df569a8f04075d40e92461128ece7cf04c82","analyzedAt":"2026-08-20T23:58:13.836Z","contentChangedAt":"2026-08-20T23:58:13.836Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}