{"record":{"id":"986f4b9cc541e782","repo":"koala73/worldmonitor","slug":"failed-to-enqueue-scenario-job","errorCode":null,"errorMessage":"Failed to enqueue scenario job","messagePattern":"Failed to enqueue scenario job","errorType":"http","errorClass":"ApiError","httpStatus":502,"severity":"error","filePath":"server/worldmonitor/scenario/v1/run-scenario.ts","lineNumber":67,"sourceCode":"  const depth = typeof depthEntry?.result === 'number' ? depthEntry.result : 0;\n  if (depth > MAX_QUEUE_DEPTH) {\n    throw new ApiError(429, 'Scenario queue is at capacity, please try again later', '');\n  }\n\n  const jobId = generateJobId();\n  const payload = JSON.stringify({\n    jobId,\n    scenarioId,\n    iso2: iso2 || null,\n    enqueuedAt: Date.now(),\n  });\n\n  // Upstash RPUSH returns the new list length; helper returns [] on transport\n  // failure. Either no entry or a non-numeric result means the enqueue never\n  // landed — surface as 502 so the caller retries.\n  const [pushEntry] = await runRedisPipeline([['RPUSH', QUEUE_KEY, payload]], true);\n  if (!pushEntry || typeof pushEntry.result !== 'number') {\n    throw new ApiError(502, 'Failed to enqueue scenario job', '');\n  }\n\n  // statusUrl is a server-computed convenience URL preserved from the legacy\n  // /api/scenario/v1/run contract so external callers can keep polling via the\n  // response body rather than hardcoding the status path. See the proto comment\n  // on RunScenarioResponse for why this matters on a v1 → v1 migration.\n  const statusUrl = `/api/scenario/v1/get-scenario-status?jobId=${encodeURIComponent(jobId)}`;\n\n  // Async-enqueue contract: the job is accepted, not complete. Restore the\n  // legacy 202 Accepted status (lost in the sebuf migration — the generated\n  // server hardcodes 200) and point at the poller via Location. The gateway\n  // applies the override on POST-200 only, so the thrown 403/429/502 paths\n  // above keep their status.\n  setSuccessStatusOverride(ctx.request, 202);\n  setResponseHeader(ctx.request, 'Location', statusUrl);\n\n  return {\n    jobId,","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/koala73/worldmonitor/blob/eeab0a219fce0f02a00603b532dbae9041b934ac/server/worldmonitor/scenario/v1/run-scenario.ts#L49-L85","documentation":"Thrown as HTTP 502 when the Upstash Redis RPUSH of the scenario job payload produces no usable result: runRedisPipeline returns [] on transport failure, and a missing or non-numeric result means the enqueue never landed. The handler maps this to 502 specifically so callers retry, since the write failed atomically and a retry cannot half-apply.","triggerScenarios":"RPUSH scenario-queue:pending fails or returns a non-numeric result: Upstash REST endpoint unreachable, missing/invalid UPSTASH_REDIS_REST credentials, Redis outage or timeout, or a response-shape regression in the _shared/redis pipeline helper.","commonSituations":"UPSTASH_REDIS_REST_URL/TOKEN missing or wrong in the deployment env; Upstash incident or quota exhaustion; egress blocked from the edge runtime; runRedisPipeline change that swallows errors and returns [].","solutions":["Verify UPSTASH_REDIS_REST_URL and UPSTASH_REDIS_REST_TOKEN are set and valid in the deployment environment","Test connectivity directly, e.g. runRedisPipeline([['PING']], true) or curl the Upstash REST endpoint","Retry the RunScenario request — the enqueue never landed, so a retry does not duplicate the job","If persistent, inspect server/_shared/redis.ts runRedisPipeline to see which failure path returns an empty array"],"exampleFix":"// before\nconst res = await fetch(runUrl, { method: 'POST', body });\nif (!res.ok) throw new Error('run failed');\n// after\nconst res = await fetch(runUrl, { method: 'POST', body });\nif (res.status === 502) {\n  // safe to retry: the enqueue never landed\n  return retryWithBackoff(() => fetch(runUrl, { method: 'POST', body }));\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { await runScenario(req); } catch (e) { if (e?.status === 502) return retryWithBackoff(() => runScenario(req)); throw e; } — the enqueue never landed, so retry cannot duplicate the job.","preventionTips":["Alert on 502 rates from /scenario/v1/run — they indicate Redis transport health, not caller error","Verify Upstash credentials during deploy preflight, not at first request","Retry 502s automatically; do not surface them to end users as permanent failures"],"tags":["redis","upstash","http-502","enqueue","infrastructure","scenario"],"backgroundTag":"redis-connection-failed","analyzedSha":"eeab0a219fce0f02a00603b532dbae9041b934ac","analyzedAt":"2026-08-21T16:51:25.751Z","contentChangedAt":"2026-08-21T16:51:25.751Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}