{"record":{"id":"9b602412a6332bdf","repo":"koala73/worldmonitor","slug":"portwatch-cache-read-returned-incomplete-results","errorCode":null,"errorMessage":"PortWatch cache read returned incomplete results","messagePattern":"PortWatch cache read returned incomplete results","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/seed-portwatch-port-activity.mjs","lineNumber":1037,"sourceCode":"  if (failures.length > 0) {\n    throw new Error(`Redis transaction: ${failures.length}/${commands.length} commands failed`);\n  }\n  return results;\n}\n\nconst CORRUPT_COUNTRY_CACHE = Symbol('corrupt country cache');\n\n// MGET-style batch read via the Upstash REST /pipeline endpoint. Returns an\n// array aligned with `keys` where each element is either the parsed JSON\n// payload, explicit miss, or confirmed corrupt value. Transport/envelope errors\n// remain fatal: only a validated upstream replacement may overwrite corruption.\n// Primes the per-country cache lookup in one round-trip instead of 174 GETs.\nasync function redisMgetJson(keys) {\n  if (keys.length === 0) return [];\n  const commands = keys.map((k) => ['GET', k]);\n  const results = await redisPipeline(commands);\n  if (!Array.isArray(results) || results.length !== keys.length) {\n    throw new Error('PortWatch cache read returned incomplete results');\n  }\n  return results.map((r) => {\n    if (r?.error || !Object.hasOwn(r ?? {}, 'result')) throw new Error('PortWatch cache read failed');\n    if (r.result === null) return null;\n    if (typeof r.result !== 'string') throw new Error('PortWatch cache read returned invalid result');\n    try {\n      const payload = JSON.parse(r.result);\n      return payload && typeof payload === 'object' && !Array.isArray(payload)\n        ? payload : CORRUPT_COUNTRY_CACHE;\n    } catch {\n      return CORRUPT_COUNTRY_CACHE;\n    }\n  });\n}\n\n// fetchAll() — pure data collection, no Redis writes.\n// Returns { countries: string[], countryData: Map<iso2, payload>, fetchedAt: string }.\n//","sourceCodeStart":1019,"sourceCodeEnd":1055,"githubUrl":"https://github.com/koala73/worldmonitor/blob/7d06c8633d256c18e38133030bc3613976a96ec9/scripts/seed-portwatch-port-activity.mjs#L1019-L1055","documentation":"Thrown by redisMgetJson in scripts/seed-portwatch-port-activity.mjs when the batched GET (MGET-style pipeline) result array is missing or has a different length than the requested keys. redisMgetJson primes the per-country cache in one round-trip and treats a length mismatch as a corrupted response it cannot map back to keys.","triggerScenarios":"The underlying redisPipeline call returns null/undefined (caught and rethrown earlier usually), a non-array, or an array whose length differs from keys.length — e.g. a partial/interleaved response, a proxy truncating the body, or redisPipeline's own error path returning a short result list.","commonSituations":"Upstash silently truncating very large pipelines (174+ keys); a middlebox returning partial JSON; a bug in redisPipeline's response validation letting a malformed body through; mixing an old cached client with a new response format.","solutions":["Verify redisPipeline's success path — this throw means its length check passed upstream or was bypassed; inspect the actual results array logged before this point","Reduce batch size (chunk keys into groups of ~50-100) to rule out truncation of large pipelines","Retry the whole MGET with backoff since partial truncation is usually transient","Confirm the endpoint is the /pipeline endpoint returning one result per command","Add logging of keys.length vs results.length to identify which response shape reaches this check"],"exampleFix":"// before\nconst results = await redisPipeline(commands);\nif (!Array.isArray(results) || results.length !== keys.length) {\n  throw new Error('PortWatch cache read returned incomplete results');\n}\n// after\nconst results = await mgetWithRetry(commands, { retries: 2, chunkSize: 100 });\nif (!Array.isArray(results) || results.length !== keys.length) {\n  throw new Error(`PortWatch cache read incomplete: got ${results?.length}/${keys.length} results`);\n}","handlingStrategy":"retry","validationCode":"const keysAreValidStrings = (keys) => Array.isArray(keys) && keys.length > 0 && keys.every((k) => typeof k === 'string' && k.length > 0);","typeGuard":"const isCompleteMgetResult = (res, n) => Array.isArray(res) && res.length === n;","tryCatchPattern":"try {\n  const cache = await redisMgetJson(keys);\n} catch (err) {\n  if (err.message.includes('incomplete results')) {\n    return retryWithBackoff(() => redisMgetJson(keys), { retries: 2 });\n  }\n  throw err;\n}","preventionTips":["Chunk MGET batches (50-100 keys) instead of one giant pipeline","Retry transient short responses with backoff before failing the seed","Log keys.length vs results.length on mismatch for diagnosis","Pin and monitor the Upstash API behavior used by the seed"],"tags":["redis","cache","mget","seeding"],"backgroundTag":"unexpected-response-shape","analyzedSha":"7d06c8633d256c18e38133030bc3613976a96ec9","analyzedAt":"2026-09-15T16:44:39.439Z","contentChangedAt":"2026-09-15T16:44:39.439Z","schemaVersion":2},"datasetVersion":"2026-09-15T18:17:12.389Z"}