{"record":{"id":"d264cd9188b688e0","repo":"jackwener/OpenCLI","slug":"server-returned-html-instead-of-json-status-res","errorCode":null,"errorMessage":"Server returned HTML instead of JSON (status=${response.status}). Likely a login wall, rate limit, or WAF challenge.","messagePattern":"Server returned HTML instead of JSON \\(status=(.+?)\\)\\. Likely a login wall, rate limit, or WAF challenge\\.","errorType":"exception","errorClass":"LoginWallError","httpStatus":null,"severity":"error","filePath":"src/utils.ts","lineNumber":148,"sourceCode":" * leading the body \\u2014 naive `JSON.parse` on these gives a cryptic\n * `SyntaxError` that callers can't distinguish from \"real\" malformed JSON.\n *\n * On real (non-HTML) JSON-parse failures, throws a regular `Error` with a\n * body preview attached so debugging doesn't require a packet capture. */\nexport async function parseJsonOrThrowLoginWall(\n  response: Response,\n  opts: { url?: string } = {},\n): Promise<unknown> {\n  const contentType = response.headers.get('content-type') || '';\n  const text = await response.text();\n  const trimmed = text.trimStart();\n\n  const looksLikeHtml =\n    contentType.toLowerCase().includes('text/html')\n    || /^<(?:!doctype|html|head|body|title)(?:[\\s>/]|$)/i.test(trimmed);\n\n  if (looksLikeHtml) {\n    throw new LoginWallError(\n      `Server returned HTML instead of JSON (status=${response.status}). `\n      + `Likely a login wall, rate limit, or WAF challenge.`,\n      response.status,\n      opts.url || response.url || '',\n      trimmed.slice(0, 100),\n    );\n  }\n\n  try {\n    return JSON.parse(text);\n  } catch (err) {\n    // Real malformed JSON \\u2014 surface body preview alongside the parser message\n    // so we don't have to repro to know what came back.\n    throw new Error(\n      `JSON parse failed (status=${response.status}, body[0..50]=${JSON.stringify(trimmed.slice(0, 50))}): `\n      + (err instanceof Error ? err.message : String(err)),\n    );\n  }","sourceCodeStart":130,"sourceCodeEnd":166,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/src/utils.ts#L130-L166","documentation":"parseJsonOrThrowLoginWall checks the response content-type and the first characters of the body; if either indicates HTML, it throws LoginWallError (code LOGIN_WALL, exit 77) carrying the HTTP status, URL, and first 100 chars of the body. It exists to convert 'server gave me a webpage' into an actionable auth/rate-limit error rather than a JSON parse stack trace.","triggerScenarios":"Calling parseJsonOrThrowLoginWall on a fetch whose response content-type includes 'text/html' or whose trimmed body starts with <!doctype/html/head/body/title — e.g. an API URL that redirected to a login page, or a 403/429 WAF page.","commonSituations":"Expired session cookies on sites like x.com/grok; scraping an endpoint that now requires auth; corporate proxies injecting HTML error pages; Cloudflare 'checking your browser' interstitials during bursts of requests.","solutions":["Log in again in the browser session the CLI uses, then retry","If status is 429 or the body mentions challenge/verify, back off and retry after a delay","Verify the URL is the JSON API endpoint, not an HTML page that redirects","Inspect err.bodyPreview to identify the wall type before automating retries"],"exampleFix":"// before\nconst data = await res.json();\n// after\nconst data = await parseJsonOrThrowLoginWall(res, { url });","handlingStrategy":"try-catch","validationCode":"const ct = res.headers.get('content-type') || '';\nif (ct.includes('text/html')) throw new LoginWallError('HTML response', res.status, res.url, (await res.text()).slice(0, 100));","typeGuard":"function isLoginWallError(e: unknown): e is LoginWallError {\n  return e instanceof LoginWallError;\n}","tryCatchPattern":"try {\n  const data = await parseJsonOrThrowLoginWall(res, { url });\n} catch (e) {\n  if (e instanceof LoginWallError) {\n    if (e.status === 429) await sleep(60_000);\n    else await relogin();\n  } else throw e;\n}","preventionTips":["Use the typed error's status/bodyPreview to branch: 429 → back off, 401/403 → re-login","Add exponential backoff for retries against rate-limiting endpoints","Prefer API endpoints over page URLs that can redirect to HTML login pages","Verify session validity with a cheap authenticated endpoint before long jobs"],"tags":["http","login-wall","rate-limit","json","network"],"backgroundTag":"html-instead-of-json","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}