headroomlabs-ai/headroom · error · OAuth2Error

token endpoint returned non-JSON

Error message

token endpoint returned non-JSON

What it means

The token endpoint returned a 2xx response whose body is not valid JSON, so `json.load(resp)` raised JSONDecodeError, re-wrapped as OAuth2Error. The URL is reachable and returns HTTP 200, but the payload is something else — an HTML login/consent page, a plaintext WAF/block page, or a misrited gateway response.

Source

Thrown at plugins/headroom-oauth2/src/headroom_oauth2/provider.py:138

            form["client_id"] = self.client_id
            form["client_secret"] = self.client_secret
        req = urllib.request.Request(
            self.token_url,
            data=urllib.parse.urlencode(form).encode(),
            headers=headers,
            method="POST",
        )
        try:
            with urllib.request.urlopen(req, timeout=self.timeout) as resp:
                payload = json.load(resp)
        except HTTPError as e:
            with suppress(Exception):
                e.read()  # drain; do NOT surface the IdP body (may echo sensitive context)
            raise OAuth2Error(f"token endpoint returned HTTP {e.code}") from None
        except (URLError, OSError) as e:
            raise OAuth2Error(f"token endpoint unreachable: {e}") from None
        except json.JSONDecodeError:
            raise OAuth2Error("token endpoint returned non-JSON") from None
        token = payload.get("access_token")
        if not token:
            raise OAuth2Error("token endpoint response had no access_token")
        raw = payload.get("expires_in")
        try:
            ttl = int(float(raw))  # tolerate "3600", "3600.0", 3600, or a JSON float
        except (TypeError, ValueError):
            ttl = 300
        ttl = max(1, ttl)  # 0/negative would cause a stale token or per-request minting
        log.info("oauth2: minted token (ttl=%ss, scopes=%s)", ttl, self.scopes or "-")
        return token, ttl

View on GitHub (pinned to 322425c43b)

Solutions

1. Confirm the exact token endpoint path from your IdP docs (Azure: `.../oauth2/v2.0/token`; Keycloak: `.../protocol/openid-connect/token`; Auth0: `.../oauth/token`) and fix token_url
2. Curl the URL and inspect content type: `curl -si -d 'grant_type=client_credentials' $TOKEN_URL | head -20` — if you see HTML, you are not hitting the token endpoint
3. If a WAF/sidecar/mesh is rewriting responses, add an exclusion for the token endpoint or route the provider around it

Example fix

# before
HEADROOM_OAUTH2_TOKEN_URL=https://login.microsoftonline.com/<tenant>/oauth2   # HTML discovery page, 200

# after
HEADROOM_OAUTH2_TOKEN_URL=https://login.microsoftonline.com/<tenant>/oauth2/v2.0/token

import json, urllib.request

def token_endpoint_speaks_json(url: str) -> bool:
    try:
        req = urllib.request.Request(url, data=b"grant_type=client_credentials", method="POST")
        with urllib.request.urlopen(req, timeout=5) as r:
            ct = r.headers.get("Content-Type", "")
            body = r.read(64)
        return "json" in ct or body.lstrip()[:1] == b"{"
    except Exception:
        return False

if not token_endpoint_speaks_json(url):
    raise RuntimeError(f"{url} does not return JSON — check the token path")

def looks_like_token_url(url: str) -> bool:
    return url.rstrip('/').endswith(("/token", "oauth/token", "openid-connect/token"))

from headroom_oauth2.provider import OAuth2Error

try:
    token = provider.get_token()
except OAuth2Error as e:
    if "non-JSON" in str(e):
        raise RuntimeError("token_url points at a non-API page; verify the IdP token path") from e
    raise

When it happens

Trigger: token_url pointing at a page that returns 200 HTML (e.g. the IdP's login page because the `/token` path is wrong or the endpoint expects browser flows), a captive portal/WAF intercepting with a 200 block page, or a gateway rewriting the response.

Common situations: Token URL copy-paste errors landing on the IdP UI base path instead of the token endpoint; service mesh sidecars injecting an HTML error page with 200; proxies that replace responses; OAuth endpoints that require a trailing path segment (e.g. `/oauth2/v2.0/token` truncated to `/oauth2`).

Related errors

• remote Kompress response field 'compressed' must be a string
• token endpoint returned HTTP {e.code}
• invalid pipeline config TOML: {0}
• recommendations file not found: {0}
• items_json must be JSON: {e}

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/b908b555b8087361. Report an issue: GitHub.