What causes "hf paper returned malformed JSON: ${err?.message ?? err}" in jackwener/OpenCLI?

Inspect the actual body: curl -sS https://huggingface.co/api/papers/ | head — if it is HTML, a proxy/challenge page is intercepting the request.

jackwener/OpenCLI · error · CommandExecutionError

hf paper returned malformed JSON: ${err?.message ?? err}

Error message

hf paper returned malformed JSON: ${err?.message ?? err}

What it means

CommandExecutionError(`hf paper returned malformed JSON: ${err?.message ?? err}`) thrown when resp.json() rejects — i.e. HF returned an OK (2xx) response whose body is not valid JSON (the wrapped message is the underlying SyntaxError from the JSON parser).

Source

Thrown at clis/hf/paper.js:58

        }
        if (resp.status === 404) {
            throw new EmptyResultError('hf paper', `Hugging Face has no paper page for "${raw}".`);
        }
        if (resp.status === 429) {
            throw new CommandExecutionError(
                'hf paper returned HTTP 429 (rate limited)',
                'Hugging Face throttles unauthenticated traffic; wait a few seconds and retry.',
            );
        }
        if (!resp.ok) {
            throw new CommandExecutionError(`hf paper returned HTTP ${resp.status}`);
        }
        let body;
        try {
            body = await resp.json();
        }
        catch (err) {
            throw new CommandExecutionError(`hf paper returned malformed JSON: ${err?.message ?? err}`);
        }
        if (!body || typeof body !== 'object' || !body.id) {
            throw new EmptyResultError('hf paper', `Hugging Face returned no paper data for "${raw}".`);
        }
        const authors = Array.isArray(body.authors)
            ? body.authors.map((a) => (typeof a === 'object' && a ? (a.name || a.fullname || '') : String(a))).filter(Boolean)
            : [];
        const aiKeywords = Array.isArray(body.ai_keywords) ? body.ai_keywords.join(', ') : '';
        return [{
            id: String(body.id),
            title: String(body.title ?? ''),
            authors: authors.join(', '),
            publishedAt: String(body.publishedAt ?? '').slice(0, 10),
            upvotes: body.upvotes != null ? Number(body.upvotes) : null,
            aiKeywords,
            summary: String(body.summary ?? ''),
            aiSummary: String(body.ai_summary ?? ''),
            url: `${endpoint}/papers/${body.id}`,

View on GitHub (pinned to 49907e53dc)

Solutions

Inspect the actual body: curl -sS https://huggingface.co/api/papers/<id> | head — if it is HTML, a proxy/challenge page is intercepting the request.
Bypass or fix the proxy (check HTTP_PROXY/HTTPS_PROXY; disable intercepting middleboxes) so the raw JSON is delivered.
Retry — a truncated body on a flaky connection is usually transient.
If using a custom HF_ENDPOINT, verify the mirror returns JSON for /api/papers or revert to the default endpoint.
Wait out any Cloudflare/browser-challenge situation or query from a network/challenge-exempt IP.

Example fix

// before (proxy injecting HTML)
export HTTPS_PROXY=http://intercept.corp:8080
opencli hf paper 1706.03762   // malformed JSON
// after
unset HTTPS_PROXY   # or bypass the interceptor for huggingface.co
opencli hf paper 1706.03762

Defensive patterns

Strategy: validation

Validate before calling

// validate the endpoint actually returns JSON before invoking the CLI
const res = await fetch(`https://huggingface.co/api/papers/${id}`);
const text = await res.text();
let parsed;
try { parsed = JSON.parse(text); } catch { throw new Error('endpoint returned non-JSON body — check proxy/captive portal/HF_ENDPOINT'); }

Type guard

function isPaperPayload(v) {
  return typeof v === 'object' && v !== null && 'id' in v && typeof v.id === 'string' && v.id.length > 0;
}

Try / catch

try {
  await run(['opencli', 'hf paper', id]);
} catch (e) {
  if (String(e.message).startsWith('hf paper returned malformed JSON:')) {
    // fetch the URL manually, inspect the raw body (HTML? challenge page? truncation?), fix proxy, then retry
  } else throw e;
}

Prevention

Inspect the raw response body with curl when JSON parsing fails — HTML usually means a proxy/challenge page.
Bypass intercepting proxies or captive portals for huggingface.co.
Retry on flaky networks — truncated bodies are often transient.
Verify custom HF_ENDPOINT mirrors serve application/json.
Validate JSON parses (and has an .id field) before feeding results downstream.

When it happens

Trigger: The fetch succeeded with a 2xx status but the response body fails JSON.parse: an HTML error/interstitial page served with 200, a truncated response from an interrupted connection, or a proxy/CAPTCHA page injected into the body.

Common situations: A transparent proxy or captive portal returning HTML with status 200; HF returning a Cloudflare/challenge page; gzip/body truncation on flaky networks; a custom HF_ENDPOINT mirror that serves HTML instead of JSON.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/7a6acf904dec6f61. Report an issue: GitHub.