xtekky/gpt4free · error · ValueError

Invalid JSON data: {rest}

Error message

Invalid JSON data: {rest}

What it means

Raised inside the SSE line parser in g4f/requests/__init__.py: for each `data:` line (after stripping and skipping empty/[DONE] lines) it calls json.loads; any line whose payload is not valid JSON raises this ValueError with the raw fragment. It means the remote sent a malformed or non-JSON payload inside a server-sent-events stream that the caller expected to be JSON frames.

Source

Thrown at g4f/requests/__init__.py:388

    await stop_browser()


async def sse_stream(iter_lines: AsyncIterator[bytes]) -> AsyncIterator[dict]:
    if hasattr(iter_lines, "content"):
        iter_lines = iter_lines.content
    elif hasattr(iter_lines, "iter_lines"):
        iter_lines = iter_lines.iter_lines()
    async for line in iter_lines:
        if line.startswith(b"data:"):
            rest = line[5:].strip()
            if not rest:
                continue
            if rest.startswith(b"[DONE]"):
                break
            try:
                yield json.loads(rest)
            except json.JSONDecodeError:
                raise ValueError(f"Invalid JSON data: {rest}")


async def iter_lines(iter_response: AsyncIterator[bytes], delimiter=None):
    """
    iterate streaming content line by line, separated by ``\\n``.

    Copied from: https://requests.readthedocs.io/en/latest/_modules/requests/models/
    which is under the License: Apache 2.0
    """
    pending = None

    async for chunk in iter_response:
        if pending is not None:
            chunk = pending + chunk
        lines = chunk.split(delimiter) if delimiter else chunk.splitlines()
        pending = (
            lines.pop()
            if lines and lines[-1] and chunk and lines[-1][-1] == chunk[-1]

View on GitHub (pinned to 973504e177)

Solutions

1. Log the offending `rest` fragment (it is included in the message) and match it against what the provider actually sends — usually it reveals an HTML error page or rate-limit notice.
2. If you control the request, re-send it with proper headers/cookies so the provider returns a genuine JSON SSE stream.
3. If parsing third-party streams where non-JSON keep-alive lines are legal, pre-filter lines before passing them to this iterator, or wrap the iteration and skip undecodable lines instead of failing the whole stream.
4. Check for provider API format changes and update g4f (`pip install -U g4f`) since provider adapters track wire formats.

Example fix

// before
async for chunk in iter_lines(response, delimiter=b"\n"):
    data = json.loads(chunk[5:])  # crashes on non-JSON data: lines

// after
async for obj in process_sse_lines(iter_lines(response)):  # library helper
    ...  # or pre-filter: skip lines not starting with b'data:{' when keep-alives are plain text

import json

def is_json_sse_line(line: bytes) -> bool:
    rest = line[5:].strip()
    if not rest or rest.startswith(b"[DONE]"):
        return False
    try:
        json.loads(rest)
        return True
    except json.JSONDecodeError:
        return False

try:
    async for obj in stream:
        handle(obj)
except ValueError as e:
    # payload fragment is embedded in the message; treat stream as corrupt
    log.warning(f"SSE stream corrupt: {e}"); raise StreamCorrupted(e)

When it happens

Trigger: Streaming a provider response where a `data:` line contains HTML (error page), plain text keep-alives, or truncated JSON; a proxy or captive portal injecting non-JSON content; the provider changing its wire format so that lines like `data: event: ping` appear.

Common situations: Provider-side changes to streaming format; rate-limit or Cloudflare HTML challenge pages returned mid-stream instead of JSON; chunked transfer cutting a JSON object across lines when the stream was preprocessed incorrectly before being handed to this iterator.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• Failed to parse message: {chunk.decode(errors='replace')}
• Failed to read response: {chunk.decode(errors='replace')}
• Failed to parse image URL: {chunk.decode(errors='replace')}
• Expected SSE response but got content-type: {content_type}
• result["error"].get("message", result["error"])

AI-assisted analysis of xtekky/gpt4free@973504e177 (2026-08-14). Data as JSON: /api/errors/232fecc3b115a53d. Report an issue: GitHub.