ZhuLinsen/daily_stock_analysis · warning · IntelligenceServiceError

invalid NewsNow response: expected object

Error message

invalid NewsNow response: expected object

What it means

IntelligenceServiceError('invalid NewsNow response: expected object') is raised by _parse_newsnow_payload when the decoded JSON payload from a NewsNow endpoint is not a dict (JSON object). The NewsNow parser expects a top-level object containing an 'items' list; a top-level array, string, number, or null fails this check.

Source

Thrown at src/services/intelligence_service.py:612

                raise IntelligenceServiceError("source url must not target private or local network addresses")

    def _parse_feed(self, content: bytes, *, source_name: str, limit: int) -> List[FeedEntry]:
        try:
            root = ET.fromstring(content)
        except ET.ParseError as exc:
            raise IntelligenceServiceError(f"invalid RSS/Atom feed: {exc}") from exc
        tag = self._strip_ns(root.tag).lower()
        if tag == "rss":
            nodes = root.findall("./channel/item")
            return [entry for entry in (self._parse_rss_item(node, source_name) for node in nodes[:limit]) if entry]
        if tag == "feed":
            nodes = root.findall("./{*}entry") or root.findall("./entry")
            return [entry for entry in (self._parse_atom_entry(node, source_name) for node in nodes[:limit]) if entry]
        raise IntelligenceServiceError("unsupported feed format; expected RSS or Atom")

    def _parse_newsnow_payload(self, payload: Any, *, source_name: str, limit: int) -> List[FeedEntry]:
        if not isinstance(payload, dict):
            raise IntelligenceServiceError("invalid NewsNow response: expected object")
        items = payload.get("items")
        if not isinstance(items, list):
            raise IntelligenceServiceError("invalid NewsNow response: missing items")
        entries = []
        for item in items[:limit]:
            if not isinstance(item, dict):
                continue
            extra = item.get("extra") if isinstance(item.get("extra"), dict) else {}
            published_raw = item.get("pubDate") or extra.get("date")
            entries.append(self._build_entry(
                str(item.get("title") or ""),
                str(extra.get("info") or extra.get("hover") or ""),
                str(item.get("url") or item.get("mobileUrl") or ""),
                source_name,
                self._parse_datetime_or_timestamp(published_raw),
            ))
        return [entry for entry in entries if entry]

View on GitHub (pinned to 5159bd72e8)

Solutions

1. Inspect the raw response: curl -sL <newsnow-url> | python -m json.tool and check the top-level type
2. If the endpoint returns {'items': [...]}, make sure nothing in the chain (proxy, transform) unwraps it into an array
3. Point the source at the correct NewsNow endpoint that returns the documented {items: [...]} shape
4. If the body is an auth/error message, fix credentials or the URL before parsing

Example fix

# before: endpoint returns a bare array [ {...}, {...} ]
url = "https://newsnow.example.com/api/sitemap?id=36kr"

# after: endpoint that returns {"items": [...]}
url = "https://newsnow.example.com/api/s?id=36kr"

import json

def is_newsnow_object(payload_text: str) -> bool:
    try:
        return isinstance(json.loads(payload_text), dict)
    except json.JSONDecodeError:
        return False

def is_newsnow_payload(payload: Any) -> bool:
    return isinstance(payload, dict) and isinstance(payload.get("items"), list)

try:
    entries = svc._parse_newsnow_payload(payload, source_name=src, limit=20)
except IntelligenceServiceError as exc:
    if "expected object" in str(exc):
        logger.warning("NewsNow endpoint shape changed for %s", src)

When it happens

Trigger: Calling a NewsNow-type source whose API returns a top-level JSON array of items, a bare string/number, 'null', or an error payload like "unauthorized" instead of an object.

Common situations: Wrong NewsNow endpoint variant (some deployments return a bare array), an API version change on self-hosted NewsNow instances, an auth failure returning a plain-text body that happens to decode as a JSON scalar, or a proxy returning a JSON-encoded error string.

Related errors

• parameters must be an object
• invalid RSS/Atom feed: {exc}
• unsupported feed format; expected RSS or Atom
• invalid_json
• [AlphaVantage] No time series data for {symbol}

AI-assisted analysis of ZhuLinsen/daily_stock_analysis@5159bd72e8 (2026-08-15). Data as JSON: /api/errors/ada21b51f3b354e9. Report an issue: GitHub.