virattt/ai-hedge-fund · error · ValueError

invalid signal {data.get('signal')!r}

Error message

invalid signal {data.get('signal')!r}

What it means

Raised by LLMAgent._parse (hedge_fund/signals/llm_agent.py:127) when the LLM's JSON response has a 'signal' field that (after lowercasing) is not one of 'bullish', 'neutral', 'bearish' (_SIGNAL_TO_SIGN maps exactly those three to +1/0/-1). Any other value — 'buy', 'hold', 'strong sell', 'N/A', or a missing field (defaults to '' via data.get('signal', '')) — is rejected.

Source

Thrown at hedge_fund/signals/llm_agent.py:127

        (macro, news); when a second snapshot TYPE exists, extract the
        implicit interface (ticker/as_of/content_hash/render) into a
        Protocol — not before."""
        return build_snapshot(ticker, date, data_client)

    def build_user_prompt(self, snapshot: FundamentalsSnapshot) -> str:
        """Default user prompt: the rendered snapshot. Override to enrich."""
        return snapshot.render()

    # ------------------------------------------------------------------
    # Private helpers
    # ------------------------------------------------------------------

    def _parse(self, response: str) -> dict:
        """Extract + validate {signal, confidence, reasoning}."""
        data = extract_json(response)
        signal = str(data.get("signal", "")).lower()
        if signal not in _SIGNAL_TO_SIGN:
            raise ValueError(f"invalid signal {data.get('signal')!r}")
        confidence = float(data.get("confidence", 0))
        if not 0 <= confidence <= 100:
            raise ValueError(f"confidence out of range: {confidence}")
        return {
            "signal": signal,
            "confidence": confidence,
            "reasoning": str(data.get("reasoning", "")),
        }

    def _to_signal(
        self,
        ticker: str,
        date: str,
        parsed: dict,
        key: str,
        snapshot: FundamentalsSnapshot,
        cached: bool,
    ) -> Signal:

View on GitHub (pinned to eff8a7320f)

Solutions

1. Fix the agent's prompt to demand exactly one of bullish|neutral|bearish (check build_prompt / the system prompt in your agent subclass) and include a matching example.
2. Catch this ValueError per ticker in the signal loop and retry the LLM call once — enum drift is transient.
3. If you control the subclass, override _parse (or normalize upstream) to map synonyms ('buy'->'bullish', 'sell'->'bearish', 'hold'->'neutral') before validation.
4. Switch to a model that follows format instructions more reliably.

Example fix

# before
# prompt: "Give your view on {ticker}." -> model replies {"signal": "buy"} -> ValueError: invalid signal 'buy'

# after
# prompt: "Respond with JSON: signal must be exactly 'bullish', 'neutral', or 'bearish'."
# and/or normalize + clamp in a subclass before validation:
import json
class MyAgent(LLMAgent):
    def _parse(self, response: str) -> dict:
        data = extract_json(response)
        alias = {"buy": "bullish", "hold": "neutral", "sell": "bearish"}
        s = str(data.get("signal", "")).lower().strip()
        data["signal"] = alias.get(s, s)
        c = float(data.get("confidence", 0) or 0)
        if 0 < c <= 1:
            c *= 100
        data["confidence"] = max(0.0, min(100.0, c))
        return super()._parse(json.dumps(data))

VALID_SIGNALS = {"bullish", "neutral", "bearish"}

def signal_is_valid(s: object) -> bool:
    return isinstance(s, str) and s.lower() in VALID_SIGNALS

_VALID = {"bullish", "neutral", "bearish"}

def is_valid_signal(value: object) -> bool:
    return isinstance(value, str) and value.lower().strip() in _VALID

for attempt in range(2):
    try:
        parsed = agent._parse(response)
        break
    except ValueError as e:
        if "invalid signal" not in str(e) or attempt == 1:
            raise
        response = re_ask(agent, ticker, snapshot,
                           correction="signal must be exactly 'bullish', 'neutral', or 'bearish'")

When it happens

Trigger: An LLM agent signal call where the model answers with synonym vocabulary: {"signal": "buy"} or {"signal": "BUY"} is fine after lowercasing only if exactly 'bullish'; 'accumulate', 'hold', 'sell' fail; omitting 'signal' entirely fails on ''. The parse runs after extract_json succeeded, so the JSON itself was valid — only the enum is wrong.

Common situations: Prompt doesn't pin the exact vocabulary and the model improvises; a different model (or version) interprets the schema loosely; few-shot examples use 'buy/sell' wording; the model substitutes localized or decorated words ('bearish!', 'neutral-ish').

Related errors

• confidence out of range: {confidence}
• no JSON object found in response: {text[:200]!r}
• No v2 client for {provider} (model {model}). Supported: {',
• {env_var} not found. Set it in your .env to use {provider} m

AI-assisted analysis of virattt/ai-hedge-fund@eff8a7320f (2026-08-15). Data as JSON: /api/errors/b3fb300c23407a18. Report an issue: GitHub.