virattt/ai-hedge-fund · error · ValueError

confidence out of range: {confidence}

Error message

confidence out of range: {confidence}

What it means

Raised by LLMAgent._parse (hedge_fund/signals/llm_agent.py:130) when the parsed JSON's 'confidence' is outside [0, 100] after float() coercion. It defaults to 0 if the field is missing, so this fires specifically on out-of-range numbers: negative, greater than 100, or a model that guessed the wrong scale.

Source

Thrown at hedge_fund/signals/llm_agent.py:130

        return build_snapshot(ticker, date, data_client)

    def build_user_prompt(self, snapshot: FundamentalsSnapshot) -> str:
        """Default user prompt: the rendered snapshot. Override to enrich."""
        return snapshot.render()

    # ------------------------------------------------------------------
    # Private helpers
    # ------------------------------------------------------------------

    def _parse(self, response: str) -> dict:
        """Extract + validate {signal, confidence, reasoning}."""
        data = extract_json(response)
        signal = str(data.get("signal", "")).lower()
        if signal not in _SIGNAL_TO_SIGN:
            raise ValueError(f"invalid signal {data.get('signal')!r}")
        confidence = float(data.get("confidence", 0))
        if not 0 <= confidence <= 100:
            raise ValueError(f"confidence out of range: {confidence}")
        return {
            "signal": signal,
            "confidence": confidence,
            "reasoning": str(data.get("reasoning", "")),
        }

    def _to_signal(
        self,
        ticker: str,
        date: str,
        parsed: dict,
        key: str,
        snapshot: FundamentalsSnapshot,
        cached: bool,
    ) -> Signal:
        value = _SIGNAL_TO_SIGN[parsed["signal"]] * parsed["confidence"] / 100.0
        return Signal(
            model_name=self.name,

View on GitHub (pinned to eff8a7320f)

Solutions

1. State the scale explicitly in the prompt: 'confidence: integer 0-100'.
2. Catch the ValueError per ticker and retry the LLM call once with corrective feedback.
3. Normalize in a subclass before validation: if 0 < c <= 1, multiply by 100; clamp into [0, 100] where clamping is acceptable.

Example fix

# before
# prompt: "Give confidence." -> {"signal": "bullish", "confidence": 150} -> ValueError

# after
# prompt: 'Respond with JSON: signal is "bullish"|"neutral"|"bearish", confidence is an integer 0-100.'
# and/or subclass clamp:
class MyAgent(LLMAgent):
    def _parse(self, response):
        data = extract_json(response)
        c = float(data.get("confidence", 0))
        if 0 < c <= 1:
            c *= 100
        data["confidence"] = max(0.0, min(100.0, c))
        ...

def confidence_in_range(c: object) -> bool:
    try:
        return 0 <= float(c) <= 100
    except (TypeError, ValueError):
        return False

def is_valid_confidence(value: object) -> bool:
    try:
        c = float(value)
    except (TypeError, ValueError):
        return False
    return 0 <= c <= 100

for attempt in range(2):
    try:
        parsed = agent._parse(response)
        break
    except ValueError as e:
        if "confidence out of range" not in str(e) or attempt == 1:
            raise
        response = re_ask(agent, ticker, snapshot,
                           correction="confidence must be a number between 0 and 100")

When it happens

Trigger: An LLM agent signal where the model replies {"signal": "bullish", "confidence": 150} or {"confidence": -10}; models answering on a 0-1 scale put values like 0.85 inside the range (silently read as 0.85/100 — a semantic bug the guard cannot catch), while out-of-range guesses raise; a non-numeric string like "high" raises float() ValueError (a different, unguarded error) before this check.

Common situations: Prompt doesn't state the confidence scale; model outputs a percentage sign or extreme values; chain-of-thought models hedging with 100+ on combined convictions.

Related errors

• invalid signal {data.get('signal')!r}
• no JSON object found in response: {text[:200]!r}
• unknown rebalance cadence {cadence!r}
• duplicate strategy names: {sorted(duplicates)}
• universe is empty — a run needs at least one ticker

AI-assisted analysis of virattt/ai-hedge-fund@eff8a7320f (2026-08-15). Data as JSON: /api/errors/6a723ab40953ba73. Report an issue: GitHub.