{"record":{"id":"6a723ab40953ba73","repo":"virattt/ai-hedge-fund","slug":"confidence-out-of-range-confidence","errorCode":null,"errorMessage":"confidence out of range: {confidence}","messagePattern":"confidence out of range: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"hedge_fund/signals/llm_agent.py","lineNumber":130,"sourceCode":"        return build_snapshot(ticker, date, data_client)\n\n    def build_user_prompt(self, snapshot: FundamentalsSnapshot) -> str:\n        \"\"\"Default user prompt: the rendered snapshot. Override to enrich.\"\"\"\n        return snapshot.render()\n\n    # ------------------------------------------------------------------\n    # Private helpers\n    # ------------------------------------------------------------------\n\n    def _parse(self, response: str) -> dict:\n        \"\"\"Extract + validate {signal, confidence, reasoning}.\"\"\"\n        data = extract_json(response)\n        signal = str(data.get(\"signal\", \"\")).lower()\n        if signal not in _SIGNAL_TO_SIGN:\n            raise ValueError(f\"invalid signal {data.get('signal')!r}\")\n        confidence = float(data.get(\"confidence\", 0))\n        if not 0 <= confidence <= 100:\n            raise ValueError(f\"confidence out of range: {confidence}\")\n        return {\n            \"signal\": signal,\n            \"confidence\": confidence,\n            \"reasoning\": str(data.get(\"reasoning\", \"\")),\n        }\n\n    def _to_signal(\n        self,\n        ticker: str,\n        date: str,\n        parsed: dict,\n        key: str,\n        snapshot: FundamentalsSnapshot,\n        cached: bool,\n    ) -> Signal:\n        value = _SIGNAL_TO_SIGN[parsed[\"signal\"]] * parsed[\"confidence\"] / 100.0\n        return Signal(\n            model_name=self.name,","sourceCodeStart":112,"sourceCodeEnd":148,"githubUrl":"https://github.com/virattt/ai-hedge-fund/blob/eff8a7320fcf0b473b135690fa1a5b0d9b022a83/hedge_fund/signals/llm_agent.py#L112-L148","documentation":"Raised by LLMAgent._parse (hedge_fund/signals/llm_agent.py:130) when the parsed JSON's 'confidence' is outside [0, 100] after float() coercion. It defaults to 0 if the field is missing, so this fires specifically on out-of-range numbers: negative, greater than 100, or a model that guessed the wrong scale.","triggerScenarios":"An LLM agent signal where the model replies {\"signal\": \"bullish\", \"confidence\": 150} or {\"confidence\": -10}; models answering on a 0-1 scale put values like 0.85 inside the range (silently read as 0.85/100 — a semantic bug the guard cannot catch), while out-of-range guesses raise; a non-numeric string like \"high\" raises float() ValueError (a different, unguarded error) before this check.","commonSituations":"Prompt doesn't state the confidence scale; model outputs a percentage sign or extreme values; chain-of-thought models hedging with 100+ on combined convictions.","solutions":["State the scale explicitly in the prompt: 'confidence: integer 0-100'.","Catch the ValueError per ticker and retry the LLM call once with corrective feedback.","Normalize in a subclass before validation: if 0 < c <= 1, multiply by 100; clamp into [0, 100] where clamping is acceptable."],"exampleFix":"# before\n# prompt: \"Give confidence.\" -> {\"signal\": \"bullish\", \"confidence\": 150} -> ValueError\n\n# after\n# prompt: 'Respond with JSON: signal is \"bullish\"|\"neutral\"|\"bearish\", confidence is an integer 0-100.'\n# and/or subclass clamp:\nclass MyAgent(LLMAgent):\n    def _parse(self, response):\n        data = extract_json(response)\n        c = float(data.get(\"confidence\", 0))\n        if 0 < c <= 1:\n            c *= 100\n        data[\"confidence\"] = max(0.0, min(100.0, c))\n        ...","handlingStrategy":"fallback","validationCode":"def confidence_in_range(c: object) -> bool:\n    try:\n        return 0 <= float(c) <= 100\n    except (TypeError, ValueError):\n        return False","typeGuard":"def is_valid_confidence(value: object) -> bool:\n    try:\n        c = float(value)\n    except (TypeError, ValueError):\n        return False\n    return 0 <= c <= 100","tryCatchPattern":"for attempt in range(2):\n    try:\n        parsed = agent._parse(response)\n        break\n    except ValueError as e:\n        if \"confidence out of range\" not in str(e) or attempt == 1:\n            raise\n        response = re_ask(agent, ticker, snapshot,\n                           correction=\"confidence must be a number between 0 and 100\")","preventionTips":["State the scale in the prompt: 'confidence: integer 0-100'.","Decide a normalization policy for 0–1-scale replies (×100) in a subclass — the library deliberately does not guess.","Validate confidence before it feeds position sizing; a bogus 100+ would inflate weights even where no exception fires.","Retry the LLM call once on range errors; persistent violations mean the prompt or model is wrong."],"tags":["llm","agent","validation","signals"],"backgroundTag":null,"analyzedSha":"eff8a7320fcf0b473b135690fa1a5b0d9b022a83","analyzedAt":"2026-08-15T00:22:46.567Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}