jd-opensource/joyagent-jdgenie · error · AgentGenerationError

Error in generating model output

Error message

Error in generating model output:
{e}

What it means

`_step_stream` in the CI agent raises `AgentGenerationError` when the LLM generation step fails for any reason (API error, malformed streaming response, serialization of model output). The original exception is chained via `from e`, so this is a wrapper around the model-call failure.

Solutions

  1. Inspect the chained cause (`e.__cause__`) for the provider error detail (auth, rate limit, context length)
  2. Validate the model provider API key and model name configuration
  3. Reduce prompt/context size if the cause indicates a token-limit error
  4. Add retry with backoff for transient provider/network failures

Example fix

// before
agent.run(task)  # opaque AgentGenerationError on failure
// after
try:
    agent.run(task)
except AgentGenerationError as e:
    logger.error(f"root cause: {e.__cause__}")
    if is_transient(e.__cause__):
        retry_with_backoff()
    raise
Defensive patterns

Strategy: try-catch

Validate before calling

assert os.environ.get("MODEL_API_KEY"), "model provider API key missing"
assert len(prompt_tokens) < model_context_limit, "prompt exceeds context window"

Type guard

def provider_ready(cfg) -> bool:
    return bool(cfg.api_key) and cfg.model_name in SUPPORTED_MODELS

Try / catch

try:
    result = agent.run(task)
except AgentGenerationError as e:
    cause = e.__cause__
    if is_rate_limit(cause) or is_transient(cause):
        result = retry_with_backoff(lambda: agent.run(task))
    else:
        raise

Prevention

When it happens

Trigger: The agent's generation step calls the model API and the model call or output handling throws — e.g. provider API error, invalid API key, context length exceeded, rate limit, or broken stream.

Common situations: Expired or wrong model provider API key; model name unavailable; prompt exceeding context window; transient provider outages; network drops mid-stream.

Related errors

AI-assisted analysis of jd-opensource/joyagent-jdgenie@2417e0b8b6 (2026-09-08). Data as JSON: /api/errors/b41277f731b35301. Report an issue: GitHub.

Appendix: source

Thrown at genie-tool/genie_tool/tool/ci_agent.py:126

            self.logger.log_markdown(
                content=output_text,
                title="Output message of the LLM:",
                level=LogLevel.DEBUG,
            )
            memory_step.model_output_message = chat_message
            output_text = chat_message.content

            # This adds <end_code> sequence to the history.
            # This will nudge ulterior LLM calls to finish with <end_code>, thus efficiently stopping generation.
            if output_text and output_text.strip().endswith("```"):
                output_text += "<end_code>"
                memory_step.model_output_message.content = output_text

            memory_step.model_output = output_text
            # This put call was missing await

        except Exception as e:
            raise AgentGenerationError(
                f"Error in generating model output:\n{e}", self.logger
            ) from e

        self.logger.log_markdown(
            content=output_text,
            title="Output message of the LLM:",
            level=LogLevel.DEBUG,
        )

        # Parse
        try:
            code_action = fix_final_answer_code(parse_code_blobs(output_text))
        except Exception as e:
            error_msg = (
                f"Error in code parsing:\n{e}\nMake sure to provide correct code blobs."
            )
            raise AgentParsingError(error_msg, self.logger)

View on GitHub (pinned to 2417e0b8b6)