ScrapeGraphAI/Scrapegraph-ai · error · ValueError

LLM configuration must include an 'api_key'.

Error message

LLM configuration must include an 'api_key'.

What it means

GenerateAnswerNode raises this ValueError when none of the state keys it checks (typically 'parsed_doc', 'doc', or 'content', whichever are configured) contain anything to summarize. It means the answer-generation step was reached without any scraped/parseed document ever being stored in the graph state, usually because the upstream fetch or parse node failed, was skipped, or wrote to a different key.

Source

Thrown at scrapegraphai/builders/graph_builder.py:67

        self.config = config
        self.llm = self._create_llm(config["llm"])
        self.nodes_description = self._generate_nodes_description()
        self.chain = self._create_extraction_chain()

    def _create_llm(self, llm_config: dict):
        """
        Creates an instance of the OpenAI class with the provided language model configuration.

        Returns:
            OpenAI: An instance of the OpenAI class.

        Raises:
            ValueError: If 'api_key' is not provided in llm_config.
        """
        llm_defaults = {"temperature": 0, "streaming": True}
        llm_params = {**llm_defaults, **llm_config}
        if "api_key" not in llm_params:
            raise ValueError("LLM configuration must include an 'api_key'.")

        if "gpt-" in llm_params["model"]:
            return ChatOpenAI(llm_params)
        elif "gemini" in llm_params["model"]:
            try:
                from langchain_google_genai import ChatGoogleGenerativeAI
            except ImportError:
                raise ImportError(
                    "langchain_google_genai is not installed. Please install it using 'pip install langchain-google-genai'."
                )
            return ChatGoogleGenerativeAI(llm_params)
        elif "ernie" in llm_params["model"]:
            return ErnieBotChat(llm_params)
        raise ValueError("Model not supported")

    def _generate_nodes_description(self):
        """
        Generates a string description of all available nodes and their arguments.

View on GitHub (pinned to 532dfffbf6)

Solutions

1. Inspect the state right before answer generation (log state.keys() and state.get('doc')) to see which keys exist and whether any content was produced.
2. Verify the upstream FetchNode/ParseNode actually ran and that their output key matches an input key of GenerateAnswerNode in your graph edge definition.
3. If the page content is empty, fix the fetcher (headless_prompt, wait times, or use ChromiumLoader for JS-heavy pages).
4. As a last resort, pre-populate state['doc'] or state['content'] yourself with the text you want summarized.

Example fix

# before
graph = SmartScraperGraph(
    prompt="Summarize",
    source="https://example.com",
    config=graph_config,
)

# after: make sure fetch/parse ran and keys align; or seed state manually
state = graph.initial_state
state["doc"] = "your document text"  # only if you bypass the fetch step

required = {"doc", "parsed_doc", "content"}
has_content = any(state.get(k) for k in required)
if not has_content:
    raise RuntimeError("Fetch/parse produced no content; check upstream nodes") before running the answer step

def has_scrapable_content(state: dict) -> bool:
    return bool(state.get("doc") or state.get("parsed_doc") or state.get("content"))

try:
    result = graph.run()
except ValueError as e:
    if "No content found" in str(e):
        # log state keys, retry with a different loader (e.g. ChromiumLoader)
        ...

When it happens

Trigger: Running a graph whose GenerateAnswerNode input_keys expect 'doc'/'parsed_doc'/'content' while the upstream node stored its output under a different key, or the fetcher returned an empty document (e.g. JS-only page rendered to nothing), or the parse node errored and the graph continued with empty state.

Common situations: Misconfigured graph where FetchNode/ParseNode output keys don't match GenerateAnswerNode input keys; a ChromiumLoader failing silently on heavy-JS sites; changing the parse framework (e.g. mongodb/vault integrations) so state['doc'] is never populated; reusing a custom node that forgets to write its result into state.

Related errors

• langchain_google_genai is not installed. Please install it u
• The 'graphviz' library is required for this functionality. P
• The browserbase module is not installed. Please install it u
• ConditionalNode '{node.node_name}' must have exactly two out
• No audio generated from the text.

AI-assisted analysis of ScrapeGraphAI/Scrapegraph-ai@532dfffbf6 (2026-08-28). Data as JSON: /api/errors/31af54aefeff1949. Report an issue: GitHub.