ScrapeGraphAI/Scrapegraph-ai · error · ValueError

SmartScraperMultiBatchGraph only supports OpenAI models. Got

Error message

SmartScraperMultiBatchGraph only supports OpenAI models. Got provider '{provider}'. Use SmartScraperMultiGraph for other providers.

What it means

SmartScraperMultiBatchGraph.__init__ inspects the llm model string: if it contains '/', the prefix before the slash is treated as the provider, and any provider other than 'openai' is rejected with this ValueError before delegating to super().__init__. The batch implementation relies on OpenAI-specific batch APIs.

Source

Thrown at scrapegraphai/graphs/smart_scraper_multi_batch_graph.py:136

    def __init__(
        self,
        prompt: str,
        source: List[str],
        config: dict,
        schema: Optional[Type[BaseModel]] = None,
    ):
        self.copy_config = safe_deepcopy(config)
        self.copy_schema = deepcopy(schema)
        self.batch_config = config.get("batch_api", {})

        # Validate that the model is OpenAI-based
        model_str = config.get("llm", {}).get("model", "")
        if "/" in model_str:
            provider = model_str.split("/")[0]
        else:
            provider = ""
        if provider and provider != "openai":
            raise ValueError(
                f"SmartScraperMultiBatchGraph only supports OpenAI models. "
                f"Got provider '{provider}'. "
                f"Use SmartScraperMultiGraph for other providers."
            )

        super().__init__(prompt, config, source, schema)

    def _create_graph(self) -> BaseGraph:
        """Creates the graph of nodes for the batch scraping pipeline.

        The graph has two phases:
        1. GraphIteratorNode runs _FetchParseOnlyGraph per URL (concurrent)
        2. BatchGenerateAnswerNode submits all prompts via Batch API
        3. MergeAnswersNode combines the results

        Returns:
            BaseGraph: A graph instance representing the batch scraping workflow.
        """

View on GitHub (pinned to 532dfffbf6)

Solutions

Use an OpenAI model for this graph (model string 'openai/gpt-4o' or plain 'gpt-4o').
Use SmartScraperMultiGraph (non-batch) for other providers.
For Azure, pass a model_instance built with AzureChatOpenAI plus model_tokens if batch semantics are not strictly required.

Example fix

# before
config = {'llm': {'model': 'azure_openai/gpt-4o', 'api_key': key}}
graph = SmartScraperMultiBatchGraph(prompt=..., source=..., config=config)

# after
config = {'llm': {'model': 'openai/gpt-4o', 'api_key': key}}
graph = SmartScraperMultiBatchGraph(prompt=..., source=..., config=config)

Defensive patterns

Strategy: validation

Validate before calling

model_str = config.get('llm', {}).get('model', '')
provider = model_str.split('/')[0] if '/' in model_str else 'openai'
if provider != 'openai':
    raise SystemExit(f'Batch graph is OpenAI-only (got {provider}); use SmartScraperMultiGraph')

Type guard

def batch_compatible(cfg: dict) -> bool:
    m = cfg.get('llm', {}).get('model', '')
    return '/' not in m or m.split('/')[0] == 'openai'

Try / catch

try:
    g = SmartScraperMultiBatchGraph(prompt=p, source=s, config=config)
except ValueError as e:
    if 'only supports OpenAI' in str(e):
        g = SmartScraperMultiGraph(prompt=p, source=s, config=config)
    else:
        raise

Prevention

Keep an OpenAI-specific config dict for batch graphs.
Remember bare model names without '/' pass the check — always qualify provider for clarity.
Fall back to SmartScraperMultiGraph for other providers.

When it happens

Trigger: config = {'llm': {'model': 'azure_openai/gpt-4o', ...}} or 'anthropic/claude-...' etc. — any 'provider/model' string where provider != 'openai'. Note: a bare model name without '/' sets provider='' and passes the check.

Common situations: Switching an existing SmartScraperMultiGraph config to the batch variant while keeping a provider-qualified model string; using Azure (azure_openai/...) or openrouter-style model ids, which are unsupported here.

Related errors

AI-assisted analysis of ScrapeGraphAI/Scrapegraph-ai@532dfffbf6 (2026-08-28). Data as JSON: /api/errors/248d3a059648221c. Report an issue: GitHub.