assafelovic/gpt-researcher · error · Exception

FireCrawl API key not found. Please set the FIRECRAWL_API_KE

Error message

FireCrawl API key not found. Please set the FIRECRAWL_API_KEY environment variable.

What it means

The FireCrawl scraper requires an API key for the FireCrawl service. During __init__, get_api_key() reads os.environ['FIRECRAWL_API_KEY']; if the lookup raises KeyError, a plain Exception is thrown telling you to set the variable. The scraper never reaches the network without a key.

Source

Thrown at gpt_researcher/scraper/firecrawl/firecrawl.py:39

class FireCrawl:

    def __init__(self, link, session=None):
        self.link = link
        self.session = session
        from firecrawl import FirecrawlApp
        self.firecrawl = FirecrawlApp(api_key=self.get_api_key(), api_url=self.get_server_url())

    def get_api_key(self) -> str:
        """
        Gets the FireCrawl API key
        Returns:
        Api key (str)
        """
        try:
            api_key = os.environ["FIRECRAWL_API_KEY"]
        except KeyError:
            raise Exception(
                "FireCrawl API key not found. Please set the FIRECRAWL_API_KEY environment variable.")
        return api_key

    def get_server_url(self) -> str:
        """
        Gets the FireCrawl server URL.
        Default to official FireCrawl server ('https://api.firecrawl.dev').
        Returns:
        server url (str)
        """
        try:
            server_url = os.environ["FIRECRAWL_SERVER_URL"]
        except KeyError:
            server_url = 'https://api.firecrawl.dev'
        return server_url

    def scrape(self) -> tuple:
        """

View on GitHub (pinned to 6f998577d5)

Solutions

Set the variable: export FIRECRAWL_API_KEY='fc-...' (or add FIRECRAWL_API_KEY=... to .env and load it at startup).
For self-hosted FireCrawl, also set FIRECRAWL_API_URL and create/use the instance's API key.
Inject the secret into Docker/CI via environment/secrets configuration.
Check availability first: python -c "import os; print(bool(os.environ.get('FIRECRAWL_API_KEY')))".

Example fix

# before
scraper = FireCrawl(url='https://example.com')  # raises: FireCrawl API key not found...

# after
import os
os.environ['FIRECRAWL_API_KEY'] = 'fc-your-key'
scraper = FireCrawl(url='https://example.com')

Defensive patterns

Strategy: validation

Validate before calling

import os

if not os.environ.get('FIRECRAWL_API_KEY'):
    raise SystemExit('FIRECRAWL_API_KEY is not set — required by the FireCrawl scraper')

Try / catch

try:
    scraper = FireCrawl(url='https://example.com')
except Exception as e:
    if 'API key not found' in str(e):
        raise SystemExit(f'Fix configuration: {e}')
    raise

Prevention

Set FIRECRAWL_API_KEY (and FIRECRAWL_API_URL for self-hosted) in .env and load it at startup.
Validate scraper provider keys during app bootstrap, fail fast with a clear message.
Use CI/Docker secrets to inject the key into every deploy target.
When switching scrapers (bs4 -> firecrawl), audit which new env vars are required.

When it happens

Trigger: Constructing the FireCrawl scraper (or selecting firecrawl as the scraper provider in GPT Researcher config) when FIRECRAWL_API_KEY is not set in the environment. Self-hosted FireCrawl users still need this key set unless their fork bypasses it.

Common situations: Forgetting the env var when switching from another scraper (e.g., bs4) to firecrawl; .env not loaded before scraper construction; container/CI secrets missing; using a self-hosted FireCrawl server but never generating/setting an API key for it.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of assafelovic/gpt-researcher@6f998577d5 (2026-08-28). Data as JSON: /api/errors/366f4cfd468baf05. Report an issue: GitHub.