{"record":{"id":"366f4cfd468baf05","repo":"assafelovic/gpt-researcher","slug":"firecrawl-api-key-not-found-please-set-the-firecr","errorCode":null,"errorMessage":"FireCrawl API key not found. Please set the FIRECRAWL_API_KEY environment variable.","messagePattern":"FireCrawl API key not found\\. Please set the FIRECRAWL_API_KEY environment variable\\.","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"error","filePath":"gpt_researcher/scraper/firecrawl/firecrawl.py","lineNumber":39,"sourceCode":"\nclass FireCrawl:\n\n    def __init__(self, link, session=None):\n        self.link = link\n        self.session = session\n        from firecrawl import FirecrawlApp\n        self.firecrawl = FirecrawlApp(api_key=self.get_api_key(), api_url=self.get_server_url())\n\n    def get_api_key(self) -> str:\n        \"\"\"\n        Gets the FireCrawl API key\n        Returns:\n        Api key (str)\n        \"\"\"\n        try:\n            api_key = os.environ[\"FIRECRAWL_API_KEY\"]\n        except KeyError:\n            raise Exception(\n                \"FireCrawl API key not found. Please set the FIRECRAWL_API_KEY environment variable.\")\n        return api_key\n\n    def get_server_url(self) -> str:\n        \"\"\"\n        Gets the FireCrawl server URL.\n        Default to official FireCrawl server ('https://api.firecrawl.dev').\n        Returns:\n        server url (str)\n        \"\"\"\n        try:\n            server_url = os.environ[\"FIRECRAWL_SERVER_URL\"]\n        except KeyError:\n            server_url = 'https://api.firecrawl.dev'\n        return server_url\n\n    def scrape(self) -> tuple:\n        \"\"\"","sourceCodeStart":21,"sourceCodeEnd":57,"githubUrl":"https://github.com/assafelovic/gpt-researcher/blob/6f998577d547b1e54ec662dac63583aa11e3b84b/gpt_researcher/scraper/firecrawl/firecrawl.py#L21-L57","documentation":"The FireCrawl scraper requires an API key for the FireCrawl service. During __init__, get_api_key() reads os.environ['FIRECRAWL_API_KEY']; if the lookup raises KeyError, a plain Exception is thrown telling you to set the variable. The scraper never reaches the network without a key.","triggerScenarios":"Constructing the FireCrawl scraper (or selecting firecrawl as the scraper provider in GPT Researcher config) when FIRECRAWL_API_KEY is not set in the environment. Self-hosted FireCrawl users still need this key set unless their fork bypasses it.","commonSituations":"Forgetting the env var when switching from another scraper (e.g., bs4) to firecrawl; .env not loaded before scraper construction; container/CI secrets missing; using a self-hosted FireCrawl server but never generating/setting an API key for it.","solutions":["Set the variable: export FIRECRAWL_API_KEY='fc-...' (or add FIRECRAWL_API_KEY=... to .env and load it at startup).","For self-hosted FireCrawl, also set FIRECRAWL_API_URL and create/use the instance's API key.","Inject the secret into Docker/CI via environment/secrets configuration.","Check availability first: python -c \"import os; print(bool(os.environ.get('FIRECRAWL_API_KEY')))\"."],"exampleFix":"# before\nscraper = FireCrawl(url='https://example.com')  # raises: FireCrawl API key not found...\n\n# after\nimport os\nos.environ['FIRECRAWL_API_KEY'] = 'fc-your-key'\nscraper = FireCrawl(url='https://example.com')","handlingStrategy":"validation","validationCode":"import os\n\nif not os.environ.get('FIRECRAWL_API_KEY'):\n    raise SystemExit('FIRECRAWL_API_KEY is not set — required by the FireCrawl scraper')","typeGuard":null,"tryCatchPattern":"try:\n    scraper = FireCrawl(url='https://example.com')\nexcept Exception as e:\n    if 'API key not found' in str(e):\n        raise SystemExit(f'Fix configuration: {e}')\n    raise","preventionTips":["Set FIRECRAWL_API_KEY (and FIRECRAWL_API_URL for self-hosted) in .env and load it at startup.","Validate scraper provider keys during app bootstrap, fail fast with a clear message.","Use CI/Docker secrets to inject the key into every deploy target.","When switching scrapers (bs4 -> firecrawl), audit which new env vars are required."],"tags":["firecrawl","api-key","environment-variable","scraper","configuration"],"backgroundTag":"missing-env-var","analyzedSha":"6f998577d547b1e54ec662dac63583aa11e3b84b","analyzedAt":"2026-08-28T17:50:07.383Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}