assafelovic/gpt-researcher · error · Exception

Tavily API key not found. Please set the TAVILY_API_KEY envi

Error message

Tavily API key not found. Please set the TAVILY_API_KEY environment variable.

What it means

TavilyExtract.get_api_key raises a generic Exception when the TAVILY_API_KEY environment variable is not set. The Tavily extraction scraper requires this key at construction time, before any scraping happens.

Source

Thrown at gpt_researcher/scraper/tavily_extract/tavily_extract.py:22

class TavilyExtract:

    def __init__(self, link, session=None):
        self.link = link
        self.session = session
        from tavily import TavilyClient
        self.tavily_client = TavilyClient(api_key=self.get_api_key())

    def get_api_key(self) -> str:
        """
        Gets the Tavily API key
        Returns:
        Api key (str)
        """
        try:
            api_key = os.environ["TAVILY_API_KEY"]
        except KeyError:
            raise Exception(
                "Tavily API key not found. Please set the TAVILY_API_KEY environment variable.")
        return api_key

    def scrape(self) -> tuple:
        """
        This function extracts content from a specified link using the Tavily Python SDK, the title and
        images from the link are extracted using the functions from `gpt_researcher/scraper/utils.py`.

        Returns:
          The `scrape` method returns a tuple containing the extracted content, a list of image URLs, and
        the title of the webpage specified by the `self.link` attribute. It uses the Tavily Python SDK to
        extract and clean content from the webpage. If any exception occurs during the process, an error
        message is printed and an empty result is returned.
        """

        try:
            response = self.tavily_client.extract(urls=self.link)
            if not isinstance(response, dict):

View on GitHub (pinned to 6f998577d5)

Solutions

export TAVILY_API_KEY=<key> (get one at tavily.com)
Add TAVILY_API_KEY=... to your .env and ensure load_dotenv() runs before scraper construction
In Docker: pass it with -e TAVILY_API_KEY=... or in the compose environment block
Or switch to a scraper that doesn't need an API key, like 'beautifulsoup' or 'pymupdf'

Example fix

# before
scraper = Scraper('tavily_extract')  # Exception: Tavily API key not found...

# after
import os
os.environ.setdefault('TAVILY_API_KEY', 'tvly-xxxx')
scraper = Scraper('tavily_extract')

Defensive patterns

Strategy: validation

Validate before calling

import os

if scraper_choice == 'tavily_extract' and not os.getenv('TAVILY_API_KEY'):
    raise SystemExit('Set TAVILY_API_KEY or use a non-API scraper')

Try / catch

try:
    result = Scraper('tavily_extract').scrape_with_url(url)
except Exception as e:
    if 'TAVILY_API key' in str(e) or 'Tavily' in str(e):
        result = Scraper('beautifulsoup').scrape_with_url(url)
    else:
        raise

Prevention

Set required API keys in .env and call load_dotenv() early
Use a startup env-var checklist that fails fast on missing keys
Keep a keyless fallback scraper configured for resilience

When it happens

Trigger: Constructing TavilyExtract (or selecting 'tavily_extract' / 'tavily' as the scraper) without TAVILY_API_KEY defined in os.environ — including .env files that were never loaded because python-dotenv's load_dotenv() wasn't called.

Common situations: Missing/duplicate .env file, deploying to Docker/Cloud Run where env vars aren't forwarded, CI pipelines lacking secrets, or a typo in the variable name (e.g. TAVILYKEY).

Related errors

AI-assisted analysis of assafelovic/gpt-researcher@6f998577d5 (2026-08-28). Data as JSON: /api/errors/f30b7a3a393fb692. Report an issue: GitHub.