assafelovic/gpt-researcher · error · ImportError

Selenium is required but not installed. See error message ab

Error message

Selenium is required but not installed. See error message above for installation instructions.

What it means

BrowserScraper lazily imports Selenium in _import_selenium() during __init__. If the selenium package (or a transitive import like its browser driver bits) cannot be imported, the scraper prints installation instructions to stdout and re-raises an ImportError chained from the original exception. The scraper cannot start without Selenium installed.

Source

Thrown at gpt_researcher/scraper/browser/browser.py:95

        try:
            global webdriver, By, EC, WebDriverWait, TimeoutException, WebDriverException
            from selenium import webdriver
            from selenium.webdriver.common.by import By
            from selenium.webdriver.support import expected_conditions as EC
            from selenium.webdriver.support.wait import WebDriverWait
            from selenium.common.exceptions import TimeoutException, WebDriverException

            global ChromeOptions, FirefoxOptions, SafariOptions
            from selenium.webdriver.chrome.options import Options as ChromeOptions
            from selenium.webdriver.firefox.options import Options as FirefoxOptions
            from selenium.webdriver.safari.options import Options as SafariOptions
        except ImportError as e:
            print(f"Failed to import Selenium: {str(e)}")
            print("Please install Selenium and its dependencies to use BrowserScraper.")
            print("You can install Selenium using pip:")
            print("    pip install selenium")
            print("If you're using a virtual environment, make sure it's activated.")
            raise ImportError(
                "Selenium is required but not installed. See error message above for installation instructions.") from e

    def setup_driver(self) -> None:
        # print(f"Setting up {self.selenium_web_browser} driver...")

        options_available = {
            "chrome": ChromeOptions,
            "firefox": FirefoxOptions,
            "safari": SafariOptions,
        }

        options = options_available[self.selenium_web_browser]()
        options.add_argument(f"user-agent={self.user_agent}")
        if self.headless:
            options.add_argument("--headless")
        options.add_argument("--enable-javascript")

        try:

View on GitHub (pinned to 6f998577d5)

Solutions

Install Selenium in the active environment: pip install selenium (plus pip install webdriver-manager if the project uses it).
Confirm the right interpreter: which python / pip -V, and reinstall if pip installed to a different env.
If the original import error names a missing submodule, reinstall cleanly: pip uninstall selenium && pip install selenium.
As an alternative, use a scraper backend without Selenium (e.g., the plain http/requests scraper or NoDriverScraper with zendriver).

Example fix

# before
scraper = BrowserScraper(url)  # ImportError: Selenium is required but not installed...

# after
# pip install selenium webdriver-manager
scraper = BrowserScraper(url)

Defensive patterns

Strategy: validation

Validate before calling

import importlib.util

if importlib.util.find_spec('selenium') is None:
    raise SystemExit('selenium is required for BrowserScraper: pip install selenium')

Type guard

def selenium_available() -> bool:
    try:
        importlib.util.find_spec('selenium')
        import selenium  # catches broken installs too
        return True
    except (ImportError, ModuleNotFoundError):
        return False

Try / catch

try:
    scraper = BrowserScraper(url)
except ImportError as e:
    if 'Selenium is required' in str(e):
        scraper = HttpScraper(url)  # non-browser fallback
    else:
        raise

Prevention

Install selenium (and webdriver-manager) in every env that uses the browser scraper.
Feature-detect selenium at startup and fall back to a plain HTTP scraper.
Pin selenium versions in requirements to avoid broken upgrades.
In containers, also install/verify the browser binary (chromium) and driver.

When it happens

Trigger: Creating a BrowserScraper when 'import selenium' fails — selenium not installed, installed in another environment, or a broken/partial install where a submodule import raises ImportError. __init__ calls _import_selenium(), which catches ImportError, prints help, and raises this error 'from e'.

Common situations: Using the browser scraper feature without pip install selenium; running under a different interpreter than expected; selenium partially upgraded leaving broken modules; missing webdriver-manager or a driver binary — though the driver itself usually fails later in setup_driver, broken imports surface here.

Understand the failure class

Background: "X is not installed. Please install it with pip install Y": missing optional dependency errors — ImportError/ValueError raised when a library's optional extra was never installed — this error's family across 22 libraries.

Related errors

AI-assisted analysis of assafelovic/gpt-researcher@6f998577d5 (2026-08-28). Data as JSON: /api/errors/decd2adda5f02f71. Report an issue: GitHub.