What causes "URL column {url_column} not found in the csv file." in stanford-oval/storm?

Inspect the CSV header (pd.read_csv(path).columns.tolist()) and pass the exact existing column name as url_column.

stanford-oval/storm · error · ValueError

URL column {url_column} not found in the csv file.

Error message

URL column {url_column} not found in the csv file.

What it means

Raised by create_or_update_vector_store in knowledge_storm/utils.py when the DataFrame loaded from the CSV does not contain the column named by the url_column argument. The function requires both a content column and a URL column to build Documents (the URL becomes per-document metadata), so it validates the CSV schema up front and aborts if the URL column is missing. This is a data-contract error: the caller's configuration does not match the actual CSV header.

Source

Thrown at knowledge_storm/utils.py:249

            )
        else:
            raise ValueError(
                "Invalid vector_db_mode. Please provide either 'online' or 'offline'."
            )
        if qdrant is None:
            raise ValueError("Qdrant client is not initialized.")

        # read the csv file
        import pandas as pd

        df = pd.read_csv(file_path)
        # check that content column exists and url column exists
        if content_column not in df.columns:
            raise ValueError(
                f"Content column {content_column} not found in the csv file."
            )
        if url_column not in df.columns:
            raise ValueError(f"URL column {url_column} not found in the csv file.")

        documents = [
            Document(
                page_content=row[content_column],
                metadata={
                    "title": row.get(title_column, ""),
                    "url": row[url_column],
                    "description": row.get(desc_column, ""),
                },
            )
            for row in df.to_dict(orient="records")
        ]

        # split the documents
        from langchain_text_splitters import RecursiveCharacterTextSplitter

        text_splitter = RecursiveCharacterTextSplitter(
            chunk_size=chunk_size,

View on GitHub (pinned to fb951af774)

Solutions

Inspect the CSV header (pd.read_csv(path).columns.tolist()) and pass the exact existing column name as url_column.
Fix the CSV: rename the link column to the expected url_column value, or strip whitespace/BOM from headers.
If the file uses a different delimiter or encoding, load it correctly (pd.read_csv(path, sep=';', encoding='utf-8-sig')) before calling the function, or fix the source export.
If no URL column exists but URLs are derivable, add one (e.g. df['url'] = base_url) and re-save before ingestion.

Example fix

// before
create_or_update_vector_store("data.csv", content_column="text", url_column="url")
# ValueError: URL column url not found in the csv file.

// after
import pandas as pd
df = pd.read_csv("data.csv")
print(df.columns.tolist())  # e.g. ['text', 'link']
create_or_update_vector_store("data.csv", content_column="text", url_column="link")

Defensive patterns

Strategy: validation

Validate before calling

import pandas as pd

def validate_csv_columns(csv_path: str, content_column: str, url_column: str) -> None:
    df = pd.read_csv(csv_path, nrows=0)
    missing = [c for c in (content_column, url_column) if c not in df.columns]
    if missing:
        raise ValueError(f"Missing columns {missing}; available: {list(df.columns)}")

Type guard

def has_required_columns(df: pd.DataFrame, url_column: str) -> bool:
    return url_column in df.columns

Try / catch

try:
    create_or_update_vector_store(csv_path, content_column="text", url_column="url")
except ValueError as e:
    if "URL column" in str(e):
        cols = pd.read_csv(csv_path, nrows=0).columns.tolist()
        # pick the real link-like column or fail with context
        candidates = [c for c in cols if c.strip().lower() in {"url", "link", "source"}]
        if not candidates:
            raise
        create_or_update_vector_store(csv_path, content_column="text", url_column=candidates[0])
    else:
        raise

Prevention

Print df.columns.tolist() once when onboarding any new CSV source.
Normalize headers on ingest: df.columns = df.columns.str.strip().str.lower().
Standardize crawl/export scripts to always emit a url column.
Validate required columns with nrows=0 reads before starting an expensive vector-store build.

When it happens

Trigger: Calling create_or_update_vector_store(csv_path, url_column='url') (or via main/CLI) where the CSV's header has no column literally named 'url' — e.g. the column is named 'link', 'URL ' (trailing space), or is absent entirely. Also triggered by case mismatches ('URL' vs 'url'), CSVs with a different delimiter so pandas parses one giant column, or passing the wrong csv_path so a different file's schema is checked.

Common situations: Swapping in a new crawl/export CSV whose link column is named differently; typos or trailing whitespace in the header row; passing the default url_column without checking a user-supplied file; reading a TSV or semicolon-delimited file without sep=';\t'/sep=';'; locale/case differences in headers ('Url' vs 'url').

Related errors

AI-assisted analysis of stanford-oval/storm@fb951af774 (2026-08-28). Data as JSON: /api/errors/c8ba7b51df7897f9. Report an issue: GitHub.