microsoft/qlib · error · ValueError

Multiple paths are found with prefix '{filename_without_suff

Error message

Multiple paths are found with prefix '{filename_without_suffix}': {paths}

What it means

_find_pickle (qlib/rl/data/pickle_styled.py:80) probes both '<name>.pkl' and '<name>.pkl.backtest'. Because later code must pick exactly one file, finding both raises ValueError("Multiple paths are found with prefix '<name>': [paths]"). This guards against silently backtesting against an ambiguous/stale data version.

Source

Thrown at qlib/rl/data/pickle_styled.py:80

        ]
    if shape == 6:
        return ["$high", "$low", "$open", "$close", "$vwap", "$volume"]
    elif shape == 5:
        return ["$high", "$low", "$open", "$close", "$volume"]
    raise ValueError(f"Unrecognized data shape: {shape}")


def _find_pickle(filename_without_suffix: Path) -> Path:
    suffix_list = [".pkl", ".pkl.backtest"]
    paths: List[Path] = []
    for suffix in suffix_list:
        path = filename_without_suffix.parent / (filename_without_suffix.name + suffix)
        if path.exists():
            paths.append(path)
    if not paths:
        raise FileNotFoundError(f"No file starting with '{filename_without_suffix}' found")
    if len(paths) > 1:
        raise ValueError(f"Multiple paths are found with prefix '{filename_without_suffix}': {paths}")
    return paths[0]


@lru_cache(maxsize=10)  # 10 * 40M = 400MB
def _read_pickle(filename_without_suffix: Path) -> pd.DataFrame:
    df = pd.read_pickle(_find_pickle(filename_without_suffix))
    index_cols = df.index.names

    df = df.reset_index()
    for date_col_name in ["date", "datetime"]:
        if date_col_name in df:
            df[date_col_name] = pd.to_datetime(df[date_col_name])
    df = df.set_index(index_cols)

    return df


class SimpleIntradayBacktestData(BaseIntradayBacktestData):

View on GitHub (pinned to 79633dd950)

Solutions

Delete or move one of the two files so only .pkl or only .pkl.backtest remains
Prefer keeping the fresh .pkl dump and removing the stale .pkl.backtest
Clean the data directory between regeneration runs

Example fix

# before
# dir contains: orders.pkl  AND  orders.pkl.backtest -> ValueError

# after (shell)
rm data/orders.pkl.backtest   # keep the fresh dump
# or in python
for extra in base.parent.glob(base.name + '.pkl.backtest'):
    extra.unlink()

Defensive patterns

Strategy: validation

Validate before calling

found = [p for p in (base.with_name(base.name + '.pkl'), base.with_name(base.name + '.pkl.backtest')) if p.is_file()]
assert len(found) == 1, f'ambiguous pickle files: {found}'

Try / catch

try:
    df = _read_pickle(base)
except ValueError as e:
    if 'Multiple paths' in str(e):
        base.with_name(base.name + '.pkl.backtest').unlink(missing_ok=True)  # keep .pkl
        df = _read_pickle(base)
    else:
        raise

Prevention

Clean stale .pkl.backtest artifacts after regenerating dumps
Use separate output directories per data generation run

When it happens

Trigger: A directory containing both orders_<name>.pkl and orders_<name>.pkl.backtest (or process data equivalents) for the same base name; regenerating backtest artifacts without cleaning the previous .pkl.backtest file.

Common situations: Re-running backtest data generation scripts that leave old .pkl.backtest copies next to new .pkl dumps; switching between backtest and non-backtest data generation modes in one workspace.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/f1367b03eb275e7a. Report an issue: GitHub.