microsoft/qlib · error · NotImplementedError

Please implement the `search_records` method.

Error message

Please implement the `search_records` method.

What it means

ExpManager.search_records is an abstract method that must return a pandas DataFrame of records across experiment_ids with metrics.*/params.*/tags.* columns. The base class only raises NotImplementedError; the MLflow subclass implements it via MlflowClient.search_runs. Seeing it means the abstract manager handled the call.

Source

Thrown at qlib/workflow/expm.py:150

        Raise
        -----
        ExpAlreadyExistError
        """
        raise NotImplementedError(f"Please implement the `create_exp` method.")

    def search_records(self, experiment_ids=None, **kwargs):
        """
        Get a pandas DataFrame of records that fit the search criteria of the experiment.
        Inputs are the search criteria user want to apply.

        Returns
        -------
        A pandas.DataFrame of records, where each metric, parameter, and tag
        are expanded into their own columns named metrics.*, params.*, and tags.*
        respectively. For records that don't have a particular metric, parameter, or tag, their
        value will be (NumPy) Nan, None, or None respectively.
        """
        raise NotImplementedError(f"Please implement the `search_records` method.")

    def get_exp(self, *, experiment_id=None, experiment_name=None, create: bool = True, start: bool = False):
        """
        Retrieve an experiment. This method includes getting an active experiment, and get_or_create a specific experiment.

        When user specify experiment id and name, the method will try to return the specific experiment.
        When user does not provide recorder id or name, the method will try to return the current active experiment.
        The `create` argument determines whether the method will automatically create a new experiment according
        to user's specification if the experiment hasn't been created before.

        * If `create` is True:

            * If `active experiment` exists:

                * no id or name specified, return the active experiment.
                * if id or name is specified, return the specified experiment. If no such exp found, create a new experiment with given id or name. If `start` is set to be True, the experiment is set to be active.

            * If `active experiment` not exists:

View on GitHub (pinned to 79633dd950)

Solutions

1. Use the default MLflowExpManager so search_records works out of the box.
2. Implement search_records(self, experiment_ids=None, **kwargs) in your subclass returning a DataFrame.
3. Fall back to querying your backend's native search API for ad-hoc analysis.

Example fix

# before
ExpManager().search_records(experiment_ids=['0'])  # NotImplementedError

# after
df = R.search_records(experiment_ids=['0'], filter_string="metrics.excess_return_with_cost > 0.05")

from qlib.workflow.expm import ExpManager
assert R.exp_manager.__class__.search_records is not ExpManager.search_records, 'search unsupported'

def manager_can_search(mgr) -> bool:
    from qlib.workflow.expm import ExpManager
    return mgr.__class__.search_records is not ExpManager.search_records

When it happens

Trigger: R.search_records(experiment_ids=[...]) when the configured exp_manager class is the base ExpManager or an incomplete subclass; direct exp_manager.search_records(...) calls in custom tooling.

Common situations: Result-analysis code (comparing backtest metrics across experiments) run against a custom/partial tracking backend; forks whose manager subclass never gained search support.

Related errors

• Please implement the `search_records` method.
• Please implement the `start_exp` method.
• Please implement the `end_exp` method.
• Please implement the `create_exp` method.
• Please implement the `_get_exp` method

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/85f1fb7328444155. Report an issue: GitHub.