microsoft/qlib · error · NotImplementedError

Please implement the `droplevel` method

Error message

Please implement the `droplevel` method

What it means

SepDataFrame deliberately stubs out pandas' DataFrame.droplevel with NotImplementedError. SepDataFrame is a container that keeps one DataFrame per instrument group in _df_dict and emulates the pandas API lazily; droplevel was never implemented because removing a MultiIndex level has ambiguous semantics across the separated frames.

Source

Thrown at qlib/contrib/data/utils/sepdf.py:127

                        col_name = col_name[0]
                    self._df_dict[_df_dict_key] = df.to_frame(col_name)
                else:
                    df_copy = df.copy()  # avoid changing df
                    df_copy.columns = pd.MultiIndex.from_tuples([(*col_name, *idx) for idx in df.columns.to_list()])
                    self._df_dict[_df_dict_key] = df_copy

    def __delitem__(self, item: str):
        del self._df_dict[item]
        self._update_join()

    def __contains__(self, item):
        return item in self._df_dict

    def __len__(self):
        return len(self._df_dict[self.join])

    def droplevel(self, *args, **kwargs):
        raise NotImplementedError(f"Please implement the `droplevel` method")

    @property
    def columns(self):
        dfs = []
        for k, df in self._df_dict.items():
            df = df.head(0)
            df.columns = pd.MultiIndex.from_product([[k], df.columns])
            dfs.append(df)
        return pd.concat(dfs, axis=1).columns

    # Useless methods
    @staticmethod
    def merge(df_dict: Dict[str, pd.DataFrame], join: str):
        all_df = df_dict[join]
        for k, df in df_dict.items():
            if k != join:
                all_df = all_df.join(df)
        return all_df

View on GitHub (pinned to 79633dd950)

Solutions

1. Materialize a real DataFrame first: df = sdf._df_dict[sdf.join] (or select the group you need) and then call droplevel on it.
2. Subclass SepDataFrame and implement droplevel to apply it to every frame in _df_dict and update the join key.
3. Restructure your code to avoid droplevel on this container (select with .loc(axis=1)[cols] instead).

Example fix

// before
flat = sdf.droplevel(0)  # NotImplementedError

// after
flat = sdf._df_dict[sdf.join].droplevel(0)

if hasattr(df, "_df_dict"):  # SepDataFrame
    df = df._df_dict[df.join]  # materialize real pandas frame
df.droplevel(0)

from qlib.contrib.data.utils import SepDataFrame

def is_sep_dataframe(x) -> bool:
    return hasattr(x, "_df_dict") and hasattr(x, "join")

try:
    out = df.droplevel(0)
except NotImplementedError:
    # SepDataFrame stub: fall back to materializing the underlying frame
    out = df._df_dict[df.join].droplevel(0)

When it happens

Trigger: Calling .droplevel(...) on an object returned by the high-frequency loader (which wraps data in SepDataFrame), e.g. sdf.droplevel(0) or df.droplevel('instrument') where df is a SepDataFrame patched to pass isinstance checks as pd.DataFrame.

Common situations: Generic pandas code that flattens MultiIndex columns/rows after loading data; using a SepDataFrame where a real DataFrame is expected because the module patches builtins.isinstance to make it pass isinstance(x, pd.DataFrame).

Related errors

• This type of input is not supported
• nfs-common is not found, please install it by execute: sudo
• Mount failed: requires sudo or permission denied
• mount {provider_uri} on {mount_path} error! Command error
• Mount failed: {e.stderr}

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/4126537025353043. Report an issue: GitHub.