jd-opensource/joyagent-jdgenie · error · ValueError

The dataframe is empty after dropna.

Error message

The dataframe {data} is empty after dropna.

What it means

A pydantic `field_validator` on the `data` DataFrame drops rows with NaN in the measure column, and raises this ValueError if nothing remains. It is raised during model construction, so instantiating/validating the data model fails whenever the measure column is entirely missing or null in the relevant subset.

Solutions

1. Check the measure column name in your config matches the DataFrame column exactly
2. Inspect the DataFrame before model construction: confirm the measure column has non-null values
3. Fix the upstream query/filter so it returns rows, or handle empty input before building the model
4. Drop or repair rows with NaN in the measure column at ingestion

Example fix

// before
df = fetch_data(); model = AnalysisDataModel(data=df, columns=cols)  # measure col all NaN
// after
df = df.dropna(subset=[measure_col])
if df.empty:
    raise ValueError(f"No data for measure column {measure_col}")
model = AnalysisDataModel(data=df, columns=cols)

if df.empty or df[measure_col].isna().all():
    raise ValueError(f"No usable data in measure column '{measure_col}'")
if measure_col not in df.columns:
    raise ValueError(f"Measure column '{measure_col}' missing from dataframe")

def has_measure_data(df, measure_col: str) -> bool:
    return measure_col in df.columns and df[measure_col].notna().any()

try:
    model = AnalysisDataModel(data=df, columns=cols)
except ValueError as e:
    if "empty after dropna" in str(e):
        fix_upstream_query_or_alert()
    raise

When it happens

Trigger: Constructing the analysis data model with a DataFrame where the configured `measure.column` has all-NaN values, or the column name is misspelled so dropna removes every row.

Common situations: Upstream query returned no/blank data for the measure; measure column name mismatch between config and dataframe; filtered rows removed all observations; schema drift after source change.

Understand the failure class

Background: EmptyResultError / "no results found": when an API or scraper succeeds but returns zero rows — this error's family across 9 libraries.

AI-assisted analysis of jd-opensource/joyagent-jdgenie@2417e0b8b6 (2026-09-08). Data as JSON: /api/errors/8767b8bd4affb44f. Report an issue: GitHub.

Appendix: source

Thrown at genie-tool/genie_tool/tool/analysis_component/data_model.py:78

class DataModel(BaseModel):
    id: str = Field(str(uuid.uuid4()), description="")
    measure: Measure = Field(None, description="度量")
    data: pd.DataFrame = Field(exclude=True)
    columns: List[Column] = Field(
        [], description="可分析的维度", validate_default=True)

    model_config = ConfigDict(arbitrary_types_allowed=True)

    def __len__(self):
        return len(self.data)

    @field_validator("data", mode="before")
    @classmethod
    def validate_data(cls, data: pd.DataFrame, values) -> pd.DataFrame:
        measure = values.data["measure"]
        data = data.dropna(subset=[measure.column])
        if len(data) == 0:
            raise ValueError(f"The dataframe {data} is empty after dropna.")
        for col in data.columns:
            if len(data[col].unique()) == 1:
                data = data.drop(col, axis=1)
        return data

    @field_validator("columns", mode="before")
    @classmethod
    def validate_columns(cls, val, values) -> List[Column]:
        data = values.data["data"]
        measure = values.data["measure"]
        if not val:
            val = [Column(name=c,
                          is_series=is_datetime64_any_dtype(data[c]),
                          is_number=is_numeric_dtype(data[c]),
                          ) for c in data.columns if c != measure.column]
        if val and isinstance(val, list) and isinstance(val[0], str):
            val = [c for c in val if c in data.columns] or data.columns
            val = [Column(name=c,

View on GitHub (pinned to 2417e0b8b6)