{"record":{"id":"febc72b0b8b3fc84","repo":"aaif-goose/goose","slug":"no-valid-csv-files-found-with-required-columns","errorCode":null,"errorMessage":"No valid CSV files found with required columns","messagePattern":"No valid CSV files found with required columns","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/bench-postprocess-scripts/generate_leaderboard.py","lineNumber":87,"sourceCode":"                \n                # For missing columns, add them with NaN values\n                for col in missing_columns:\n                    df[col] = float('nan')\n            \n            # Select only the columns we care about\n            df_subset = df[selected_columns].copy()  # Create a copy to avoid SettingWithCopyWarning\n            \n            # Add model folder name as additional context\n            model_folder = csv_file.parent.parent.name\n            df_subset['model_folder'] = model_folder\n            \n            all_data.append(df_subset)\n            \n        except Exception as e:\n            print(f\"Error processing {csv_file}: {str(e)}\")\n    \n    if not all_data:\n        raise ValueError(\"No valid CSV files found with required columns\")\n    \n    # Concatenate all dataframes to create a union\n    union_df = pd.concat(all_data, ignore_index=True)\n    \n    # Create leaderboard by grouping and averaging numerical columns\n    numeric_columns = [\n        'total_tool_calls_mean', \n        'prompt_execution_time_mean', \n        'total_tokens_mean', \n        'score_mean', \n        'prompt_error_mean',\n        'server_error_mean'\n    ]\n    \n    # Group by provider and model_name, then calculate averages for numeric columns\n    leaderboard_df = union_df.groupby(['provider', 'model_name'])[numeric_columns].mean().reset_index()\n    \n    # Sort by score_mean in descending order (highest scores first)","sourceCodeStart":69,"sourceCodeEnd":105,"githubUrl":"https://github.com/aaif-goose/goose/blob/3810898a7447ec3299be72e223d3570a7aabf0ab/scripts/bench-postprocess-scripts/generate_leaderboard.py#L69-L105","documentation":"Raised by the benchmark leaderboard generator after it walks candidate result CSVs, keeps only those whose columns include the required *_mean metrics (total_tool_calls_mean, prompt_execution_time_mean, total_tokens_mean, score_mean, ...), and finds the accumulator still empty. Individual files that are unreadable or lack columns are caught per-file and only printed as 'Error processing ...' lines; this error means every candidate failed, so no leaderboard can be built.","triggerScenarios":"Running the script in a directory that contains no postprocessed benchmark CSVs at all; CSVs produced by an older/newer postprocessing step whose schema lacks the *_mean columns; pointing the file glob at raw eval output instead of aggregated metrics; every file failing with its own printed error (permissions, malformed CSV).","commonSituations":"Skipping the postprocessing step that computes mean columns; schema drift after a bench-harness upgrade renaming columns; running from the wrong directory so the glob matches nothing; hand-exported CSVs from spreadsheets with renamed headers.","solutions":["Re-run the bench postprocessing step first so the CSVs contain the *_mean columns, then regenerate the leaderboard","Check the printed 'Error processing <file>: ...' lines immediately above the traceback — they name the per-file reason (usually missing columns)","Run the script from the directory/glob it expects (per-model run folders with a metrics CSV two levels down)","If column names drifted, align REQUIRED_COLUMNS in the script with the current postprocessing schema"],"exampleFix":"# before\nif not all_data:\n    raise ValueError('No valid CSV files found with required columns')\n\n# after\nproblems = [f'{f}: missing {sorted(required_columns - set(df.columns))}' for f in problems]\nif not csv_files:\n    raise ValueError(f'No candidate CSVs found under {root}')\nif not all_data:\n    raise ValueError('No valid CSV files found. Per-file issues:\\n' + '\\n'.join(problems))","handlingStrategy":"validation","validationCode":"import pandas as pd\n\nREQUIRED = {\n    'total_tool_calls_mean',\n    'prompt_execution_time_mean',\n    'total_tokens_mean',\n    'score_mean',\n}\nusable = [f for f in csv_files if REQUIRED.issubset(pd.read_csv(f).columns)]\nassert usable, 'no benchmark CSVs contain the required *_mean columns; run postprocessing first'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always run the postprocessing step before leaderboard generation","Print each CSV's columns when schema drift is suspected","Run the script from the results root the glob expects"],"tags":["pandas","csv","leaderboard","benchmark","postprocessing"],"backgroundTag":null,"analyzedSha":"3810898a7447ec3299be72e223d3570a7aabf0ab","analyzedAt":"2026-08-16T10:14:26.282Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}