aaif-goose/goose · error · ValueError

No valid CSV files found with required columns

Error message

No valid CSV files found with required columns

What it means

Raised by the benchmark leaderboard generator after it walks candidate result CSVs, keeps only those whose columns include the required *_mean metrics (total_tool_calls_mean, prompt_execution_time_mean, total_tokens_mean, score_mean, ...), and finds the accumulator still empty. Individual files that are unreadable or lack columns are caught per-file and only printed as 'Error processing ...' lines; this error means every candidate failed, so no leaderboard can be built.

Source

Thrown at scripts/bench-postprocess-scripts/generate_leaderboard.py:87

                
                # For missing columns, add them with NaN values
                for col in missing_columns:
                    df[col] = float('nan')
            
            # Select only the columns we care about
            df_subset = df[selected_columns].copy()  # Create a copy to avoid SettingWithCopyWarning
            
            # Add model folder name as additional context
            model_folder = csv_file.parent.parent.name
            df_subset['model_folder'] = model_folder
            
            all_data.append(df_subset)
            
        except Exception as e:
            print(f"Error processing {csv_file}: {str(e)}")
    
    if not all_data:
        raise ValueError("No valid CSV files found with required columns")
    
    # Concatenate all dataframes to create a union
    union_df = pd.concat(all_data, ignore_index=True)
    
    # Create leaderboard by grouping and averaging numerical columns
    numeric_columns = [
        'total_tool_calls_mean', 
        'prompt_execution_time_mean', 
        'total_tokens_mean', 
        'score_mean', 
        'prompt_error_mean',
        'server_error_mean'
    ]
    
    # Group by provider and model_name, then calculate averages for numeric columns
    leaderboard_df = union_df.groupby(['provider', 'model_name'])[numeric_columns].mean().reset_index()
    
    # Sort by score_mean in descending order (highest scores first)

View on GitHub (pinned to 3810898a74)

Solutions

1. Re-run the bench postprocessing step first so the CSVs contain the *_mean columns, then regenerate the leaderboard
2. Check the printed 'Error processing <file>: ...' lines immediately above the traceback — they name the per-file reason (usually missing columns)
3. Run the script from the directory/glob it expects (per-model run folders with a metrics CSV two levels down)
4. If column names drifted, align REQUIRED_COLUMNS in the script with the current postprocessing schema

Example fix

# before
if not all_data:
    raise ValueError('No valid CSV files found with required columns')

# after
problems = [f'{f}: missing {sorted(required_columns - set(df.columns))}' for f in problems]
if not csv_files:
    raise ValueError(f'No candidate CSVs found under {root}')
if not all_data:
    raise ValueError('No valid CSV files found. Per-file issues:\n' + '\n'.join(problems))

import pandas as pd

REQUIRED = {
    'total_tool_calls_mean',
    'prompt_execution_time_mean',
    'total_tokens_mean',
    'score_mean',
}
usable = [f for f in csv_files if REQUIRED.issubset(pd.read_csv(f).columns)]
assert usable, 'no benchmark CSVs contain the required *_mean columns; run postprocessing first'

When it happens

Trigger: Running the script in a directory that contains no postprocessed benchmark CSVs at all; CSVs produced by an older/newer postprocessing step whose schema lacks the *_mean columns; pointing the file glob at raw eval output instead of aggregated metrics; every file failing with its own printed error (permissions, malformed CSV).

Common situations: Skipping the postprocessing step that computes mean columns; schema drift after a bench-harness upgrade renaming columns; running from the wrong directory so the glob matches nothing; hand-exported CSVs from spreadsheets with renamed headers.

Related errors

• Job name must start with a letter or number and contain only
• --model must be in provider/model form, e.g. anthropic/claud
• --trials must be at least 1
• --concurrency must be at least 1
• --timeout-multiplier must be positive

AI-assisted analysis of aaif-goose/goose@3810898a74 (2026-08-16). Data as JSON: /api/errors/febc72b0b8b3fc84. Report an issue: GitHub.