huggingface/open-r1 · error

Could not determine problem ID from files

Error message

Could not determine problem ID from files

What it means

The MorphCloud morph_client derives the IOI problem ID by scanning the uploaded files (looking for filenames that identify the problem, e.g. a task folder or graders layout). If no file name yields a recognizable problem ID, _prepare_files raises ValueError before building the grader config.

Source

Thrown at src/open_r1/utils/competitive_programming/morph_client.py:109

            tuple: (problem_id, grader_config, local_files)

        Raises:
            ValueError: If problem ID cannot be determined
        """
        # Extract problem ID
        problem_id = None
        graders_files = []
        for file in data["files"]:
            if file["name"].startswith("graders/") and file["name"].endswith(".cpp"):
                potential_id = os.path.basename(file["name"]).split(".")[0]
                if potential_id not in ["grader", "manager", "stub"]:
                    problem_id = potential_id

            if file["name"].startswith("graders/"):
                graders_files.append(file)

        if not problem_id:
            raise ValueError("Could not determine problem ID from files")

        grader_config = {
            "task_type": "Batch",
            "code": problem_id,
            "time_limit": data["run_timeout"] / 1000,
            "memory_limit": data["run_memory_limit"] * 1024 * 1024,
        }

        for file in graders_files:
            if "manager.cpp" in file["name"]:
                grader_config["task_type"] = "Communication"
                grader_config["task_type_parameters_Communication_num_processes"] = 1
                grader_config["task_type_parameters_Communication_user_io"] = "std_io"
                break

        config_path = os.path.join(temp_dir, "grader_config.json")
        with open(config_path, "w") as f:
            json.dump(grader_config, f)

View on GitHub (pinned to 1416fa0cf2)

Solutions

1. Include the problem ID in the uploaded file paths, e.g. name files 'taskname/graders/...' or ensure the identifiable component is present in a filename.
2. Inspect _prepare_files to see which filename patterns set potential_id and align your file naming with them.
3. Pass the problem ID explicitly if your client version supports it, or patch _prepare_files to accept data['problem_id'] as an override.
4. Add a pre-flight assertion that problem_id extraction succeeds before submitting the batch.

Example fix

// before
files = [{"name": "main.cpp", "content": code}]
// after
files = [{"name": "tasks/collections/main.cpp", "content": code}, {"name": "tasks/collections/graders/grader.cpp", "content": grader}]

def assert_problem_id_present(files):
    import re
    pattern = re.compile(r'(?:tasks?/)([A-Za-z0-9_-]+)/')
    ok = any(pattern.search(f['name']) for f in files)
    assert ok, 'No file path encodes the problem ID; fix file naming before submission'

def files_have_problem_id(files: list) -> bool:
    return any(f.get('name', '').startswith(('tasks/', 'graders/')) or '/' in f.get('name', '') for f in files)

try:
    score, feedback = await client._execute_with_instance(instance, data)
except ValueError as e:
    if 'problem ID' in str(e):
        logger.error('Upload layout invalid: %s — expected task-prefixed paths', [f['name'] for f in data['files']])
    raise

When it happens

Trigger: Calling _execute_with_instance with a data['files'] list whose 'name' fields don't match the patterns _prepare_files checks (no problem-id-like name, no 'graders/' prefix structure) — e.g. files uploaded as 'sol.cpp' / 'main.cpp' without any problem-identifying path component.

Common situations: Adapting the client to a new problem package whose directory layout differs; flat uploads that stripped directory prefixes; renamed test/grader folders; building file lists programmatically with generic names.

Related errors

• Either `dataset_name` or `dataset_mixture` must be provided
• dataset_mixture must be a dictionary with a 'datasets' key.
• 'datasets' must be a list of dataset configurations
• Column names must be consistent across all dataset configura
• Callback {callback_name} not found in CALLBACKS.

AI-assisted analysis of huggingface/open-r1@1416fa0cf2 (2026-08-30). Data as JSON: /api/errors/f52872f3ddb12fc7. Report an issue: GitHub.