{"record":{"id":"e88b7b6a90524c23","repo":"huggingface/open-r1","slug":"invalid-submission-language-submission-language","errorCode":null,"errorMessage":"Invalid submission language: {submission_language}","messagePattern":"Invalid submission language: (.+?)","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/open_r1/utils/competitive_programming/cf_scoring.py","lineNumber":21,"sourceCode":"from io import BytesIO\nfrom typing import Literal\n\nfrom async_lru import alru_cache\n\nfrom .piston_client import PistonClient\nfrom .utils import batched\n\n\nasync def score_single_test_case(\n    client: PistonClient,\n    problem_data: dict,\n    test_input: str,\n    test_output: str,\n    submission: str,\n    submission_language: str = \"cpp\",\n) -> tuple[str, str]:\n    if submission_language not in [\"python\", \"cpp\"]:\n        raise ValueError(f\"Invalid submission language: {submission_language}\")\n    try:\n        result = await client.send_execute(\n            {\n                \"files\": [\n                    {\"name\": f\"main.{submission_language}\", \"content\": submission},\n                    *(\n                        [{\"name\": \"checker.py\", \"content\": problem_data[\"generated_checker\"]}]\n                        if problem_data[\"generated_checker\"]\n                        else []\n                    ),\n                    {\"name\": \"input.txt\", \"content\": test_input},\n                    {\"name\": \"correct_output.txt\", \"content\": test_output},\n                    {\n                        \"name\": \"grader_config\",\n                        \"content\": \"\\n\".join(\n                            f\"{key}={value}\"\n                            for key, value in {\n                                \"TIME_LIMIT\": problem_data[\"time_limit\"],","sourceCodeStart":3,"sourceCodeEnd":39,"githubUrl":"https://github.com/huggingface/open-r1/blob/1416fa0cf21595d2083b399a2a0bbddd7f6e9563/src/open_r1/utils/competitive_programming/cf_scoring.py#L3-L39","documentation":"score_single_test_case only supports \"python\" and \"cpp\" submissions; any other submission_language raises ValueError before attempting remote execution. The language also determines the file extension (main.{language}) sent to the execution backend, so unsupported values cannot proceed.","triggerScenarios":"Calling score_single_test_case (directly or via score_submission) with submission_language set to anything outside [\"python\", \"cpp\"] — e.g. \"py\", \"python3\", \"java\", \"rust\", or a None passed through from config.","commonSituations":"Configuring a dataset with a language column containing \"python3\" or \"py3\"; attempting to score other Codeforces languages (Java, Rust); a pipeline default of \"cpp\" being overridden with a raw language name from dataset metadata.","solutions":["Set submission_language to \"python\" or \"cpp\" (exact strings)","Normalize dataset language labels before scoring (map \"python3\"/\"py\" -> \"python\")","Filter or skip samples whose language isn't supported","Extend the allowed list in cf_scoring.py if the execution backend actually supports more languages"],"exampleFix":"// before\nscore = await score_single_test_case(case, inp, out, sub, \"python3\")  # ValueError\n// after\nlang = \"python\" if submission_language.startswith(\"py\") else \"cpp\"\nscore = await score_single_test_case(case, inp, out, sub, lang)","handlingStrategy":"validation","validationCode":"ALLOWED_LANGS = {\"python\", \"cpp\"}\nLANG_ALIASES = {\"py\": \"python\", \"python3\": \"python\", \"c++\": \"cpp\", \"cxx\": \"cpp\"}\n\ndef normalize_lang(lang: str) -> str:\n    lang = LANG_ALIASES.get(lang.lower(), lang.lower())\n    if lang not in ALLOWED_LANGS:\n        raise SystemExit(f\"Unsupported submission language {lang!r}; use 'python' or 'cpp'\")\n    return lang","typeGuard":"def is_supported_lang(lang) -> bool:\n    return isinstance(lang, str) and lang in (\"python\", \"cpp\")","tryCatchPattern":"try:\n    result = await score_single_test_case(client, case, inp, out, sub, lang)\nexcept ValueError as e:\n    if str(e).startswith(\"Invalid submission language\"):\n        logger.error(\"%s — normalize language to 'python' or 'cpp'\", e)\n        return None\n    raise","preventionTips":["Normalize dataset language columns with an alias map before scoring","Pre-filter samples whose language is not python/cpp","Keep validation in both score_submission and score_single_test_case in sync if extending","Assert languages at dataset-load time rather than per test case"],"tags":["python","validation","invalid-argument","async"],"backgroundTag":"invalid-enum-value","analyzedSha":"1416fa0cf21595d2083b399a2a0bbddd7f6e9563","analyzedAt":"2026-08-30T08:56:53.400Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}