{"record":{"id":"bd37e0de958c1585","repo":"huggingface/open-r1","slug":"cf-tests-folder-environment-variable-not-set-plea","errorCode":null,"errorMessage":"CF_TESTS_FOLDER environment variable not set! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path. See https://huggingface.co/datasets/open-r1/codeforces for more information.","messagePattern":"CF_TESTS_FOLDER environment variable not set! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path\\. See https://huggingface\\.co/datasets/open-r1/codeforces for more information\\.","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/open_r1/utils/competitive_programming/cf_scoring.py","lineNumber":67,"sourceCode":"            language=\"cf_python3\" if submission_language == \"python\" else \"c++17\",\n        )\n    except Exception as e:\n        print(f\"Error scoring submission: {e}\")\n        return False\n\n    return result\n\n\n@alru_cache(maxsize=32)  # TODO make this configurable\nasync def get_generated_contest_tests(contest_id: str) -> list[dict]:\n    import pandas as pd\n\n    import aiofiles\n    import aiofiles.os\n\n    tests_folder = os.environ.get(\"CF_TESTS_FOLDER\", None)\n    if not tests_folder:\n        raise ValueError(\n            \"CF_TESTS_FOLDER environment variable not set! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path. See https://huggingface.co/datasets/open-r1/codeforces for more information.\"\n        )\n    if not await aiofiles.os.path.exists(tests_folder):\n        raise ValueError(\n            f\"CF_TESTS_FOLDER path '{tests_folder}' does not exist! Please download the codeforces generated tests and set CF_TESTS_FOLDER to the folder path. See https://huggingface.co/datasets/open-r1/codeforces for more information.\"\n        )\n    parquet_path = os.path.join(tests_folder, f\"test_cases_{int(contest_id):04d}.parquet\")\n    if not await aiofiles.os.path.exists(parquet_path):\n        return {}\n\n    # Read parquet file asynchronously\n    async with aiofiles.open(parquet_path, \"rb\") as f:\n        content = await f.read()\n        df = pd.read_parquet(BytesIO(content))\n\n    # Group by problem_id and convert to dictionary of lists\n    grouped_tests = df.groupby(\"problem_id\").apply(lambda x: x[[\"input\", \"output\"]].to_dict(\"records\")).to_dict()\n","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/huggingface/open-r1/blob/1416fa0cf21595d2083b399a2a0bbddd7f6e9563/src/open_r1/utils/competitive_programming/cf_scoring.py#L49-L85","documentation":"get_generated_contest_tests requires the CF_TESTS_FOLDER environment variable to locate locally downloaded Codeforces generated test parquet files. If the variable is unset, it raises ValueError with a link to the open-r1/codeforces dataset explaining how to download the tests. Without the folder, generated (hidden) test cases cannot be added to official tests for scoring.","triggerScenarios":"Calling get_generated_tests / score_submission when CF_TESTS_FOLDER is not present in os.environ at scoring time.","commonSituations":"Running evaluation on a cluster/CI node where .env or export of CF_TESTS_FOLDER wasn't propagated; downloading the dataset to a laptop but running scoring elsewhere; new contributor following README partially; env var named incorrectly (CF_TEST_FOLDER).","solutions":["Download the codeforces generated tests from https://huggingface.co/datasets/open-r1/codeforces","Set CF_TESTS_FOLDER to the absolute path of the folder containing the test_cases_*.parquet files before launching","Persist it via .env, sbatch --export, or CI secret so worker nodes see it","Verify: CF_TESTS_FOLDER=/path python -c \"import os; print(os.environ['CF_TESTS_FOLDER'])\""],"exampleFix":"// before\nscore = await score_submission(problem, code)  # ValueError: CF_TESTS_FOLDER not set\n// after\n# shell: export CF_TESTS_FOLDER=/data/codeforces_tests\nscore = await score_submission(problem, code)","handlingStrategy":"validation","validationCode":"import os\n\ndef require_cf_tests_folder():\n    path = os.environ.get(\"CF_TESTS_FOLDER\")\n    if not path:\n        raise SystemExit(\n            \"Set CF_TESTS_FOLDER to the folder with codeforces test parquet files \"\n            \"(see https://huggingface.co/datasets/open-r1/codeforces)\"\n        )\n    if not os.path.isdir(path):\n        raise SystemExit(f\"CF_TESTS_FOLDER {path!r} does not exist\")","typeGuard":"def cf_tests_env_ready() -> bool:\n    import os\n    return bool(os.environ.get(\"CF_TESTS_FOLDER\"))","tryCatchPattern":"try:\n    tests = await get_generated_tests(problem_id)\nexcept ValueError as e:\n    if \"CF_TESTS_FOLDER\" in str(e):\n        logger.error(\"%s — export CF_TESTS_FOLDER before running\", e)\n        raise SystemExit(1) from e\n    raise","preventionTips":["Download tests once to a shared volume and export CF_TESTS_FOLDER in job templates/SLURM scripts","Add the variable to .env.example and CI secret setup docs","Preflight-check env vars at job startup for all required paths","Use absolute paths so relative-path cwd differences can't bite"],"tags":["python","missing-env-var","configuration","datasets"],"backgroundTag":"missing-env-var","analyzedSha":"1416fa0cf21595d2083b399a2a0bbddd7f6e9563","analyzedAt":"2026-08-30T08:56:53.400Z","schemaVersion":2},"datasetVersion":"2026-08-30T13:17:10.514Z"}