For IOI/CF problems Piston endpoints running our IOI package

Error message

For IOI/CF problems Piston endpoints running our IOI package are required. Please add a list of valid Piston endpoints to a PISTON_ENDPOINTS variable in a `.env` file.

What it means

get_piston_client_from_env requires the PISTON_ENDPOINTS environment variable listing Piston endpoints that run the custom IOI package. If the variable is unset, it raises ValueError with instructions, since IOI/CF scoring cannot work against plain Piston instances.

Source

Thrown at src/open_r1/utils/competitive_programming/piston_client.py:20

import os
import random
import re
import subprocess
from collections import Counter
from functools import lru_cache

import aiohttp


class PistonError(Exception):
    pass


@lru_cache(maxsize=1)
def get_piston_client_from_env(session=None):
    piston_endpoints = os.getenv("PISTON_ENDPOINTS")
    if piston_endpoints is None:
        raise ValueError(
            "For IOI/CF problems Piston endpoints running our IOI package are required. Please add a list of valid Piston endpoints to a PISTON_ENDPOINTS variable in a `.env` file."
        )
    piston_endpoints = sorted(
        piston_endpoints.split(",") if piston_endpoints != "slurm" else get_slurm_piston_endpoints()
    )
    gpu_nb = int(os.getenv("LOCAL_RANK", 0))  # per‑GPU index
    world = int(os.getenv("WORLD_SIZE", 1))  # total GPUs
    if world > 1:
        print(f"Using a subset of piston endpoints for GPU#{gpu_nb}")
        piston_endpoints = piston_endpoints[gpu_nb::world]
    random.shuffle(piston_endpoints)
    max_requests_per_endpoint = os.getenv("PISTON_MAX_REQUESTS_PER_ENDPOINT", "1")
    return PistonClient(piston_endpoints, session, max_requests_per_endpoint=int(max_requests_per_endpoint))


class PistonClient:
    """
    A client that will automatically load balance across multiple Piston (https://github.com/engineer-man/piston) workers.

View on GitHub (pinned to 1416fa0cf2)

Solutions

Set PISTON_ENDPOINTS=http://host1:2000,http://host2:2000 in your .env (comma-separated list of endpoints running the IOI package).
Set PISTON_ENDPOINTS=slurm if your Piston workers are launched via Slurm and get_slurm_piston_endpoints() applies.
Start a Piston worker with the custom IOI package and point the variable at it.
Confirm the variable is visible in the launch environment (printenv PISTON_ENDPOINTS) before running evaluation.

Example fix

// before
# .env (no piston config)
// after
PISTON_ENDPOINTS=http://127.0.0.1:2000,http://127.0.0.2:2000

Defensive patterns

Strategy: validation

Validate before calling

import os
from dotenv import load_dotenv
load_dotenv()
assert os.getenv('PISTON_ENDPOINTS'), 'PISTON_ENDPOINTS missing: list IOI-package Piston endpoints in .env'

Type guard

def piston_endpoints_configured() -> bool:
    val = os.getenv('PISTON_ENDPOINTS')
    return bool(val and (val == 'slurm' or any(e.strip() for e in val.split(','))))

Try / catch

try:
    client = get_piston_client_from_env()
except ValueError as e:
    if 'PISTON_ENDPOINTS' in str(e):
        raise SystemExit('Configure PISTON_ENDPOINTS (running the IOI package) in .env') from e
    raise

Prevention

Include PISTON_ENDPOINTS in .env.example and setup docs
Verify endpoints serve the cms_ioi runtime, not just any Piston
Check env vars on every node that runs evaluation (multi-node setups)
Use PISTON_ENDPOINTS=slurm when workers are Slurm-managed

When it happens

Trigger: ioi_code_reward or cf_code_reward calls get_piston_client_from_env on a machine where PISTON_ENDPOINTS was never set (no .env, or .env lacks the key).

Common situations: Running evaluation for the first time without completing the Piston setup; .env present but PISTON_ENDPOINTS omitted; running on a different node/worker where env vars weren't replicated.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of huggingface/open-r1@1416fa0cf2 (2026-08-30). Data as JSON: /api/errors/d363b441d8ab0d14. Report an issue: GitHub.