openai/openai-python · error · ValueError

Expected a non-empty value for `eval_id` but received {eval_

Error message

Expected a non-empty value for `eval_id` but received {eval_id!r}

What it means

The sync EvalRunOutputItem.retrieve() method validates that eval_id is truthy before building the request path /evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}. Because eval_id is interpolated into the URL, an empty string or None would produce a malformed path (e.g. /evals//runs/...), so the SDK raises ValueError immediately instead of sending a doomed HTTP request.

Source

Thrown at src/openai/resources/evals/runs/output_items.py:72

        extra_headers: Headers | None = None,
        extra_query: Query | None = None,
        extra_body: Body | None = None,
        timeout: float | httpx2.Timeout | None | NotGiven = not_given,
    ) -> OutputItemRetrieveResponse:
        """
        Get an evaluation run output item by ID.

        Args:
          extra_headers: Send extra headers

          extra_query: Add additional query parameters to the request

          extra_body: Add additional JSON properties to the request

          timeout: Override the client-level default timeout for this request, in seconds
        """
        if not eval_id:
            raise ValueError(f"Expected a non-empty value for `eval_id` but received {eval_id!r}")
        if not run_id:
            raise ValueError(f"Expected a non-empty value for `run_id` but received {run_id!r}")
        if not output_item_id:
            raise ValueError(f"Expected a non-empty value for `output_item_id` but received {output_item_id!r}")
        return self._get(
            path_template(
                "/evals/{eval_id}/runs/{run_id}/output_items/{output_item_id}",
                eval_id=eval_id,
                run_id=run_id,
                output_item_id=output_item_id,
            ),
            options=make_request_options(
                extra_headers=extra_headers,
                extra_query=extra_query,
                extra_body=extra_body,
                timeout=timeout,
                security={"bearer_auth": True},
            ),

View on GitHub (pinned to 9917c6e28e)

Solutions

  1. Check where eval_id comes from and ensure it is a non-empty string before the call (e.g. assert eval_id, log its value).
  2. If it comes from an env var, verify the variable name and that it is set in the environment you are running.
  3. If it comes from a previous API response (e.g. client.evals.create(...).id), inspect that response — the eval may not have been created.
  4. Add a small guard: if not eval_id: raise ValueError('eval_id missing') with context about the caller.

Example fix

// before
item = client.evals.runs.output_items.retrieve(eval_id=eval_id, run_id=run_id, output_item_id=output_item_id)

// after
if not eval_id:
    raise ValueError(f"eval_id is required, got {eval_id!r}")
item = client.evals.runs.output_items.retrieve(eval_id=eval_id, run_id=run_id, output_item_id=output_item_id)
Defensive patterns

Strategy: validation

Validate before calling

if not isinstance(eval_id, str) or not eval_id.strip():
    raise ValueError(f"eval_id must be a non-empty string, got {eval_id!r}")

Type guard

def is_valid_eval_id(eval_id: object) -> bool:
    return isinstance(eval_id, str) and bool(eval_id.strip())

Try / catch

try:
    item = client.evals.runs.output_items.retrieve(eval_id=eval_id, run_id=run_id, output_item_id=output_item_id)
except ValueError as e:
    if 'eval_id' in str(e):
        raise ValueError(f'Skipping retrieve: {e}') from e
    raise

Prevention

When it happens

Trigger: Calling client.evals.runs.output_items.retrieve(eval_id='', run_id='run_123', output_item_id='item_abc') (or passing None / a whitespace-only id) on the synchronous client. Any code that builds eval_id dynamically (from a variable, config, or previous response) and passes it without checking it can trigger this.

Common situations: Loading eval ids from environment variables or config files that are unset, copying example code without replacing placeholder ids, iterating over a list where some entries are empty, or destructuring a prior response object whose field was missing and defaulted to ''.

Related errors

AI-assisted analysis of openai/openai-python@9917c6e28e (2026-08-28). Data as JSON: /api/errors/816e7af591ebb380. Report an issue: GitHub.