oobabooga/textgen · error · Exception

Unexpected response format: 'completion_probabilities' not f

Error message

Unexpected response format: 'completion_probabilities' not found in {result}

What it means

Error "Unexpected response format: 'completion_probabilities' not found in {result}" thrown in oobabooga/textgen.

Source

Thrown at modules/llama_cpp_server.py:321

            logger.info("GENERATE_PARAMS=")
            printable_payload = {k: v for k, v in payload.items() if k != "prompt"}
            pprint.PrettyPrinter(indent=4, sort_dicts=False).pprint(printable_payload)
            print()

        def _try_fetch_logits():
            for retry in range(5):
                response = self.session.post(url, json=payload)
                result = response.json()

                if "completion_probabilities" in result:
                    if use_samplers:
                        return result["completion_probabilities"][0]["top_probs"]
                    else:
                        return result["completion_probabilities"][0]["top_logprobs"]

                time.sleep(0.05)
            else:
                raise Exception(f"Unexpected response format: 'completion_probabilities' not found in {result}")

        result = _try_fetch_logits()
        for entry in result:
            if not entry.get('token'):
                entry['token'] = self.decode([entry['id']])
        return result

    def get_prompt_logprob_entries(self, token_ids, n_probs=5, prompt=""):
        """Get logprob entries for prompt tokens via a single n_predict=0 request.

        Requires llama.cpp server with prompt_logprobs support.
        Returns entries in the standard format for format_completion_logprobs().
        """
        token_ids_list = token_ids.tolist() if hasattr(token_ids, 'tolist') else list(token_ids)

        url = f"http://127.0.0.1:{self.port}/completion"
        payload = {
            "prompt": token_ids_list,

View on GitHub (pinned to ed888c71f2)

Solutions

1. Enable logprobs/completion probabilities on the llama.cpp server request so 'completion_probabilities' is returned.
2. Check that the server response is not an error payload before reading 'completion_probabilities'.

When it happens

Trigger: Raised when the llama.cpp server response to a logprobs/completion-probabilities request does not contain the 'completion_probabilities' field. Triggers when the loaded model or llama.cpp server build does not support probabilities for the request, or when the server returned an error-shaped response instead.

Common situations: See trigger scenarios.

AI-assisted analysis of oobabooga/textgen@ed888c71f2 (2026-08-15). Data as JSON: /api/errors/464305a7000ba569. Report an issue: GitHub.