apache/beam · error · RuntimeException

Model returned no structured responses

Error message

Model returned no structured responses

What it means

OpenAIModelHandler.request() parses the model's structured-output response into an OpenAIModelOutput schema object. If the parsed content is null or its responses field is null, there is nothing to map back to inputs, so it throws a RuntimeException with this message rather than returning an empty or null result.

Solutions

1. Ensure the request sets the structured-output response format matching OpenAIModelOutput's schema.
2. Log the raw response to see refusals, truncation (finish_reason), or content-filter outcomes.
3. Check finish_reason == 'length' and increase max output tokens if output was cut off.
4. Handle refusals explicitly: retry with adjusted prompt or emit a fallback OutputT.

Example fix

// before
ChatCompletionCreateParams params = client.chat().completionCreateParams().model(model).addMessage(msg);
// after
ChatCompletionCreateParams params = client.chat().completionCreateParams().model(model)
    .responseFormat(ChatCompletionCreateParams.ResponseFormat.JSON_SCHEMA)
    .addMessage(msg);

try { results = handler.request(batch); } catch (RuntimeException e) { if (e.getMessage().contains("no structured responses")) { /* inspect raw response / retry with adjusted prompt */ } else { throw e; } }

When it happens

Trigger: Calling request() when the model response contains no content/output_text parts (refusal, empty completion, content filter), or when the structured output JSON does not match the expected schema so deserialization yields null fields.

Common situations: Model refuses the prompt (safety refusal), max output tokens hit so text is truncated away, the response_format/structured-output schema was changed client-side but the prompt does not request it, or a model version returns a different content layout.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• 2xx codes should not be exceptions. Got status code
• A 'datagen' table requires either 'rows-per-second' (for…
• A function must be provided to convert the input type into…
• A list of URNs for overriding transforms was provided but…
• A method marked with SchemaCreate in class

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/838c7b3e0a0780d8. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/ml/inference/openai/src/main/java/org/apache/beam/sdk/ml/inference/openai/OpenAIModelHandler.java:120

      StructuredResponseCreateParams<StructuredInputOutput> clientParams =
          ResponseCreateParams.builder()
              .model(modelParameters.getModelName())
              .input(inputBatch)
              .text(StructuredInputOutput.class, JsonSchemaLocalValidation.NO)
              .instructions(modelParameters.getInstructionPrompt())
              .build();

      // Get structured output from the model
      StructuredInputOutput structuredOutput =
          client.responses().create(clientParams).output().stream()
              .flatMap(item -> item.message().stream())
              .flatMap(message -> message.content().stream())
              .flatMap(content -> content.outputText().stream())
              .findFirst()
              .orElse(null);

      if (structuredOutput == null || structuredOutput.responses == null) {
        throw new RuntimeException("Model returned no structured responses");
      }

      // return PredictionResults
      return structuredOutput.responses.stream()
          .map(
              response ->
                  PredictionResult.create(
                      OpenAIModelInput.create(response.input),
                      OpenAIModelResponse.create(response.output)))
          .collect(Collectors.toList());

    } catch (JsonProcessingException e) {
      throw new RuntimeException("Failed to serialize input batch", e);
    }
  }

  /**
   * Schema class for structured output response.

View on GitHub (pinned to 12126d8942)