conductor-oss/conductor · error · RuntimeException

Output does not Confirm to the schema. errors: %s

Error message

Output does not Confirm to the schema.  errors: %s

What it means

Thrown by LLMHelper.tryToConvertToJSON() when the LLM response text was successfully parsed as JSON but failed JSON schema validation against the configured output schema. The message includes the specific validation error(s). Note: the message says 'confirm' but means 'conform' — a typo in the source.

Source

Thrown at ai/src/main/java/org/conductoross/conductor/ai/LLMHelper.java:300

            }
            default -> llmResponse.setResult(result.toString());
        }
    }

    @SneakyThrows
    private Object tryToConvertToJSON(String responseText, ChatCompletion chatCompletion) {
        try {
            responseText = responseText.trim();
            if (responseText.startsWith("```json")) {
                responseText = responseText.substring("```json".length());
                responseText =
                        responseText.substring(0, responseText.length() - "```".length() - 1);
            }
            Map<String, Object> map = objectMapper.readValue(responseText, MAP_OF_STRING_TO_OBJ);
            if (chatCompletion.getOutputSchema() != null) {
                String error = validateJsonSchema(chatCompletion.getInputSchema(), map);
                if (error != null) {
                    throw new RuntimeException(
                            String.format(
                                    "Output does not confirm to the schema.  errors: %s", error));
                }
            }
            // llmResponse.setResult(map);
            return map;

        } catch (JsonProcessingException e) {
            if (chatCompletion.isJsonOutput()) {
                log.error(
                        "error converting to json, response: {}, error: {}",
                        responseText,
                        e.getMessage(),
                        e);
                Map<String, Object> outputErrors =
                        Map.of("error", e.getMessage(), "response", responseText);
                throw new RuntimeException(objectMapper.writeValueAsString(outputErrors));
            }

View on GitHub (pinned to cf7c3e4a8a)

Solutions

1. Review the validation error details in the exception message to identify which fields failed
2. Improve the prompt to clearly specify the expected JSON structure, possibly include an example
3. Relax the schema if appropriate (e.g. allow additional properties, make fields optional)
4. Lower the model temperature or add explicit formatting instructions in the system prompt

Example fix

// before: schema requires {"name": string, "age": int}
// LLM returns: {"name": "Alice"}  // missing required 'age'
// after: improve prompt to say 'You must include both name and age fields'

// Before running the LLM, validate that the schema is well-formed
if (chatCompletion.getOutputSchema() != null) {
    String schemaJson = objectMapper.writeValueAsString(chatCompletion.getOutputSchema().getData());
    // Optionally pre-validate with an external JSON Schema validator
    // to catch schema issues before the LLM call
}

try {
    LLMResponse response = llmHelper.chatComplete(task, llm, chatCompletion, location, logger);
} catch (RuntimeException e) {
    if (e.getMessage().contains("Output does not confirm to the schema")) {
        // Schema validation failed — the LLM output didn't match
        // Consider retrying with a more specific prompt or relaxing the schema
        log.warn("Schema validation failed for LLM output: {}", e.getMessage());
        taskResult.setStatus(TaskResult.Status.FAILED);
    } else {
        throw e;
    }
}

When it happens

Trigger: The LLM returns text that is valid JSON (a Map) but does not match the outputSchema defined on the ChatCompletion. The schema validation runs via validateJsonSchema() using the networknt JSON schema validator, and validation messages are collected and formatted into the error string.

Common situations: The LLM omitted required fields or added extra fields not allowed by additionalProperties:false. The LLM returned a different structure than expected (e.g. an array where an object was expected). The schema is too strict or the prompt doesn't adequately describe the expected output format. Model temperature is too high causing inconsistent output.

Related errors

• Bad/Unsupported schema? : {validationMessages}
• Error parsing the json schema : {message}
• {location}: uses unsupported JSON Schema keyword '{key}' at
• {location}: unknown JSON Schema keyword '{key}' at {path}. A
• llmProvider not specified: {name}

AI-assisted analysis of conductor-oss/conductor@cf7c3e4a8a (2026-08-14). Data as JSON: /api/errors/1e43ea0cbd42c957. Report an issue: GitHub.