What causes "Parse response error: {0}" in BoundaryML/baml?

Strengthen the prompt/output-format instructions so the model returns parseable output

BoundaryML/baml · error · LlmOpError

Parse response error: {0}

Error message

Parse response error: {0}

What it means

Generic wrapper in LlmOpError for failures while parsing an LLM provider's response body into the expected structure (the {0} inner message carries the specifics from the underlying parser). It fires in the SAP parse entry points when response text cannot be converted to the declared output type — malformed JSON, unexpected shape, or converter failure.

Source

Thrown at baml_language/crates/sys_ops/src/sap.rs:58

        >,
        &DefKey,
    > {
        self.db().resolve_with_meta(self.types.stream_ty().as_ref())
    }
}

/// Errors that can occur during LLM operations. Relocated verbatim from
/// `sys_llm`; only `ParseResponseError`, `JsonishError` and `SapError` are
/// still constructible now that the SAP parse entry points are the sole users.
#[derive(Debug, thiserror::Error)]
pub enum LlmOpError {
    #[error("Expected {expected}, got {actual}")]
    TypeError {
        expected: &'static str,
        actual: String,
    },

    #[error("Parse response error: {0}")]
    ParseResponseError(String),

    #[error("Jsonish error: {0}")]
    JsonishError(::bex_sap::jsonish::JsonishError),

    #[error("SAP error: {0}")]
    SapError(::bex_sap::deserializer::coercer::ParsingError),
}

impl From<LlmOpError> for ::sys_types::VmRustFnError {
    fn from(e: LlmOpError) -> Self {
        let baml: ::sys_types::VmBamlError = match e {
            LlmOpError::TypeError { expected, actual } => {
                ::sys_types::VmBamlError::InvalidArgument {
                    message: format!("expected {expected}, got {actual}"),
                }
            }
            LlmOpError::ParseResponseError(e) => ::sys_types::VmBamlError::LlmClient { message: e },

View on GitHub (pinned to bd85ce9dee)

Solutions

Strengthen the prompt/output-format instructions so the model returns parseable output
Enable or rely on jsonish repair/coercion for slightly malformed responses
Check the raw response for truncation and increase max tokens if needed

Example fix

// before
raw_response.parse()
// after
parse_with_repair(raw_response).map_err(|e| ParseResponseError(e.to_string()))

Defensive patterns

Strategy: retry

Validate before calling

if !raw.trim_start().starts_with('{') && !raw.trim_start().starts_with('[') {
    // not parseable output; retry or repair before calling the parser
}

Try / catch

match result {
    Err(LlmOpError::ParseResponseError(msg)) => { log::warn!("parse failed: {msg}"); retry_with_stricter_prompt() }
    other => other,
}

Prevention

Use structured-output/JSON mode when the provider supports it
Check for truncation and raise max tokens
Retry once on parse failure — LLM formatting faults are often transient

When it happens

Trigger: The SAP response-parsing entry point fails to interpret the raw LLM output (non-JSON text, truncated output, unexpected structure) and returns ParseResponseError with a descriptive message.

Common situations: Model returns prose instead of JSON; response cut off by token limits; prompt asks for a format the parser doesn't support.

Understand the failure class

Background: "Invalid JSON response" and "Failed to parse response" errors: when an API answers 200 but the body isn't the JSON your library expected — this error's family across 28 libraries.

Related errors

AI-assisted analysis of BoundaryML/baml@bd85ce9dee (2026-09-12). Data as JSON: /api/errors/acbea0f8d4438005. Report an issue: GitHub.