BoundaryML/baml · error · anyhow::Error

(test constraints internal error)

Error message

{details} (test constraints internal error)

What it means

BAML test runner failed to evaluate the test's constraints (checks/assertions) due to an internal error, so the test is reported as a generic failure with the internal details attached. The library throws this when constraint evaluation itself crashed rather than produced pass/fail check results, and the test harness surfaces it via TestStatus::status() for expr-function tests.

Solutions

1. Read the {details} text in the message — it names the internal constraint evaluation error
2. Check the test's checks/assert clauses for references to fields that may be missing in the function output
3. Re-run with verbose logging to capture the underlying constraint evaluation panic
4. Update the BAML CLI/runtime to matching versions and retry

Example fix

// before
assert foo.bar == "x" {{
  "bar must be x"
}}
// after
assert foo != null && foo.bar == "x" {{
  "bar must be x (guarded for missing output fields)"
}}

// before running the test, sanity-check constraint inputs
const requiredFields = ['foo'];
for (const f of requiredFields) {
  if (output == null || !(f in output)) throw new Error(`constraint input missing field: ${f}`);
}

function hasChecksResult(r) {
  return r != null && typeof r === 'object' && Array.isArray(r.checks) && r.failed_assert !== undefined;
}

try {
  await runner.runTest(name);
} catch (e) {
  if (String(e).includes('test constraints internal error')) {
    console.error('Constraint evaluation failed:', e.message);
  }
  throw e;
}

When it happens

Trigger: Calling status() on a FunctionResult whose expr_function_response succeeded but whose constraints_result is TestConstraintsResult::InternalError{details}; typically raised during `baml test` runs when a check/assert expression panicked or hit an internal evaluation bug.

Common situations: An assert/check clause in a BAML test references a field absent from the parsed output, or a constraint expression throws an unexpected error during evaluation; also seen after BAML version changes where constraint IR shifted.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• BAML internal error (AWSBedrock): file should have been…
• BamlError: Unexpected error from BAML
• Class not found for instance
• ExposedError (formatted via format_last_error_with_details)
• Expr function error

AI-assisted analysis of BoundaryML/baml@bd85ce9dee (2026-09-12). Data as JSON: /api/errors/15b6e0aad43d6e23. Report an issue: GitHub.

Appendix: source

Thrown at engine/baml-runtime/src/types/response.rs:430

            }
            (Self::TestFinishReasonFailed(a), Self::TestFinishReasonFailed(b)) => {
                a.to_string() == b.to_string()
            }
            _ => false,
        }
    }
}

impl Eq for TestFailReason<'_> {}

impl TestResponse {
    pub fn status(&self) -> TestStatus<'_> {
        // Handle expr function response
        if let Some(expr_res) = &self.expr_function_response {
            return if expr_res.is_ok() {
                match self.constraints_result.clone() {
                    TestConstraintsResult::InternalError { details } => {
                        TestStatus::Fail(TestFailReason::TestUnspecified(anyhow::anyhow!(details)))
                    }
                    TestConstraintsResult::Completed {
                        checks,
                        failed_assert,
                    } => {
                        let n_failed_checks: usize =
                            checks.iter().filter(|(_, pass)| !pass).count();
                        if failed_assert.is_some() || n_failed_checks > 0 {
                            TestStatus::Fail(TestFailReason::TestConstraintsFailure {
                                checks,
                                failed_assert,
                            })
                        } else {
                            TestStatus::Pass
                        }
                    }
                }
            } else {

View on GitHub (pinned to bd85ce9dee)