BoundaryML/baml · warning

failed to get cached input tokens

Error message

failed to get cached input tokens: %w

What it means

CachedInputTokens() invokes 'cached_input_tokens' on the raw usage object; on failure it wraps the error with 'failed to get cached input tokens: %w'. Cached-token stats may not exist for all providers or call states.

Solutions

1. Confirm the provider/model reports cached input tokens before reading
2. Treat this error as 'not available' and fall back gracefully (e.g. skip caching metrics)
3. Read usage only after call/stream completion
4. Check the wrapped root error for the actual cause

Example fix

// before
cached, err := usage.CachedInputTokens()
if err != nil { return err }
// after
cached, err := usage.CachedInputTokens()
if err != nil {
    cached = 0 // provider may not report cache stats
}

// no cheap pre-check; cached stats availability depends on provider/model

cached, err := usage.CachedInputTokens()
if err != nil {
    cached = 0 // treat as unavailable
}

When it happens

Trigger: Requesting cached input tokens from a provider/model that doesn't report cache stats, or from an incomplete/failed call's usage object.

Common situations: Reading cache metrics for providers without prompt caching; non-OpenAI models where cached_input_tokens is unsupported; premature usage reads during streaming.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

• failed to get input tokens
• failed to get output tokens
• `baml update` is ambiguous. To use the latest version of…
• decodeCheckedValue: valueChecked is nil
• decodeLiteralValue: valueLiteral is nil

AI-assisted analysis of BoundaryML/baml@bd85ce9dee (2026-09-12). Data as JSON: /api/errors/ae0aeb83eb3fc5a6. Report an issue: GitHub.

Appendix: source

Thrown at engine/language_client_go/pkg/rawobjects_usage.go:58

func (u *usage) OutputTokens() (int64, error) {
	result, err := raw_objects.CallMethod(u, "output_tokens", nil)
	if err != nil {
		return 0, fmt.Errorf("failed to get output tokens: %w", err)
	}

	tokens, ok := result.(int64)
	if !ok {
		return 0, fmt.Errorf("unexpected type for output tokens: %T", result)
	}

	return tokens, nil
}

func (u *usage) CachedInputTokens() (int64, error) {
	result, err := raw_objects.CallMethod(u, "cached_input_tokens", nil)
	if err != nil {
		return 0, fmt.Errorf("failed to get cached input tokens: %w", err)
	}

	tokens, ok := result.(int64)
	if !ok {
		return 0, fmt.Errorf("unexpected type for cached input tokens: %T", result)
	}

	return tokens, nil
}

View on GitHub (pinned to bd85ce9dee)