JuliusBrussee/caveman · error

unsupported role %q

Error message

unsupported role %q

What it means

A corpus message carried a role outside the fixed allow-list 'system', 'developer', 'user', 'assistant', 'tool'. The switch's default branch rejects anything else, including case variants and legacy role names, because downstream prefix/cache logic only reasons about these five roles.

Source

Thrown at cacheengine/cachebench/corpus.go:350

	if len(messages) == 0 || len(messages) > limits.MaxMessagesPerRequest {
		return CorpusRow{}, fmt.Errorf("message count %d outside 1..%d", len(messages), limits.MaxMessagesPerRequest)
	}
	for messageIndex, message := range messages {
		if err := validateCorpusMessage(message, limits); err != nil {
			return CorpusRow{}, fmt.Errorf("message %d: %w", messageIndex, err)
		}
	}
	return CorpusRow{
		RowIndex: index, SessionID: wire.SessionID, Model: wire.Model, Input: messages,
		OutputLength: wire.OutputLength, PreGap: wire.PreGap,
	}, nil
}

func validateCorpusMessage(message CorpusMessage, limits CorpusLimits) error {
	switch message.Role {
	case "system", "developer", "user", "assistant", "tool":
	default:
		return fmt.Errorf("unsupported role %q", message.Role)
	}
	if !validBoundedText(message.ToolCallID, 2048, true) || !validBoundedText(message.Name, 512, true) {
		return errors.New("invalid message identity")
	}
	if len(message.Content) == 0 && len(message.ToolCalls) == 0 {
		return errors.New("content or tool_calls required")
	}
	if len(message.Content) > limits.MaxMessageBytes || (len(message.Content) > 0 && !json.Valid(message.Content)) {
		return errors.New("content invalid or exceeds limit")
	}
	messageBytes := len(message.Content) + len(message.Role) + len(message.ToolCallID) + len(message.Name)
	if len(message.Content) > 0 {
		var content any
		if err := json.Unmarshal(message.Content, &content); err != nil {
			return errors.New("content must decode")
		}
		if content != nil {
			if _, ok := content.(string); !ok {

View on GitHub (pinned to 27d5a3981a)

Solutions

1. Set the role to one of the five exact lowercase strings; map legacy 'function' to 'tool'.
2. Re-export the corpus with a role-normalization step.
3. Drop or rewrite messages with custom roles before loading if they are not needed for the cache benchmark.

Example fix

// before
{"role":"function","content":"..."}
// after
{"role":"tool","tool_call_id":"call_1","content":"..."}

var allowedRoles = map[string]bool{"system": true, "developer": true, "user": true, "assistant": true, "tool": true}

func normalizeRole(role string) (string, error) {
	switch strings.ToLower(strings.TrimSpace(role)) {
	case "function":
		return "tool", nil
	case "system", "developer", "user", "assistant", "tool":
		return strings.ToLower(strings.TrimSpace(role)), nil
	default:
		return "", fmt.Errorf("unsupported role %q", role)
	}
}

func isValidRole(role string) bool {
	switch role {
	case "system", "developer", "user", "assistant", "tool":
		return true
	}
	return false
}

if err := validateCorpusMessage(msg, limits); err != nil {
	if strings.Contains(err.Error(), "unsupported role") {
		msg.Role, err = normalizeRole(msg.Role)
		if err != nil {
			return err
		}
	}

When it happens

Trigger: validateCorpusMessage runs over each message of a decoded corpus row; any message whose Role string is not exactly one of the five allowed values returns this error, which then surfaces wrapped as 'message %d: unsupported role %q'.

Common situations: Corpora exported from older OpenAI formats using 'function' role, capitalized 'User'/'Assistant', or custom roles injected by middleware; sed/regex edits that mangle the role field.

Related errors

• cachebench: nil corpus reader
• message exceeds byte limit
• tool message requires tool_call_id
• cachebench: invalid corpus
• cachebench: no providers

AI-assisted analysis of JuliusBrussee/caveman@27d5a3981a (2026-08-15). Data as JSON: /api/errors/c235e294444bf7fa. Report an issue: GitHub.