siyuan-note/siyuan · error

unsupported multimodal provider protocol

Error message

unsupported multimodal provider protocol: %s

What it means

validateImageModel checks the AI image-generation provider's configured protocol before calling it. SiYuan only supports the OpenAI chat-completions and OpenAI responses protocols for multimodal (image) providers; when the provider's Protocol field is set to any other non-empty value, this error is thrown and image generation is aborted. An empty Protocol is allowed and treated as the default chat-completions protocol.

Solutions

  1. Open the provider settings and change the protocol to one of the supported values (OpenAI chat-completions or responses), or clear the protocol field to use the default
  2. If editing conf.json directly, set the provider's "protocol" to exactly the supported constant or remove the key entirely
  3. If the provider genuinely only speaks another protocol, route it through an OpenAI-compatible proxy/gateway that exposes chat-completions or responses endpoints
  4. Update the kernel to a version that supports the protocol string if it was introduced in a newer release

Example fix

// before
provider.Protocol = "anthropic"
// after
provider.Protocol = "openai-chat-completions" // or "openai-responses", or "" for default
Defensive patterns

Strategy: validation

Validate before calling

func validImageProtocol(p *conf.Provider) bool {
    return p == nil || p.Protocol == "" || p.Protocol == util.OpenAIProtocolChatCompletions || p.Protocol == util.OpenAIProtocolResponses
}

Type guard

func isOpenAIProtocol(p *conf.Provider) bool {
    return p != nil && (p.Protocol == "" || p.Protocol == util.OpenAIProtocolChatCompletions || p.Protocol == util.OpenAIProtocolResponses)
}

Try / catch

if err := model.ValidateImageModel(provider, imageModel); err != nil {
    if strings.HasPrefix(err.Error(), "unsupported multimodal provider protocol") {
        // fall back to a known-good provider or surface a config-fix message
    }
    return err
}

Prevention

When it happens

Trigger: Calling GenerateImage with a conf.Provider whose Protocol field is a non-empty string other than util.OpenAIProtocolChatCompletions or util.OpenAIProtocolResponses (e.g. a hand-edited or plugin-written provider config with protocol "ollama", "gemini", "azure", or a typo like "chat-completions").

Common situations: Users copy a provider config from another tool that uses different protocol names (e.g. "anthropic", "google-ai"), manually edit conf.json to add a provider without knowing the exact accepted strings, or a sync/version change introduces a provider entry with a newer protocol value the running kernel build does not recognize.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of siyuan-note/siyuan@9f775e8a12 (2026-09-19). Data as JSON: /api/errors/ead0b71668a05afe. Report an issue: GitHub.

Appendix: source

Thrown at kernel/model/assets.go:593

		RevisedPrompt: generated.RevisedPrompt,
	}, nil
}

func resolveMultimodalDocument(documentID string) (*treenode.BlockTree, error) {
	bt := treenode.GetBlockTree(strings.TrimSpace(documentID))
	if bt == nil {
		return nil, errors.New("document not found: " + documentID)
	}
	return bt, nil
}

func validateImageModel(provider *conf.Provider, imageModel *conf.Model) error {
	if provider == nil || imageModel == nil {
		return errors.New("image model is not configured")
	}
	if provider.Protocol != "" && provider.Protocol != util.OpenAIProtocolChatCompletions &&
		provider.Protocol != util.OpenAIProtocolResponses {
		return fmt.Errorf("unsupported multimodal provider protocol: %s", provider.Protocol)
	}
	return nil
}

func documentReferencesImage(rootID, assetPath string) bool {
	paths, err := DocImageAssets(rootID)
	if err != nil {
		return false
	}
	wanted := AssetPathWithoutQuery(assetPath)
	for _, current := range paths {
		if AssetPathWithoutQuery(current) == wanted {
			return true
		}
	}
	return false
}

View on GitHub (pinned to 9f775e8a12)