siyuan-note/siyuan · error

HTML to Markdown conversion failed: %s

Error message

HTML to Markdown conversion failed: %s

What it means

Returned at webfetch.go:110 when safeHTML2Markdown returns a non-nil err. Per safeHTML2Markdown (webfetch.go:122-130) that err comes from one of two sources: a normal error returned by engine.HTML2Markdown, or a panic recovered by the deferred func (which becomes error 1187). Either way, Lute could not convert the fetched HTML to Markdown. Note the full HTML body (up to 5 MiB) is fed to Lute untruncated — truncation only happens AFTER conversion.

Source

Thrown at kernel/util/webfetch.go:110

	isHTML := strings.HasPrefix(contentType, "text/html")
	if !isHTML {
		return truncateRunes(htmlStr, maxWebFetchChars), nil
	}

	if htmlStr == "" {
		return "", nil
	}

	engine := NewLute()
	var result string
	switch format {
	case "text":
		result, _ = safeHTML2Text(engine, htmlStr)
	default: // markdown
		md, mdErr := safeHTML2Markdown(engine, htmlStr)
		if mdErr != nil {
			return "", errors.New("HTML to Markdown conversion failed: " + mdErr.Error())
		}
		result = md
	}

	if result == "" {
		return htmlStr, nil
	}

	return truncateRunes(result, maxWebFetchChars), nil
}

func safeHTML2Markdown(engine *lute.Lute, htmlStr string) (result string, err error) {
	defer func() {
		if r := recover(); r != nil {
			err = fmt.Errorf("HTML to Markdown panicked: %v", r)
		}
	}()
	result, err = engine.HTML2Markdown(htmlStr)

View on GitHub (pinned to 251596fc0d)

Solutions

1. Retry the same URL with format="text" — safeHTML2Text takes a different code path and may succeed.
2. Try a cleaner/alternate URL for the same content.
3. Update lute (it is built from 88250/lute); a newer build may fix the grammar path.
4. Capture the failing HTML sample and reduce it to a minimal reproducer for the lute repo.

Example fix

// before
out, err := util.WebFetch(url, "markdown")

// after: fall back to the text path which uses safeHTML2Text
out, err := util.WebFetch(url, "markdown")
if err != nil && strings.HasPrefix(err.Error(), "HTML to Markdown") {
    out, err = util.WebFetch(url, "text")
}

func isHTML2MarkdownFailed(err error) bool {
    return err != nil && strings.HasPrefix(err.Error(), "HTML to Markdown")
}

out, err := util.WebFetch(raw, "markdown")
if err != nil && isHTML2MarkdownFailed(err) {
    // fall back to the text-extraction path, then to raw HTML
    out, err = util.WebFetch(raw, "text")
}

When it happens

Trigger: Severely malformed HTML, pages served as text/html but actually JSON/binary, very large minified/obfuscated HTML, deeply nested tables, or any input that trips an internal lute grammar path.

Common situations: Scraping SPAs whose real content is JS-rendered, pages dominated by inline script/style blobs, obfuscated pages, lute version regression.

Related errors

• parse tree [%s] failed
• HTML to Markdown panicked: %v
• HTML to text panicked: %v
• name must end in .html or .htm
• empty result

AI-assisted analysis of siyuan-note/siyuan@251596fc0d (2026-08-12). Data as JSON: /api/errors/7b577c767d450162. Report an issue: GitHub.