{"record":{"id":"7b577c767d450162","repo":"siyuan-note/siyuan","slug":"html-to-markdown-conversion-failed-s","errorCode":null,"errorMessage":"HTML to Markdown conversion failed: %s","messagePattern":"HTML to Markdown conversion failed: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"kernel/util/webfetch.go","lineNumber":110,"sourceCode":"\n\tisHTML := strings.HasPrefix(contentType, \"text/html\")\n\tif !isHTML {\n\t\treturn truncateRunes(htmlStr, maxWebFetchChars), nil\n\t}\n\n\tif htmlStr == \"\" {\n\t\treturn \"\", nil\n\t}\n\n\tengine := NewLute()\n\tvar result string\n\tswitch format {\n\tcase \"text\":\n\t\tresult, _ = safeHTML2Text(engine, htmlStr)\n\tdefault: // markdown\n\t\tmd, mdErr := safeHTML2Markdown(engine, htmlStr)\n\t\tif mdErr != nil {\n\t\t\treturn \"\", errors.New(\"HTML to Markdown conversion failed: \" + mdErr.Error())\n\t\t}\n\t\tresult = md\n\t}\n\n\tif result == \"\" {\n\t\treturn htmlStr, nil\n\t}\n\n\treturn truncateRunes(result, maxWebFetchChars), nil\n}\n\nfunc safeHTML2Markdown(engine *lute.Lute, htmlStr string) (result string, err error) {\n\tdefer func() {\n\t\tif r := recover(); r != nil {\n\t\t\terr = fmt.Errorf(\"HTML to Markdown panicked: %v\", r)\n\t\t}\n\t}()\n\tresult, err = engine.HTML2Markdown(htmlStr)","sourceCodeStart":92,"sourceCodeEnd":128,"githubUrl":"https://github.com/siyuan-note/siyuan/blob/251596fc0de2f9528c00c224252fd073a99973f4/kernel/util/webfetch.go#L92-L128","documentation":"Returned at webfetch.go:110 when safeHTML2Markdown returns a non-nil err. Per safeHTML2Markdown (webfetch.go:122-130) that err comes from one of two sources: a normal error returned by engine.HTML2Markdown, or a panic recovered by the deferred func (which becomes error 1187). Either way, Lute could not convert the fetched HTML to Markdown. Note the full HTML body (up to 5 MiB) is fed to Lute untruncated — truncation only happens AFTER conversion.","triggerScenarios":"Severely malformed HTML, pages served as text/html but actually JSON/binary, very large minified/obfuscated HTML, deeply nested tables, or any input that trips an internal lute grammar path.","commonSituations":"Scraping SPAs whose real content is JS-rendered, pages dominated by inline script/style blobs, obfuscated pages, lute version regression.","solutions":["Retry the same URL with format=\"text\" — safeHTML2Text takes a different code path and may succeed.","Try a cleaner/alternate URL for the same content.","Update lute (it is built from 88250/lute); a newer build may fix the grammar path.","Capture the failing HTML sample and reduce it to a minimal reproducer for the lute repo."],"exampleFix":"// before\nout, err := util.WebFetch(url, \"markdown\")\n\n// after: fall back to the text path which uses safeHTML2Text\nout, err := util.WebFetch(url, \"markdown\")\nif err != nil && strings.HasPrefix(err.Error(), \"HTML to Markdown\") {\n    out, err = util.WebFetch(url, \"text\")\n}","handlingStrategy":"fallback","validationCode":null,"typeGuard":"func isHTML2MarkdownFailed(err error) bool {\n    return err != nil && strings.HasPrefix(err.Error(), \"HTML to Markdown\")\n}","tryCatchPattern":"out, err := util.WebFetch(raw, \"markdown\")\nif err != nil && isHTML2MarkdownFailed(err) {\n    // fall back to the text-extraction path, then to raw HTML\n    out, err = util.WebFetch(raw, \"text\")\n}","preventionTips":["Keep a format=\"text\" fallback ready for pages lute cannot convert to Markdown.","Avoid pointing WebFetch at JS-rendered SPAs whose raw HTML is dominated by script blobs.","Stay on a current lute build; conversion fixes land upstream."],"tags":["parsing","lute","html","conversion"],"backgroundTag":null,"analyzedSha":"251596fc0de2f9528c00c224252fd073a99973f4","analyzedAt":"2026-08-12T21:18:37.123Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}