{"record":{"id":"3e56f6f56e550d58","repo":"siyuan-note/siyuan","slug":"http-d","errorCode":null,"errorMessage":"HTTP %d","messagePattern":"HTTP %d","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"kernel/util/webfetch.go","lineNumber":61,"sourceCode":"\tif err != nil || (u.Scheme != \"http\" && u.Scheme != \"https\") {\n\t\treturn \"\", errors.New(\"URL must start with http:// or https://\")\n\t}\n\tif u.Host == \"\" {\n\t\treturn \"\", errors.New(\"URL has no host\")\n\t}\n\n\tif err := CheckHostSSRF(u.Hostname()); err != nil {\n\t\treturn \"\", err\n\t}\n\n\tresp, err := httpclient.NewBrowserRequest().Get(rawURL)\n\tif err != nil {\n\t\treturn \"\", errors.New(\"fetch failed: \" + err.Error())\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode >= 400 {\n\t\treturn \"\", fmt.Errorf(\"HTTP %d\", resp.StatusCode)\n\t}\n\n\tcontentType := resp.Header.Get(\"Content-Type\")\n\tmaxReadBytes := int64(maxWebFetchBytes)\n\tif !strings.HasPrefix(contentType, \"text/html\") && !strings.HasPrefix(contentType, \"text/plain\") {\n\t\tmaxReadBytes = maxWebFetchFileBytes\n\t}\n\tif resp.ContentLength > maxReadBytes {\n\t\treturn \"\", errors.New(\"response too large\")\n\t}\n\n\tbody, err := io.ReadAll(io.LimitReader(resp.Body, maxReadBytes))\n\tif err != nil {\n\t\treturn \"\", errors.New(\"read body failed: \" + err.Error())\n\t}\n\n\tif !strings.HasPrefix(contentType, \"text/html\") && !strings.HasPrefix(contentType, \"text/plain\") {\n\t\timportDir := filepath.Join(TempDir, \"import\")","sourceCodeStart":43,"sourceCodeEnd":79,"githubUrl":"https://github.com/siyuan-note/siyuan/blob/251596fc0de2f9528c00c224252fd073a99973f4/kernel/util/webfetch.go#L43-L79","documentation":"Returned at webfetch.go:61 when the server responds with an HTTP status >= 400. The connection succeeded and a response was received, but the endpoint signalled an error. fmt.Errorf(\"HTTP %d\", resp.StatusCode) encodes only the status code; the response body is discarded.","triggerScenarios":"Calling util.WebFetch against a dead link (404), a page behind auth or a bot block (401/403), a rate-limited endpoint (429), an upstream fault (500/502/503), or a geographically/legal-restricted resource (451).","commonSituations":"Sites that gate scrapers by User-Agent/Referer, Cloudflare bot-challenge pages, linkrot in stored notes, API endpoints requiring tokens, temporary 5xx during an outage.","solutions":["Open the same URL in a browser to see the real status and any challenge page.","If 403/429, the host is blocking the fetcher or rate-limiting; use a different URL or wait.","If 404, the link is dead — correct or remove it.","If 5xx, retry after a short backoff; it is usually transient on the server side.","For auth-required resources, fetch a public mirror or supply the content another way."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":"// The status-code error has the form \"HTTP 4xx/5xx\".\nfunc httpStatusOf(err error) (code int, ok bool) {\n    var n int\n    if _, e := fmt.Sscanf(err.Error(), \"HTTP %d\", &n); e == nil {\n        return n, true\n    }\n    return 0, false\n}","tryCatchPattern":"out, err := util.WebFetch(raw, \"markdown\")\nif err != nil {\n    if code, ok := httpStatusOf(err); ok {\n        switch {\n        case code == 429:\n            // rate-limited: back off and retry\n        case code >= 500:\n            // server-side: retry with backoff\n        default:\n            // 4xx other: do not retry, surface to user\n        }\n    }\n}","preventionTips":["Do not retry 4xx (except 429) — they will not self-heal.","Retry 5xx and 429 with exponential backoff.","Distinguish status errors from transport errors so retry policy matches the failure class."],"tags":["http","status-code","fetch"],"backgroundTag":null,"analyzedSha":"251596fc0de2f9528c00c224252fd073a99973f4","analyzedAt":"2026-08-12T21:18:37.123Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}