siyuan-note/siyuan · error

HTTP %d

Error message

HTTP %d

What it means

Returned at webfetch.go:61 when the server responds with an HTTP status >= 400. The connection succeeded and a response was received, but the endpoint signalled an error. fmt.Errorf("HTTP %d", resp.StatusCode) encodes only the status code; the response body is discarded.

Source

Thrown at kernel/util/webfetch.go:61

	if err != nil || (u.Scheme != "http" && u.Scheme != "https") {
		return "", errors.New("URL must start with http:// or https://")
	}
	if u.Host == "" {
		return "", errors.New("URL has no host")
	}

	if err := CheckHostSSRF(u.Hostname()); err != nil {
		return "", err
	}

	resp, err := httpclient.NewBrowserRequest().Get(rawURL)
	if err != nil {
		return "", errors.New("fetch failed: " + err.Error())
	}
	defer resp.Body.Close()

	if resp.StatusCode >= 400 {
		return "", fmt.Errorf("HTTP %d", resp.StatusCode)
	}

	contentType := resp.Header.Get("Content-Type")
	maxReadBytes := int64(maxWebFetchBytes)
	if !strings.HasPrefix(contentType, "text/html") && !strings.HasPrefix(contentType, "text/plain") {
		maxReadBytes = maxWebFetchFileBytes
	}
	if resp.ContentLength > maxReadBytes {
		return "", errors.New("response too large")
	}

	body, err := io.ReadAll(io.LimitReader(resp.Body, maxReadBytes))
	if err != nil {
		return "", errors.New("read body failed: " + err.Error())
	}

	if !strings.HasPrefix(contentType, "text/html") && !strings.HasPrefix(contentType, "text/plain") {
		importDir := filepath.Join(TempDir, "import")

View on GitHub (pinned to 251596fc0d)

Solutions

1. Open the same URL in a browser to see the real status and any challenge page.
2. If 403/429, the host is blocking the fetcher or rate-limiting; use a different URL or wait.
3. If 404, the link is dead — correct or remove it.
4. If 5xx, retry after a short backoff; it is usually transient on the server side.
5. For auth-required resources, fetch a public mirror or supply the content another way.

// The status-code error has the form "HTTP 4xx/5xx".
func httpStatusOf(err error) (code int, ok bool) {
    var n int
    if _, e := fmt.Sscanf(err.Error(), "HTTP %d", &n); e == nil {
        return n, true
    }
    return 0, false
}

out, err := util.WebFetch(raw, "markdown")
if err != nil {
    if code, ok := httpStatusOf(err); ok {
        switch {
        case code == 429:
            // rate-limited: back off and retry
        case code >= 500:
            // server-side: retry with backoff
        default:
            // 4xx other: do not retry, surface to user
        }
    }
}

When it happens

Trigger: Calling util.WebFetch against a dead link (404), a page behind auth or a bot block (401/403), a rate-limited endpoint (429), an upstream fault (500/502/503), or a geographically/legal-restricted resource (451).

Common situations: Sites that gate scrapers by User-Agent/Referer, Cloudflare bot-challenge pages, linkrot in stored notes, API endpoints requiring tokens, temporary 5xx during an outage.

Related errors

• fetch failed: %s
• response too large
• download custom emoji failed with status %d
• get bazaar package failed: %s
• get stage index failed

AI-assisted analysis of siyuan-note/siyuan@251596fc0d (2026-08-12). Data as JSON: /api/errors/3e56f6f56e550d58. Report an issue: GitHub.