{"record":{"id":"c0e7613f0708b9e8","repo":"Tencent/WeKnora","slug":"http-d-s","errorCode":null,"errorMessage":"HTTP %d %s","messagePattern":"HTTP %d %s","errorType":"http","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/datasource/connector/rss/client.go","lineNumber":84,"sourceCode":"\t\t\treq.Header.Set(k, v)\n\t\t}\n\t}\n\tif req.Header.Get(\"User-Agent\") == \"\" {\n\t\treq.Header.Set(\"User-Agent\", defaultUserAgent)\n\t}\n\tif req.Header.Get(\"Accept\") == \"\" {\n\t\treq.Header.Set(\"Accept\",\n\t\t\t\"application/rss+xml, application/atom+xml, application/xml, text/xml, application/json, text/html;q=0.9, */*;q=0.8\")\n\t}\n\n\tresp, err := c.httpClient.Do(req)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"fetch failed: %w\", err)\n\t}\n\tdefer resp.Body.Close()\n\n\tif resp.StatusCode < 200 || resp.StatusCode >= 300 {\n\t\treturn nil, fmt.Errorf(\"HTTP %d %s\", resp.StatusCode, resp.Status)\n\t}\n\n\tbody, err := io.ReadAll(io.LimitReader(resp.Body, maxSize))\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"read body failed: %w\", err)\n\t}\n\treturn body, nil\n}\n\n// fetchFeed retrieves the raw bytes of a feed document.\nfunc (c *client) fetchFeed(ctx context.Context, feedURL string) ([]byte, error) {\n\treturn c.fetch(ctx, feedURL, maxFeedSize, true)\n}\n\n// extractArticle fetches an article page and returns the readability-cleaned\n// main content as HTML, plus the extracted title (may be empty). Returns an\n// error if the page can't be fetched or no readable content is found, so the\n// caller can fall back to feed-provided content.","sourceCodeStart":66,"sourceCodeEnd":102,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/datasource/connector/rss/client.go#L66-L102","documentation":"Returned when the feed or article server responds with a non-2xx HTTP status. The error message carries the numeric status code and Go's canonical status text (e.g. \"HTTP 404 Not Found\"). No body is read or parsed in this case, so any error page content is discarded.","triggerScenarios":"The remote server returned 4xx or 5xx for a GET on the feed or article URL: 404 after a feed moved, 401/403 when auth headers are missing/invalid (feed fetches attach configured headers; article fetches deliberately do not), 429 rate limiting, or 5xx server errors.","commonSituations":"Blog migrated and removed the old feed URL (404); feed requires a subscription token that was rotated (401); Cloudflare/WAF blocking the default User-Agent (403); aggressive per-IP rate limits on polling (429); transient upstream errors (502/503).","solutions":["Read the embedded status code: for 404 verify the feed URL still exists and update the config; for 401/403 check the configured auth headers in the datasource config.","For 429, back off and retry later; reduce polling frequency.","For 5xx, retry with backoff — the error is usually transient on the feed host's side.","For 403 on article fetches, note that auth headers are intentionally withheld from third-party pages; if the article needs cookies, that content cannot be fetched by design."],"exampleFix":"// before\nif err := connector.Validate(ctx, cfg); err != nil {\n    return err\n}\n// after\nif err := connector.Validate(ctx, cfg); err != nil {\n    var se *statusError // or strings.Contains on \"HTTP 4\"\n    if errors.As(err, &se) && se.Code >= 500 {\n        return retryLater(err)\n    }\n    return err\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"data, err := cli.fetchFeed(ctx, feedURL)\nif err != nil {\n    msg := err.Error()\n    switch {\n    case strings.Contains(msg, \"HTTP 429\"):\n        return backoffRetryAfterRateLimit(msg)\n    case strings.Contains(msg, \"HTTP 404\"):\n        return markFeedDead(feedURL)\n    case strings.Contains(msg, \"HTTP 5\"):\n        return backoffRetry(msg)\n    default:\n        return err\n    }\n}","preventionTips":["Match on the embedded status code text to branch on 4xx vs 5xx vs 429.","Rotate or verify auth headers for feeds returning 401/403.","Back off on 429/503 and honor Retry-After semantics by pausing the sync.","Monitor feed URLs with periodic probes so dead URLs are caught before syncs."],"tags":["http","network","rss","status-code"],"backgroundTag":"http-error-status","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}