{"record":{"id":"93deaa04696df68f","repo":"Tencent/WeKnora","slug":"failed-to-unmarshal-response-w-93deaa","errorCode":null,"errorMessage":"failed to unmarshal response: %w","messagePattern":"failed to unmarshal response: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"internal/infrastructure/web_search/bing.go","lineNumber":116,"sourceCode":"\tresp, err := p.client.Do(req)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\tdefer resp.Body.Close()\n\n\tbody, err := io.ReadAll(resp.Body)\n\tif err != nil {\n\t\treturn nil, err\n\t}\n\n\tif resp.StatusCode != http.StatusOK {\n\t\tlogger.Warnf(ctx, \"[WebSearch][Bing] API returned status %d: %s\", resp.StatusCode, string(body))\n\t\treturn nil, fmt.Errorf(\"bing API returned status %d: %s\", resp.StatusCode, string(body))\n\t}\n\n\tvar respData bingSearchResponse\n\tif err := json.Unmarshal(body, &respData); err != nil {\n\t\treturn nil, fmt.Errorf(\"failed to unmarshal response: %w\", err)\n\t}\n\tresults := make([]*types.WebSearchResult, 0, len(respData.WebPages.Value))\n\tfor _, item := range respData.WebPages.Value {\n\t\tresults = append(results, &types.WebSearchResult{\n\t\t\tTitle:       item.Name,\n\t\t\tURL:         item.URL,\n\t\t\tSnippet:     item.Snippet,\n\t\t\tSource:      \"bing\",\n\t\t\tPublishedAt: &item.DateLastCrawled,\n\t\t})\n\t}\n\treturn results, nil\n}\n\n// bingSearchResponse defines the response structure for Bing search API.\ntype bingSearchResponse struct {\n\tType         string `json:\"_type\"`\n\tQueryContext struct {","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/Tencent/WeKnora/blob/988cbb03305e055d8ebb7d46d9ac6cc0803cd074/internal/infrastructure/web_search/bing.go#L98-L134","documentation":"json.Unmarshal could not decode the Bing response body into bingSearchResponse, so doSearch wraps the underlying error with this message. It means the body is not valid JSON or its shape doesn't match the expected struct.","triggerScenarios":"Bing returns HTML/XML error pages, a proxy/CAPTCHA response, or JSON missing the expected webPages.value structure; body was truncated by a network fault.","commonSituations":"Hitting a captive proxy that injects HTML; Bing changing/retiring its response schema; response gzip/compression not handled; a 200 response from a gateway instead of the real API.","solutions":["Log or inspect the raw body (the provider already has it in doSearch) to see what was actually returned.","Confirm no corporate proxy or WAF is intercepting the request (headers, TLS MITM).","Verify the Bing API version and that the endpoint still matches the struct definition (bingSearchResponse).","Ensure the HTTP client is not double-decompressing or skipping gzip responses."],"exampleFix":"// before\nvar respData bingSearchResponse\nif err := json.Unmarshal(body, &respData); err != nil { ... }\n// after\nif !json.Valid(body) {\n    return nil, fmt.Errorf(\"non-JSON response body: %.200s\", body)\n}\nvar respData bingSearchResponse\nif err := json.Unmarshal(body, &respData); err != nil { ... }","handlingStrategy":"try-catch","validationCode":null,"typeGuard":"func isUnmarshalErr(err error) bool {\n    return err != nil && strings.Contains(err.Error(), \"failed to unmarshal response\")\n}","tryCatchPattern":"results, err := provider.Search(ctx, query, 10, false)\nif err != nil && strings.Contains(err.Error(), \"failed to unmarshal response\") {\n    // log raw body via debug logging, fall back to another provider\n    return fallbackProvider.Search(ctx, query, 10, false)\n}","preventionTips":["Enable request/response logging in staging to catch schema drift early.","Keep bingSearchResponse struct in sync with the API version in use.","Check json.Valid() on bodies when debugging unexpected unmarshal errors.","Watch for proxy/WAF products injecting non-JSON content."],"tags":["json","parsing","web-search","go"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"988cbb03305e055d8ebb7d46d9ac6cc0803cd074","analyzedAt":"2026-09-02T14:41:08.344Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}