Tencent/WeKnora · error

failed to unmarshal Metaso response: %w

Error message

failed to unmarshal Metaso response: %w

What it means

Returned by MetasoProvider.Search when the HTTP response body was read successfully (status 200) but json.Unmarshal cannot decode it into metasoSearchResponse. This indicates the body is not the expected JSON shape — HTML from a proxy, truncated JSON, or a Metaso API schema change that no longer matches the struct. The json error is wrapped with %w for diagnosis.

Source

Thrown at internal/infrastructure/web_search/metaso.go:117

	req.Header.Set("Content-Type", "application/json")

	logger.Infof(ctx, "[WebSearch][Metaso] query=%q maxResults=%d scope=%s", query, maxResults, p.scope)
	resp, err := p.client.Do(req)
	if err != nil {
		return nil, fmt.Errorf("failed to execute Metaso request: %w", err)
	}
	defer resp.Body.Close()
	respBody, err := readMetasoResponseBody(resp.Body)
	if err != nil {
		return nil, err
	}
	if resp.StatusCode != http.StatusOK {
		return nil, metasoHTTPError(resp.StatusCode, respBody)
	}

	var response metasoSearchResponse
	if err := json.Unmarshal(respBody, &response); err != nil {
		return nil, fmt.Errorf("failed to unmarshal Metaso response: %w", err)
	}
	results := make([]*types.WebSearchResult, 0, len(response.Webpages))
	for _, item := range response.Webpages {
		if strings.TrimSpace(item.Title) == "" && strings.TrimSpace(item.Link) == "" {
			continue
		}
		snippet := strings.TrimSpace(item.Summary)
		if snippet == "" {
			snippet = strings.TrimSpace(item.Snippet)
		}
		result := &types.WebSearchResult{Title: item.Title, URL: item.Link, Snippet: snippet, Content: item.RawContent, Source: "metaso"}
		if includeDate {
			if publishedAt, ok := parseMetasoDate(item.Date); ok {
				result.PublishedAt = &publishedAt
			}
		}
		results = append(results, result)
		if len(results) >= maxResults {

View on GitHub (pinned to 988cbb0330)

Solutions

1. Log the raw respBody on failure to see whether it is HTML, empty, or truncated JSON.
2. If the body is HTML, eliminate the intercepting proxy/captive portal or point the client directly at the Metaso endpoint.
3. Compare the actual JSON against metasoSearchResponse and update the struct if Metaso changed its schema.
4. Retry — a truncated body from a timeout may simply succeed on retry with a longer deadline.

Example fix

// before: opaque failure
if err := json.Unmarshal(respBody, &response); err != nil {
    return nil, fmt.Errorf("failed to unmarshal Metaso response: %w", err)
}

// after: surface body snippet for diagnosis
if err := json.Unmarshal(respBody, &response); err != nil {
    return nil, fmt.Errorf("failed to unmarshal Metaso response: %w (body: %.200s)", err, string(respBody))
}

func looksLikeMetasoResponse(b []byte) bool {
    var probe struct {
        Webpages []json.RawMessage `json:"webpages"`
    }
    return json.Unmarshal(b, &probe) == nil
}

// before full decode:
if !looksLikeMetasoResponse(respBody) {
    return nil, fmt.Errorf("unexpected Metaso response shape: %.200s", string(respBody))
}

if err := json.Unmarshal(respBody, &response); err != nil {
    log.Printf("metaso unmarshal failed: %v, body=%.500s", err, string(respBody))
    if !utf8.Valid(respBody) || len(bytes.TrimSpace(respBody)) == 0 {
        return nil, fmt.Errorf("empty/invalid body from metaso, retry advised: %w", err)
    }
    return nil, fmt.Errorf("failed to unmarshal Metaso response: %w", err)
}

When it happens

Trigger: Search (internal/infrastructure/web_search/metaso.go:117): after a 200 status, json.Unmarshal(respBody, &response) fails — non-JSON HTML body, empty body, malformed/truncated JSON, or response fields with unexpected types vs metasoSearchResponse.

Common situations: TLS-intercepting proxy or captive portal injecting HTML with a 200 status; Metaso API contract change; response cut off by an aggressive timeout; custom gateway returning its own JSON envelope instead of Metaso's.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

• failed to unmarshal response: %w
• unmarshal page: %w
• unmarshal database: %w
• unmarshal data_source: %w
• unmarshal block children response: %w

AI-assisted analysis of Tencent/WeKnora@988cbb0330 (2026-09-02). Data as JSON: /api/errors/37a5af13d598a16f. Report an issue: GitHub.