{"record":{"id":"3f7837c5cbb88869","repo":"koala73/worldmonitor","slug":"exa-returned-no-content-for-url","errorCode":null,"errorMessage":"Exa returned no content for ${url}","messagePattern":"Exa returned no content for (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"consumer-prices-core/src/acquisition/exa.ts","lineNumber":23,"sourceCode":"  readonly name = 'exa' as const;\n\n  private readonly apiKey: string;\n  private readonly baseUrl = 'https://api.exa.ai';\n  private client: Exa;\n\n  constructor(apiKey: string) {\n    this.apiKey = apiKey;\n    this.client = new Exa(apiKey);\n  }\n\n  async fetch(url: string, _opts: FetchOptions = {}): Promise<FetchResult> {\n    const result = await this.client.getContents([url], {\n      text: { maxCharacters: 100_000 },\n      highlights: { numSentences: 5, highlightsPerUrl: 3 },\n    });\n\n    const item = result.results[0];\n    if (!item) throw new Error(`Exa returned no content for ${url}`);\n\n    return {\n      url,\n      html: item.text ?? '',\n      markdown: item.text ?? '',\n      statusCode: 200,\n      provider: this.name,\n      fetchedAt: new Date(),\n      metadata: { highlights: item.highlights },\n    };\n  }\n\n  async search(query: string, opts: SearchOptions = {}): Promise<SearchResult[]> {\n    const result = await this.request<{\n      results?: Array<{\n        url: string;\n        title?: string;\n        text?: string;","sourceCodeStart":5,"sourceCodeEnd":41,"githubUrl":"https://github.com/koala73/worldmonitor/blob/eeab0a219fce0f02a00603b532dbae9041b934ac/consumer-prices-core/src/acquisition/exa.ts#L5-L41","documentation":"ExaProvider.fetch calls exa-js getContents([url]) and indexes results[0]. Exa answered HTTP 200 but returned an empty results array — it has no cached or crawlable content for that exact URL — so the provider throws rather than fabricate an empty page (html/markdown stay coupled in FetchResult).","triggerScenarios":"URL unknown to Exa's index (freshly published page, obscure site, typo'd URL); paywalled or robots-excluded content Exa will not serve; a URL whose canonical form differs from the requested one, so Exa stores it under another key.","commonSituations":"Scraping newly published product pages; deep product URLs carrying session or tracking parameters; niche retailer domains with thin Exa coverage.","solutions":["Verify the URL loads in a browser to rule out a typo or 404","Fall back to another acquisition provider for that URL (e.g. Firecrawl scrape) instead of retrying Exa","Normalize to the canonical URL (strip tracking params) or submit the URL to Exa for crawling","Treat it as a per-URL condition: skip and continue the batch, not a provider outage"],"exampleFix":"// before\nconst result = await exa.fetch(url);   // throws 'Exa returned no content for ...'\n\n// after — degrade to the scrape provider for this URL only\nlet result;\ntry {\n  result = await exa.fetch(url);\n} catch (err) {\n  if (!/returned no content/.test(err.message)) throw err;\n  result = await firecrawl.fetch(url);\n}","handlingStrategy":"fallback","validationCode":"// Cheap liveness check before spending an Exa call\nasync function urlLooksLive(url) {\n  try { const res = await fetch(url, { method: 'HEAD' }); return res.status < 400; }\n  catch { return false; }\n}","typeGuard":"interface ExaContentsResult { results?: Array<{ text?: string; highlights?: string[] }> }\nfunction hasContentRow(r: ExaContentsResult | undefined): r is { results: NonNullable<ExaContentsResult['results']> } {\n  return Array.isArray(r?.results) && r.results.length > 0;\n}","tryCatchPattern":"try {\n  return await exa.fetch(url);\n} catch (err) {\n  if (err instanceof Error && /returned no content/.test(err.message)) {\n    return await firecrawl.fetch(url);      // per-URL fallback, provider stays healthy\n  }\n  throw err;                                 // auth/quota/HTTP errors are not content gaps\n}","preventionTips":["Chain providers (Exa → Firecrawl) so empty-content pages degrade instead of failing the run","Canonicalize URLs (strip tracking params) to maximize Exa index hits","Track per-URL no-content rates separately from provider error rates"],"tags":["exa","content-fetch","api-response","scraping"],"backgroundTag":"empty-api-response","analyzedSha":"eeab0a219fce0f02a00603b532dbae9041b934ac","analyzedAt":"2026-08-21T16:51:25.751Z","schemaVersion":2},"datasetVersion":"2026-08-23T13:39:53.451Z"}