{"record":{"id":"62e3d28332e2a033","repo":"koala73/worldmonitor","slug":"invalid-sitemap-document-location","errorCode":null,"errorMessage":"invalid sitemap document: ${location}","messagePattern":"invalid sitemap document: (.+?)","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/seo-indexnow-submit.mjs","lineNumber":184,"sourceCode":"  while (pending.length > 0) {\n    const location = pending.shift();\n    const sitemapUrl = new URL(location);\n    if (sitemapUrl.origin !== origin || sitemapUrl.username || sitemapUrl.password || sitemapUrl.search || sitemapUrl.hash || !sitemapUrl.pathname.endsWith('.xml')) {\n      throw new Error(`invalid sitemap location: ${location}`);\n    }\n    if (seen.has(location)) continue;\n    seen.add(location);\n    if (seen.size > 50) throw new Error('published sitemap tree exceeds 50 documents');\n    const response = await fetchImpl(location, {\n      method: 'GET',\n      redirect: 'manual',\n      signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),\n      headers: { Accept: 'application/xml', 'User-Agent': USER_AGENT },\n    });\n    if (response.status !== 200) throw new Error(`${location} returned ${response.status}, expected direct 200`);\n    const source = (await response.text()).trim();\n    const root = /^(?:<\\?xml[^?]*\\?>\\s*)?<(sitemapindex|urlset)\\b[^>]*>([\\s\\S]*)<\\/\\1>\\s*$/.exec(source);\n    if (!root) throw new Error(`invalid sitemap document: ${location}`);\n    const tag = root[1] === 'sitemapindex' ? 'sitemap' : 'url';\n    const entryPattern = new RegExp(`<!--[\\\\s\\\\S]*?-->|<${tag}>[\\\\s\\\\S]*?<\\\\/${tag}>`, 'g');\n    const entries = [...root[2].matchAll(entryPattern)].filter(([entry]) => !entry.startsWith('<!--'));\n    if (root[2].replace(entryPattern, '').trim()) throw new Error(`invalid sitemap entries: ${location}`);\n    const urls = entries.map(([entry]) => {\n      const locations = [...entry.matchAll(/<loc>\\s*([^<]+?)\\s*<\\/loc>/g)];\n      if (locations.length !== 1) throw new Error(`expected one location per sitemap entry: ${location}`);\n      return decodeXml(locations[0][1].trim());\n    });\n    if (urls.length === 0) throw new Error(`empty sitemap: ${location}`);\n    if (location === `${origin}/sitemap.xml`\n      && (root[1] !== 'sitemapindex' || urls.length !== SITEMAP_INDEX_MEMBERS.length\n        || new Set(urls).size !== urls.length || urls.some(url => !SITEMAP_INDEX_MEMBERS.includes(url)))) {\n      throw new Error('published root sitemap members do not match the declared inventory');\n    }\n    if (root[1] === 'sitemapindex') {\n      pending.push(...urls);\n    } else {","sourceCodeStart":166,"sourceCodeEnd":202,"githubUrl":"https://github.com/koala73/worldmonitor/blob/7d06c8633d256c18e38133030bc3613976a96ec9/scripts/seo-indexnow-submit.mjs#L166-L202","documentation":"After a 200 fetch the script validates the response body is a well-formed sitemap: an optional XML declaration followed by exactly one <sitemapindex> or <urlset> root element. If the body does not match this structure the script throws 'invalid sitemap document: <location>' instead of parsing arbitrary HTML or error pages.","triggerScenarios":"The 200 response body is HTML (a soft-404/captcha/login page served with 200), an XML parse error page, an empty body, gzip bytes not decompressed, extra content outside the root element, or misspelled root tags.","commonSituations":"WAF serving an interstitial with 200; server returning HTML error page with 200 status; compression middleware breaking XML; a generator emitting fragment/invalid XML; content-type confusion returning JSON instead of XML.","solutions":["Fetch the URL manually and confirm the body is a valid <sitemapindex> or <urlset> document","Fix whatever serves HTML/error pages with 200 at that URL (WAF interstitial, SPA fallback for .xml routes)","Verify the sitemap generator emits a single well-formed root element with XML declaration only","Check Content-Encoding handling so the script receives decompressed XML"],"exampleFix":"// before (soft-404 with 200)\n<html><body>Not Found</body></html>\n// after\n<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">...</urlset>","handlingStrategy":"validation","validationCode":"const body = (await (await fetch(url)).text()).trim();\nif (!/^<\\?xml|<(sitemapindex|urlset)\\b/.test(body)) console.warn(`${url} is not a sitemap document`);","typeGuard":"const isSitemapDoc = (source) => /^(?:<\\?xml[^?]*\\?>\\s*)?<(sitemapindex|urlset)\\b[^>]*>[\\s\\S]*<\\/(sitemapindex|urlset)>\\s*$/.test(source.trim());","tryCatchPattern":"try {\n  await getPublishedBatches();\n} catch (err) {\n  if (err.message.startsWith('invalid sitemap document:')) {\n    const loc = err.message.split(': ').slice(1).join(': ');\n    logger.error({ loc }, 'non-XML 200 body; check WAF soft-404 or generator output');\n  } else throw err;\n}","preventionTips":["Return application/xml with well-formed sitemaps only","Configure SPA/HTML fallbacks to exclude *.xml routes","Verify the sitemap renders without HTML interstitials before submitting"],"tags":["seo","sitemap","xml"],"backgroundTag":"xml-parse-error","analyzedSha":"7d06c8633d256c18e38133030bc3613976a96ec9","analyzedAt":"2026-09-15T16:44:39.439Z","contentChangedAt":"2026-09-15T16:44:39.439Z","schemaVersion":2},"datasetVersion":"2026-09-15T18:17:12.389Z"}