{"record":{"id":"2ea18921bc1cd1d7","repo":"koala73/worldmonitor","slug":"invalid-sitemap-location-location","errorCode":null,"errorMessage":"invalid sitemap location: ${location}","messagePattern":"invalid sitemap location: (.+?)","errorType":"validation","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/seo-indexnow-submit.mjs","lineNumber":170,"sourceCode":"export const INDEXNOW_BATCHES = Object.freeze([\n  batch(APEX_HOST, APEX_URLS, APEX_INDEXNOW_KEY),\n  batch(WWW_HOST, WWW_URLS),\n  // The sitemap lists each variant's canonical /dashboard; the bare root is the\n  // AI-crawler stub surface middleware.ts serves, so submit both.\n  ...INDEXNOW_VARIANT_HOSTS.map((host) => batch(host, urlsForHost(host, [`https://${host}/`]))),\n]);\n\nexport async function getPublishedBatches({ fetchImpl = globalThis.fetch } = {}) {\n  const origin = SITE_ORIGIN;\n  const pending = [`${origin}/sitemap.xml`];\n  const seen = new Set();\n  const pages = new Set();\n  const hosts = new Set(INDEXNOW_BATCHES.map(config => config.host));\n  while (pending.length > 0) {\n    const location = pending.shift();\n    const sitemapUrl = new URL(location);\n    if (sitemapUrl.origin !== origin || sitemapUrl.username || sitemapUrl.password || sitemapUrl.search || sitemapUrl.hash || !sitemapUrl.pathname.endsWith('.xml')) {\n      throw new Error(`invalid sitemap location: ${location}`);\n    }\n    if (seen.has(location)) continue;\n    seen.add(location);\n    if (seen.size > 50) throw new Error('published sitemap tree exceeds 50 documents');\n    const response = await fetchImpl(location, {\n      method: 'GET',\n      redirect: 'manual',\n      signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS),\n      headers: { Accept: 'application/xml', 'User-Agent': USER_AGENT },\n    });\n    if (response.status !== 200) throw new Error(`${location} returned ${response.status}, expected direct 200`);\n    const source = (await response.text()).trim();\n    const root = /^(?:<\\?xml[^?]*\\?>\\s*)?<(sitemapindex|urlset)\\b[^>]*>([\\s\\S]*)<\\/\\1>\\s*$/.exec(source);\n    if (!root) throw new Error(`invalid sitemap document: ${location}`);\n    const tag = root[1] === 'sitemapindex' ? 'sitemap' : 'url';\n    const entryPattern = new RegExp(`<!--[\\\\s\\\\S]*?-->|<${tag}>[\\\\s\\\\S]*?<\\\\/${tag}>`, 'g');\n    const entries = [...root[2].matchAll(entryPattern)].filter(([entry]) => !entry.startsWith('<!--'));\n    if (root[2].replace(entryPattern, '').trim()) throw new Error(`invalid sitemap entries: ${location}`);","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/koala73/worldmonitor/blob/7d06c8633d256c18e38133030bc3613976a96ec9/scripts/seo-indexnow-submit.mjs#L152-L188","documentation":"getPublishedBatches in the IndexNow submit script crawls the published sitemap tree starting from known locations. Each discovered location must be a same-origin URL with no credentials, query string, or hash, ending in .xml; anything else throws 'invalid sitemap location: <location>' to stop following malicious or malformed links found inside sitemap documents.","triggerScenarios":"A <loc> entry in a sitemapindex points to a different origin, contains ?query or #fragment, embeds user:pass@ credentials, or its pathname does not end with .xml — then it is shifted off the pending queue and rejected before fetching.","commonSituations":"Hosting rewrites that add query strings to sitemap URLs; sitemaps generated for the www domain while the script uses the apex origin; sitemap plugin outputting .xml.gz or HTML links.","solutions":["Fix the sitemap generation so all sitemap <loc> URLs are same-origin absolute https URLs ending in .xml with no query/hash","Compare the origin in INDEXNOW_BATCHES config with the origins actually emitted in sitemaps and align them","If a discovered URL is an HTML page, exclude it — only sitemapindex/urlset documents belong in the tree","Check for redirect chains: fetch the canonical sitemap URL and use the final same-origin URL"],"exampleFix":"// before\n<loc>https://cdn.example.com/sitemap-pages.xml?ts=123</loc>\n// after\n<loc>https://example.com/sitemap-pages.xml</loc>","handlingStrategy":"validation","validationCode":"const u = new URL(loc);\nconst isValidSitemapLocation = u.origin === ORIGIN && !u.username && !u.password && !u.search && !u.hash && u.pathname.endsWith('.xml');\nif (!isValidSitemapLocation) skip(loc);","typeGuard":"const isSafeSitemapUrl = (loc, origin) => {\n  try {\n    const u = new URL(loc);\n    return u.origin === origin && !u.username && !u.password && !u.search && !u.hash && u.pathname.endsWith('.xml');\n  } catch { return false; }\n};","tryCatchPattern":"try {\n  await getPublishedBatches();\n} catch (err) {\n  if (err.message.startsWith('invalid sitemap location:')) {\n    logger.error({ location: err.message.split(': ').pop() }, 'off-origin or malformed sitemap URL in published tree');\n  } else throw err;\n}","preventionTips":["Generate sitemap <loc> URLs from a single canonical origin constant","Never append query strings or fragments to sitemap URLs","Periodically audit the published sitemap tree for cross-origin links"],"tags":["seo","sitemap","url-validation"],"backgroundTag":"invalid-url-format","analyzedSha":"7d06c8633d256c18e38133030bc3613976a96ec9","analyzedAt":"2026-09-15T16:44:39.439Z","contentChangedAt":"2026-09-15T16:44:39.439Z","schemaVersion":2},"datasetVersion":"2026-09-15T18:17:12.389Z"}