{"record":{"id":"300699ff1593b7b6","repo":"jackwener/OpenCLI","slug":"malformed-toutiao-recommend-row-off-domain-source","errorCode":null,"errorMessage":"Malformed toutiao recommend row: off-domain source_url ${JSON.stringify(raw)}.","messagePattern":"Malformed toutiao recommend row: off-domain source_url (.+?)\\.","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/toutiao/utils.js","lineNumber":209,"sourceCode":"];\n\nfunction pickFeedImage(item) {\n    const raw = trimOrNull(item?.image_url) || trimOrNull(item?.middle_image?.url);\n    if (!raw) return null;\n    // The feed serves protocol-relative image URLs (`//p3.pstatp.com/...`);\n    // hot-board already returns absolute ones, so normalise for parity.\n    return raw.startsWith('//') ? `https:${raw}` : raw;\n}\n\nfunction firstPartyToutiaoUrl(sourceUrl) {\n    const raw = trimOrNull(sourceUrl);\n    if (!raw) return null;\n    try {\n        const url = raw.startsWith('/')\n            ? new URL(raw, 'https://www.toutiao.com')\n            : new URL(raw);\n        if (!/^https?:$/i.test(url.protocol) || !/(^|\\.)toutiao\\.com$/i.test(url.hostname)) {\n            throw new CommandExecutionError(`Malformed toutiao recommend row: off-domain source_url ${JSON.stringify(raw)}.`);\n        }\n        return url;\n    } catch (error) {\n        if (error instanceof CommandExecutionError) throw error;\n        throw new CommandExecutionError(`Malformed toutiao recommend row: invalid source_url ${JSON.stringify(raw)}.`);\n    }\n}\n\nfunction articleIdFromToutiaoPath(pathname) {\n    return String(pathname || '').match(/^\\/(?:group|article)\\/([A-Za-z0-9_-]+)\\/?$/)?.[1] || '';\n}\n\nfunction buildGroupIdentity(sourceUrl, rawGroupId) {\n    const source = firstPartyToutiaoUrl(sourceUrl);\n    const sourceId = source ? articleIdFromToutiaoPath(source.pathname) : '';\n    const explicitId = trimOrNull(rawGroupId);\n    const groupId = explicitId || trimOrNull(sourceId);\n    if (!groupId) {","sourceCodeStart":191,"sourceCodeEnd":227,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/toutiao/utils.js#L191-L227","documentation":"firstPartyToutiaoUrl normalizes a recommend row's source_url into an absolute URL and asserts it points at toutiao.com over http(s). If the URL parses but the protocol is not http/https or the hostname is not toutiao.com (or a subdomain), it throws this CommandExecutionError because downstream group-identity building requires first-party Toutiao links.","triggerScenarios":"A recommend feed row whose source_url points to another domain (e.g. https://example.com/article/123, a partner site, or a shortlink service like t.cn) or a non-http scheme (javascript:, ftp:).","commonSituations":"Toutiao mixing third-party/partner content into the recommend feed; scraped fixtures from a different site; a CDN or redirect domain appearing in the field; test data built with placeholder URLs.","solutions":["Filter out rows whose source_url host is not toutiao.com before processing the feed.","Resolve redirects yourself first; if the final target is off-domain, skip the row.","If you own the data source, fix source_url to a www.toutiao.com article/group URL.","Wrap the row parse in try/catch on CommandExecutionError and record the row as skipped."],"exampleFix":"// before\nconst url = firstPartyToutiaoUrl(row.source_url);\n// after\nlet url = null;\ntry { url = firstPartyToutiaoUrl(row.source_url); }\ncatch (e) { skipRow(row, e.message); }","handlingStrategy":"validation","validationCode":"function isToutiaoUrl(u) {\n  try {\n    const url = new URL(u, 'https://www.toutiao.com');\n    return /^https?:$/.test(url.protocol) && /(^|\\.)toutiao\\.com$/i.test(url.hostname);\n  } catch { return false; }\n}","typeGuard":"const isToutiaoSource = (raw) => !!raw && (() => { try { const u = new URL(raw.startsWith('/') ? new URL(raw, 'https://www.toutiao.com') : raw); return /^https?:$/i.test(u.protocol) && /(^|\\.)toutiao\\.com$/i.test(u.hostname); } catch { return false; } })();","tryCatchPattern":"try {\n  const url = firstPartyToutiaoUrl(row.source_url);\n} catch (e) {\n  if (e instanceof CommandExecutionError) return skipRow(row, e.message);\n  throw e;\n}","preventionTips":["Validate host against /(^|\\.)toutiao\\.com$/i before processing feed rows.","Filter out partner/third-party rows at ingestion time.","Resolve redirects before passing URLs so final targets are checked."],"tags":["url-validation","data-integrity","toutiao"],"backgroundTag":"off-domain-url-rejected","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}