jackwener/OpenCLI · error · CommandExecutionError

Malformed toutiao recommend row: invalid source_url ${JSON.s

Error message

Malformed toutiao recommend row: invalid source_url ${JSON.stringify(raw)}.

What it means

firstPartyToutiaoUrl catches failures from the URL constructor itself (invalid URL syntax) and rethrows them as this CommandExecutionError. It means the row's source_url could not be parsed as a URL at all — distinct from 3824, where the URL parses but is off-domain.

Source

Thrown at clis/toutiao/utils.js:214

    // The feed serves protocol-relative image URLs (`//p3.pstatp.com/...`);
    // hot-board already returns absolute ones, so normalise for parity.
    return raw.startsWith('//') ? `https:${raw}` : raw;
}

function firstPartyToutiaoUrl(sourceUrl) {
    const raw = trimOrNull(sourceUrl);
    if (!raw) return null;
    try {
        const url = raw.startsWith('/')
            ? new URL(raw, 'https://www.toutiao.com')
            : new URL(raw);
        if (!/^https?:$/i.test(url.protocol) || !/(^|\.)toutiao\.com$/i.test(url.hostname)) {
            throw new CommandExecutionError(`Malformed toutiao recommend row: off-domain source_url ${JSON.stringify(raw)}.`);
        }
        return url;
    } catch (error) {
        if (error instanceof CommandExecutionError) throw error;
        throw new CommandExecutionError(`Malformed toutiao recommend row: invalid source_url ${JSON.stringify(raw)}.`);
    }
}

function articleIdFromToutiaoPath(pathname) {
    return String(pathname || '').match(/^\/(?:group|article)\/([A-Za-z0-9_-]+)\/?$/)?.[1] || '';
}

function buildGroupIdentity(sourceUrl, rawGroupId) {
    const source = firstPartyToutiaoUrl(sourceUrl);
    const sourceId = source ? articleIdFromToutiaoPath(source.pathname) : '';
    const explicitId = trimOrNull(rawGroupId);
    const groupId = explicitId || trimOrNull(sourceId);
    if (!groupId) {
        throw new CommandExecutionError('Malformed toutiao recommend row: missing group_id and article source_url.');
    }
    if (!/^[A-Za-z0-9_-]+$/.test(groupId)) {
        throw new CommandExecutionError(`Malformed toutiao recommend row: invalid group_id ${JSON.stringify(rawGroupId)}.`);
    }

View on GitHub (pinned to 49907e53dc)

Solutions

  1. Inspect the raw row and fix or drop it; the URL is unparseable so it cannot be salvaged as-is.
  2. Trim whitespace and decode HTML entities (&) before passing the value.
  3. Validate with a try { new URL(v) } catch wrapper before calling the command.
  4. If the value starts with a path, pass it with the leading '/' so the toutiao.com base is applied.

Example fix

// before
firstPartyToutiaoUrl('  https://www.toutiao.com/article/123/  ');
// after
firstPartyToutiaoUrl(raw.trim().replace(/&/g, '&'));
Defensive patterns

Strategy: type-guard

Validate before calling

function isParsableUrl(u) { try { new URL(u, 'https://www.toutiao.com'); return true; } catch { return false; } }
if (!isParsableUrl(raw)) skipRow(raw);

Type guard

const isParseableUrl = (v) => { try { new URL(String(v), 'https://www.toutiao.com'); return true; } catch { return false; } };

Try / catch

try {
  const url = firstPartyToutiaoUrl(row.source_url);
} catch (e) {
  if (e instanceof CommandExecutionError && e.message.includes('invalid source_url')) return dropRow(row);
  throw e;
}

Prevention

When it happens

Trigger: A source_url like 'not a url', 'http://', or containing raw spaces/characters that make new URL() throw, when the value does not start with '/' (relative '/group/...' values get a base URL and usually parse).

Common situations: Upstream feed returning empty/truncated HTML in the source_url field; escaped or double-encoded URLs; fields shifted during CSV/JSON extraction so an unrelated string lands in source_url.

Understand the failure class

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/cb95752f896de41c. Report an issue: GitHub.