jackwener/OpenCLI · error · CommandExecutionError

Browser session required for substack archive

Error message

Browser session required for substack archive

What it means

loadSubstackArchive requires an active browser session to scrape a publication's /archive page, which is rendered client-side. Without a page it throws a CommandExecutionError before any navigation happens.

Source

Thrown at clis/substack/utils.js:80

          title,
          author,
          date,
          readTime,
          description: description.slice(0, 150),
          url: postUrl,
        });

        if (posts.length >= limit) break;
      }

      return posts;
    })()
  `);
    return Array.isArray(data) ? data : [];
}
export async function loadSubstackArchive(page, baseUrl, limit) {
    if (!page)
        throw new CommandExecutionError('Browser session required for substack archive');
    await page.goto(`${baseUrl}/archive`);
    await page.wait({ selector: ARCHIVE_POST_LINK_SELECTOR, timeout: 5 });
    const data = await page.evaluate(`
    (async () => {
      await new Promise((resolve) => setTimeout(resolve, 3000));
      const normalize = (value) => (value || '').replace(/\\s+/g, ' ').trim();
      const limit = ${Math.max(1, Math.min(limit, 50))};
      const grouped = new Map();

      for (const link of Array.from(document.querySelectorAll('a[href*="/p/"]'))) {
        const rawHref = link.getAttribute('href') || '';
        if (!rawHref || rawHref === '/p/upgrade') continue;

        const url = rawHref.startsWith('http') ? rawHref : ${JSON.stringify(baseUrl)} + rawHref;
        const text = normalize(link.textContent);
        if (!text) continue;
        if (/^(subscribe|paid|home|about|latest|top|discussions)$/i.test(text)) continue;
        if (/^[\\d,]+$/.test(text)) continue;

View on GitHub (pinned to 49907e53dc)

Solutions

  1. Open a browser session before calling loadSubstackArchive
  2. Add an explicit session-creation step and assert the page object exists
  3. Install/verify the browser runtime in CI or container environments
  4. Use the publication's RSS feed as a fallback for simple archive listings

Example fix

// before
await loadSubstackArchive(null, 'https://example.substack.com', 20); // throws
// after
if (!page) page = await openBrowserSession();
await loadSubstackArchive(page, 'https://example.substack.com', 20);
Defensive patterns

Strategy: validation

Validate before calling

function requirePage(page) { if (!page || typeof page.goto !== 'function') throw new Error('open a browser session before loading the substack archive'); }
requirePage(page);

Type guard

function hasPage(p) { return p != null && typeof p.goto === 'function' && typeof p.evaluate === 'function'; }

Try / catch

try { await loadSubstackArchive(page, baseUrl, limit); } catch (e) { if (/Browser session required/.test(e.message)) { page = await openBrowserSession(); await loadSubstackArchive(page, baseUrl, limit); } else throw e; }

Prevention

When it happens

Trigger: Calling loadSubstackArchive with page = null/undefined, e.g. invoking the archive command without a browser session, or automation that never launched a browser.

Common situations: Batch archive scraping scripts run headlessly without session setup, browser launch failures treated as non-fatal earlier in the pipeline, CI environments lacking a browser binary.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/5a540b64b4f49d5d. Report an issue: GitHub.