santifer/career-ops · error · Error

4dayweek: unexpected API response on page ${page} — expected

Error message

4dayweek: unexpected API response on page ${page} — expected { jobs: [...] }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]

What it means

In the 4dayweek provider's paginated fetch loop, each page's JSON from ctx.fetchJson() must contain a jobs array. When the response is null/non-object or lacks jobs (e.g. the API returned an error envelope, HTML, or a rate-limit body), the provider throws with the page number and the actual top-level keys so you can see what came back instead.

Source

Thrown at providers/4dayweek.mjs:145

/** @type {Provider} */
export default {
  id: '4dayweek',

  detect: detectFourDayEntry,

  async fetch(entry, ctx) {
    assertFourDayUrl(FEED_BASE);
    const maxPages = resolveMaxPages(entry);
    const fallbackCompany = entry?.name;
    const out = [];

    for (let page = 1; page <= maxPages; page++) {
      const url = `${FEED_BASE}?page=${page}`;
      // redirect:'error' prevents SSRF via server-side redirects
      const json = await ctx.fetchJson(url, { redirect: 'error' });
      if (!json || !Array.isArray(json.jobs)) {
        throw new Error(
          `4dayweek: unexpected API response on page ${page} — expected { jobs: [...] }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]`,
        );
      }
      for (const j of json.jobs) {
        const normalized = normalize4dwJob(j, fallbackCompany);
        if (normalized) out.push(normalized);
      }
      if (json.has_more === false) break; // last page per the API flag
      if (json.jobs.length < PER_PAGE) break; // short page → last page
    }
    return out;
  },
};

View on GitHub (pinned to 1696bec4d0)

Solutions

Log the full response body for the failing page (the message lists the keys) to identify what the API actually returned.
Retry the failing page — transient blocks/rate limits often present as malformed bodies; add backoff.
Lower max_pages on the provider entry so pagination stops at the real last page instead of over-fetching into error pages.
Check whether 4dayweek changed its API shape and update normalize4dwJob/expectations accordingly.
If a CDN/WAF is intercepting, send a proper User-Agent or fetch from an allowed network.

Example fix

// before (over-fetching past the end)
// { "max_pages": 20 } while API only has 3 pages -> page 4 returns {}
// after
// { "max_pages": 3 }  or handle empty last page:
if (!json || !Array.isArray(json.jobs)) {
  if (json && Object.keys(json).length === 0) break; // empty page = end of results
  throw new Error(`4dayweek: unexpected API response on page ${page}`);
}

Defensive patterns

Strategy: retry

Validate before calling

function looksLikeJobsPage(json) {
  return json !== null && typeof json === 'object' && Array.isArray(json.jobs);
}
// after fetchJson:
if (!looksLikeJobsPage(json)) {
  if (json && Object.keys(json).length === 0) return out; // empty last page
  throw new Error(`4dayweek page ${page}: bad shape, keys=${Object.keys(json ?? {})}`);
}

Type guard

function isJobsPage(v) {
  return typeof v === 'object' && v !== null && Array.isArray(v.jobs);
}

Try / catch

try {
  json = await ctx.fetchJson(url, { redirect: 'error' });
} catch (err) {
  if (isJobsPage(err)) throw err;
  if (attempt < 3) { await sleep(2 ** attempt * 1000); continue; } // backoff retry
  throw err;
}

Prevention

Set max_pages no higher than the real page count so pagination never over-fetches into error pages.
Treat an empty object on the first page of a batch as end-of-results rather than an error.
Add exponential backoff on malformed pages — they're often transient WAF/rate-limit bodies.
Monitor for 4dayweek API schema changes with a periodic smoke test asserting { jobs: [] } shape.

When it happens

Trigger: fetching https://4dayweek.io feed with ?page=N where the API returns an unexpected shape: an error object {error: ...}, a rate-limit/throttle body, an empty page beyond the last (some APIs return {} instead of {jobs: []}), a Cloudflare HTML interstitial that fetchJson parsed loosely, or an API schema change.

Common situations: Hitting past the last page while resolveMaxPages() allowed more pages; the 4dayweek API being temporarily down or changed its response schema; a WAF/CDN blocking the client with a non-Job JSON/HTML body; network middleware returning a JSON error envelope.

Understand the failure class

Background: "invalid response format", "malformed payload", "missing data field": when an API returns 200 but the response shape is wrong — this error's family across 23 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@1696bec4d0 (2026-09-01). Data as JSON: /api/errors/49bec4804fc3093f. Report an issue: GitHub.