arbeitnow: unexpected API response on page

Error message

arbeitnow: unexpected API response on page ${page} — expected { data: [...] }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]

What it means

The arbeitnow feed must return { data: [...] } per page. If the response is null, not an object, or `data` is missing/not an array, the provider throws — including the actual top-level keys it saw — because the documented contract changed or a non-API payload (error page, HTML) came back. It treats this as a shape change, not an empty board.

Solutions

Read the got-keys list in the message: an HTML block page means the string starts with '<' — check for WAF/rate limiting and slow down or add delays between page requests.
If keys show an error envelope, read its message (e.g. throttle or auth) and adjust (backoff, API key).
Diff the live response (curl https://www.arbeitnow.com/api/job-board-api?page=1) against the documented { data, links, meta } shape and update the parser if the contract changed.
Retry later if transient — a single bad page mid-run is fatal here, so confirm with a manual rerun before treating it as a code issue.

Example fix

// before: generic retry that hammers the WAF
for (const page of pages) await fetchPage(page);
// after: pace requests under rate limits
for (const page of pages) {
  await fetchPage(page);
  await new Promise(r => setTimeout(r, 1500));
}

Defensive patterns

Strategy: type-guard

Validate before calling

const res = await fetch('https://www.arbeitnow.com/api/job-board-api?page=1');
const ct = res.headers.get('content-type') || '';
if (!ct.includes('application/json')) throw new Error(`arbeitnow returned ${ct} — likely WAF/rate-limit page`);
const json = await res.json();
if (!Array.isArray(json?.data)) throw new Error(`unexpected envelope, keys: ${Object.keys(json || {})}`);

Type guard

function hasArbeitnowPageShape(json) {
  return !!json && typeof json === 'object' && Array.isArray(json.data);
}

Try / catch

try {
  jobs = await provider.fetch(entry, ctx);
} catch (err) {
  if (String(err.message).includes('unexpected API response')) {
    console.error('Envelope check the got-keys list in the message — HTML means bot protection: back off and slow page pacing');
  }
  throw err;
}

Prevention

Pace page requests (delays between pages) to avoid triggering arbeitnow's rate limiting.
Check content-type before JSON.parse so HTML block pages are recognized immediately.
Before long sweeps, probe page 1 to confirm the { data, links, meta } envelope.

When it happens

Trigger: GET https://www.arbeitnow.com/api/job-board-api?page=N returns anything without an array `data` — a CDN/WAF HTML block page, a JSON error envelope ({ message, errors }), rate-limit JSON, or an API version bump renaming data.

Common situations: Rate limiting or bot protection from arbeitnow serving an interstitial during aggressive sweeps; the API changed its envelope; a captive network/proxy intercepts TLS and returns its own page; transient 5xx body parsed as JSON.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/28f2db7f7ffc90b9. Report an issue: GitHub.

Appendix: source

Thrown at providers/arbeitnow.mjs:115

/** @type {Provider} */
export default {
  id: 'arbeitnow',

  async fetch(entry, ctx) {
    assertArbeitnowUrl(FEED_BASE);
    const maxPages = resolveMaxPages(entry);
    const fallbackCompany = entry?.name;
    const out = [];

    for (let page = 1; page <= maxPages; page++) {
      // Build the page URL directly (do NOT follow links.next — it carries a
      // featured `?search=` term that would narrow the board).
      const url = `${FEED_BASE}?page=${page}`;
      // redirect:'error' prevents SSRF via server-side redirects
      const json = await ctx.fetchJson(url, { redirect: 'error' });
      if (!json || !Array.isArray(json.data)) {
        throw new Error(
          `arbeitnow: unexpected API response on page ${page} — expected { data: [...] }, got keys: [${json ? Object.keys(json).join(', ') : 'null'}]`,
        );
      }
      for (const j of json.data) {
        const normalized = normalizeArbeitnowJob(j, fallbackCompany);
        if (normalized) out.push(normalized);
      }
      if (json.data.length < PER_PAGE) break; // short page → last page reached
    }
    return out;
  },
};

View on GitHub (pinned to aac998c7ed)