santifer/career-ops · error · Error

arbeitnow: invalid URL

Error message

arbeitnow: invalid URL: ${url}

What it means

assertArbeitnowUrl first checks that the URL parses with the URL constructor; if new URL(url) throws (malformed URL), this error is thrown with the offending string. It's the initial sanity gate before the HTTPS and trusted-host checks on every URL the arbeitnow provider fetches.

Solutions

  1. Log/print the exact url string in the message and fix its source — usually an undefined/empty base variable or a relative path missing the https://www.arbeitnow.com origin.
  2. Use the provider's own FEED_BASE ('https://www.arbeitnow.com/api/job-board-api') when building page URLs instead of hand-rolling strings.
  3. Validate configured URLs at startup (new URL(entry.url)) so a bad portals.yml entry fails fast with a clear config error.
  4. If testing, always pass absolute URLs to assertArbeitnowUrl.

Example fix

// before
const url = `${base}?page=${page}`; // base undefined -> "undefined?page=1"
assertArbeitnowUrl(url); // throws: invalid URL
// after
const base = 'https://www.arbeitnow.com/api/job-board-api';
assertArbeitnowUrl(`${base}?page=${page}`);
Defensive patterns

Strategy: validation

Validate before calling

let parsed;
try { parsed = new URL(entryUrl); } catch { throw new Error(`portal entry has a malformed URL: ${JSON.stringify(entryUrl)}`); }
if (!parsed.hostname) throw new Error('portal entry URL has no host');

Type guard

function isAbsoluteHttpUrl(s) {
  if (typeof s !== 'string') return false;
  try { const u = new URL(s); return u.protocol === 'http:' || u.protocol === 'https:'; } catch { return false; }
}

Try / catch

try {
  await provider.fetch(entry, ctx);
} catch (err) {
  if (String(err.message).includes('invalid URL')) {
    console.error(`Bad URL for entry "${entry.name}": check that the base is a full absolute https:// URL and no interpolated variable was undefined`);
  }
  throw err;
}

Prevention

When it happens

Trigger: new URL(url) throws for input like an empty string, a relative path ('api/job-board-api?page=1' without origin), whitespace/control characters, or a template literal that interpolated undefined ('https://www.arbeitnow.com/api/job-board-api?page=undefined' parses fine — so this fires mainly for missing origin, null/undefined coerced to 'null'/'undefined', or garbage config values).

Common situations: Config/portal entry with a missing or mistyped base URL; a variable holding the URL is undefined and string-interpolates into a broken value; code refactor accidentally passes a path instead of an absolute URL; tests passing relative URLs.

Understand the failure class

Background: "Invalid URL" errors: why new URL(), URI.parse, and reqwest::Url reject your string — missing scheme, whitespace, and bad path format — this error's family across 39 libraries.

Related errors

AI-assisted analysis of santifer/career-ops@aac998c7ed (2026-09-16). Data as JSON: /api/errors/fd22a60af5b0b5cb. Report an issue: GitHub.

Appendix: source

Thrown at providers/arbeitnow.mjs:30

// scan.mjs's title_filter then gates on the configured titles. Pages are fetched
// until one comes back short/empty or the page cap is reached (default 3,
// override with `max_pages` on the portal entry).
//
// Wire in via a `job_boards:` entry with `provider: arbeitnow`.

const FEED_BASE = 'https://www.arbeitnow.com/api/job-board-api';
const TRUSTED_HOST = 'www.arbeitnow.com';
const PER_PAGE = 100;
const DEFAULT_MAX_PAGES = 3;
const MAX_PAGES_CAP = 50;

/** @param {string} url */
function assertArbeitnowUrl(url) {
  let parsed;
  try {
    parsed = new URL(url);
  } catch {
    throw new Error(`arbeitnow: invalid URL: ${url}`);
  }
  if (parsed.protocol !== 'https:') throw new Error(`arbeitnow: URL must use HTTPS: ${url}`);
  if (parsed.hostname !== TRUSTED_HOST) {
    throw new Error(`arbeitnow: untrusted hostname "${parsed.hostname}" — must be ${TRUSTED_HOST}`);
  }
  return url;
}

/** Resolve the page cap: a positive integer `max_pages` on the entry, capped. */
function resolveMaxPages(entry) {
  const v = entry?.max_pages;
  if (Number.isInteger(v) && v > 0) return Math.min(v, MAX_PAGES_CAP);
  return DEFAULT_MAX_PAGES;
}

/**
 * Normalize a single Arbeitnow job. Exported for unit tests.
 *

View on GitHub (pinned to aac998c7ed)