nexu-io/open-design · error

Could not fetch ${url} — the site may block server-side requ

Error message

Could not fetch ${url} — the site may block server-side requests.

What it means

Thrown by buildFromUrl in the deterministic, no-LLM brand builder. prefetchBrand(url, tmpDir) returned a falsy material, meaning the site could not be fetched, returned no usable logos/content, or the request failed upstream. This path is offline and deterministic, so it does not fall back to an agent.

Source

Thrown at apps/daemon/src/brands/engine/build.ts:420

      radius: `${seed.borderRadius}px`,
      borderWeight: `${seed.lineWidth}px`,
      spacing: "8px baseline grid",
      postureRules: [],
    },
  };
}

/**
 * Build a BrandSystem straight from a site URL — the deterministic, no-LLM
 * path. `prefetchBrand` must write to a brand dir, so we hand it a throwaway
 * temp dir, read back any downloaded logos into the bundle, then assemble.
 */
export async function buildFromUrl(url: string, opts?: { slug?: string }): Promise<BrandSystem> {
  const tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), "brand-prefetch-"));
  try {
    const material = await prefetchBrand(url, tmpDir);
    if (!material) {
      throw new Error(`Could not fetch ${url} — the site may block server-side requests.`);
    }

    const seed = seedFromMaterial(material);
    const brand = brandFromMaterial(material, seed);
    const slug = opts?.slug ? slugify(opts.slug) : slugify(brand.name);

    // Pull any downloaded logos back into the in-memory bundle.
    const extraFiles: Record<string, string> = {};
    for (const logo of material.logos ?? []) {
      const abs = path.join(tmpDir, "logos", logo.file);
      try {
        if (logo.contentType?.includes("svg") || logo.file.endsWith(".svg")) {
          extraFiles[`logos/${logo.file}`] = fs.readFileSync(abs, "utf8");
        } else {
          // Binary assets: keep as base64 data so the bundle stays string-only.
          const b64 = fs.readFileSync(abs).toString("base64");
          extraFiles[`logos/${logo.file}.b64`] = b64;
        }

View on GitHub (pinned to 5be4028344)

Solutions

1. Switch to the agent-driven extraction path (startBrandExtraction) which is more tolerant of JS-heavy and protected sites.
2. From the daemon host, curl the URL to confirm reachability and inspect what the server returns.
3. If you control the site, allowlist the daemon's egress IP or set a recognizable User-Agent.
4. Fall back to pasting a DESIGN.md (startBrandExtraction accepts designMd instead of url).

Example fix

// before: deterministic builder cannot scrape a JS-only site
const brand = await buildFromUrl('https://app.example.com');

// after: use the agent-driven path, or supply design markdown
const result = await startBrandExtraction({ url: 'https://app.example.com', /* deps */ });
// or
const result = await startBrandExtraction({ designMd: '<paste DESIGN.md>', description: '...', /* deps */ });

// Probe reachability before invoking the deterministic builder.
async function isFetchable(url: string): Promise<boolean> {
  try {
    const res = await fetch(url, { redirect: 'follow', headers: { 'user-agent': 'OpenDesign-Brand-Build/1.0' } });
    return res.ok && (await res.text()).length > 0;
  } catch {
    return false;
  }
}
if (!await isFetchable(url)) {
  // skip deterministic build, go straight to agent-driven extraction
}

let brand;
try {
  brand = await buildFromUrl(url);
} catch (err) {
  if (err instanceof Error && err.message.startsWith('Could not fetch ')) {
    // fall back to the agent-driven path which tolerates JS/protected sites
    brand = await startBrandExtraction({ url, /* deps */ }).then(r => r.brand);
  } else {
    throw err;
  }
}

When it happens

Trigger: The target site blocks server-side scrapers (403/429), requires JavaScript to render, has no logos or readable content, returns an empty body, the daemon host cannot reach it (DNS/firewall), or the TLS cert is invalid.

Common situations: Cloudflare/Akamai-protected marketing sites, JS-only SPAs with no SSR HTML, intranet URLs unreachable from the daemon, geo-blocked domains, sites that need cookies/consent.

Related errors

• Enter a valid http(s) website URL.
• brand not found: ${opts.id}
• brand backing project not found: ${projectId}
• brand not found: ${id}
• brand.json not found in the extraction project — the agent h

AI-assisted analysis of nexu-io/open-design@5be4028344 (2026-08-12). Data as JSON: /api/errors/0186bcc243d7ab7b. Report an issue: GitHub.