firecrawl/firecrawl · error

Schema $ref resolution limit exceeded

Error message

Schema $ref resolution limit exceeded

What it means

dereferenceSchema walks the JSON schema and inlines every $ref it can resolve against the document root, counting each resolution. A hard cap of MAX_REF_RESOLUTIONS (50,000) protects against pathologically large or exponential-blowup schemas. Once the counter exceeds the cap the walk aborts with this error rather than consuming unbounded memory/CPU.

Source

Thrown at apps/api/src/lib/extract/helpers/dereference-schema.ts:56

  function walk(node: any, activeRefs: Set<string>): any {
    if (Array.isArray(node)) {
      return node.map(item => walk(item, activeRefs));
    }
    if (node === null || typeof node !== "object") {
      return node;
    }
    if (isRefObject(node)) {
      const ref = node.$ref;
      // External ref, cycle, or unresolvable pointer: leave the node as-is.
      if (!ref.startsWith("#") || activeRefs.has(ref)) {
        return { ...node };
      }
      const target = resolveJsonPointer(root, ref);
      if (target === undefined) {
        return { ...node };
      }
      if (++resolutions > MAX_REF_RESOLUTIONS) {
        throw new Error("Schema $ref resolution limit exceeded");
      }
      const nextActive = new Set(activeRefs);
      nextActive.add(ref);
      return walk(target, nextActive);
    }
    const result: Record<string, any> = {};
    for (const key of Object.keys(node)) {
      result[key] = walk(node[key], activeRefs);
    }
    return result;
  }

  try {
    return walk(root, new Set());
  } catch (error) {
    console.error("Failed to dereference schema:", error);
    throw error;
  }

View on GitHub (pinned to 656bffcc28)

Solutions

1. Simplify the schema: inline the most-referenced definitions manually and remove unused branches before passing it to extraction.
2. Run dereferenceSchema in isolation on the suspect schema and log the resolution count to find the offending sub-tree.
3. Split a monolithic schema into several smaller extract calls.
4. If the cap is genuinely too low for a legitimate schema, raise MAX_REF_RESOLUTIONS deliberately (it is a module-level const) after confirming memory/CPU headroom.

Example fix

// before
const deref = await dereferenceSchema(hugeOpenApiSchema);

// after
// pre-trim to only the definitions actually referenced by the extract schema
const trimmed = pickDefinitions(hugeOpenApiSchema, usedRefs);
const deref = await dereferenceSchema(trimmed);

function countRefs(schema: any, seen = new Set()): number {
  if (!schema || typeof schema !== "object") return 0;
  let n = 0;
  for (const k of Object.keys(schema)) {
    if (k === "$ref" && typeof schema[k] === "string") n++;
    n += countRefs(schema[k]);
  }
  return n;
}

const refCount = countRefs(schema);
if (refCount > 50_000) {
  throw new Error(`Schema has ${refCount} $ref nodes; simplify before dereference.`);
}

function isResolvableSchema(schema: any): boolean {
  return schema !== null && typeof schema === "object" && !Array.isArray(schema);
}

try {
  const deref = await dereferenceSchema(schema);
} catch (e) {
  if (e instanceof Error && e.message === "Schema $ref resolution limit exceeded") {
    // simplify and retry, or reject the schema as too complex
    throw new Error("Schema too complex to dereference inline; reduce the number of $refs.");
  }
  throw e;
}

When it happens

Trigger: A submitted schema whose $ref graph, once inlined, expands past 50,000 resolutions. Common shapes: deeply nested schemas, many distinct pointers into a large shared definitions block, schemas that fan out combinatorially, or schemas authored with thousands of repeated refs.

Common situations: Large OpenAPI/JSON-Schema documents auto-converted into an extract schema; a user-supplied schema generated by a tool that emits one $ref per field; a schema with deeply recursive type definitions that, while cycle-broken, still expands hugely.

Related errors

• Item schema not found for array key
• Array key not found in schema
• Item schema not found for array key
• Array key not found in schema
• document.markdown is undefined -- this is unexpected

AI-assisted analysis of firecrawl/firecrawl@656bffcc28 (2026-08-12). Data as JSON: /api/errors/df765d9000fc22dc. Report an issue: GitHub.