ruvnet/ruflo · critical · Error

Failed to load any models from upstream

Error message

Failed to load any models from upstream

What it means

Thrown by applyModelState when buildModels returned an empty array. applyModelState is the single guard that prevents the global models array from being replaced with []; if the freshly-built list is empty, it refuses to swap state and throws. Empty builds happen when the upstream /models response yields zero usable entries, or when MODELS overrides filter the upstream list down to nothing.

Source

Thrown at ruflo/src/ruvocal/src/lib/server/models.ts:246

		prepromptUrl: model.prepromptUrl,
		endpoints:
			model.endpoints?.map((endpoint) => {
				if (endpoint.type === "openai") {
					const { type, baseURL } = endpoint;
					return { type, baseURL };
				}
				return { type: endpoint.type };
			}) ?? null,
		multimodal: model.multimodal,
		multimodalAcceptedMimetypes: model.multimodalAcceptedMimetypes,
		supportsTools: (model as unknown as { supportsTools?: boolean }).supportsTools ?? false,
		isRouter: model.isRouter,
		hasInferenceAPI: model.hasInferenceAPI,
	});

const applyModelState = (newModels: ProcessedModel[], startedAt: number): ModelsRefreshSummary => {
	if (newModels.length === 0) {
		throw new Error("Failed to load any models from upstream");
	}

	const previousIds = new Set(models.map((m) => m.id));
	const previousSignatures = new Map(models.map((m) => [m.id, signatureForModel(m)]));
	const refreshedAt = new Date();
	const durationMs = Date.now() - startedAt;

	models = newModels;
	defaultModel = models[0];
	taskModel = resolveTaskModel(models);
	validModelIdSchema = createValidModelIdSchema(models);
	lastModelRefresh = refreshedAt;
	lastModelRefreshDurationMs = durationMs;

	const added = newModels.map((m) => m.id).filter((id) => !previousIds.has(id));
	const removed = Array.from(previousIds).filter(
		(id) => !newModels.some((model) => model.id === id)
	);

View on GitHub (pinned to 6b01dc5a68)

Solutions

1. Confirm the upstream /models endpoint actually returns models: curl -H "Authorization: Bearer $TOKEN" $OPENAI_BASE_URL/models.
2. If using MODELS overrides, ensure at least one override id matches a real upstream model id, or remove overrides temporarily to load all upstream models.
3. Point OPENAI_BASE_URL at a known-good OpenAI-compatible router (e.g. https://router.huggingface.co/v1) and verify a non-empty data array.
4. Check logs for the '[models] Parsed models count' line to see what buildModels actually received.

Example fix

// before
OPENAI_BASE_URL=https://internal-gateway/v1
MODELS='[{"id":"retired-model"}]'

// after
OPENAI_BASE_URL=https://router.huggingface.co/v1
# remove or update MODELS to include a currently-available model

async function canFetchModels(): Promise<{ ok: boolean; reason?: string }> {
  if (!config.OPENAI_BASE_URL) return { ok: false, reason: "missing-base-url" };
  try {
    const r = await fetch(`${config.OPENAI_BASE_URL.replace(/\/$/, "")}/models`, {
      headers: config.OPENAI_API_KEY ? { Authorization: `Bearer ${config.OPENAI_API_KEY}` } : undefined,
    });
    if (!r.ok) return { ok: false, reason: `http-${r.status}` };
    const j = await r.json();
    return { ok: Array.isArray(j?.data) && j.data.length > 0, reason: j?.data?.length ? undefined : "empty-data" };
  } catch (e) {
    return { ok: false, reason: String(e) };
  }
}

function isNonEmptyModelList(list: unknown): list is { id: string }[] {
  return Array.isArray(list) && list.length > 0 && list.every((m) => m && typeof m.id === "string");
}

try { await rebuildModels(); }
catch (e) {
  if (e instanceof Error && /Failed to load any models from upstream/.test(e.message)) {
    // keep previous models if present, alert operator, do not crash on refresh
    if (models.length === 0) process.exit(1);
  } else throw e;
}

When it happens

Trigger: OPENAI_BASE_URL points at an OpenAI-compatible service whose /models returns {data: []}; MODELS overrides all reference model ids that do not exist upstream AND filteredAndOrdered ends up empty (note: when no override matches, buildModels falls back to applying overrides on top of all upstream models, so a truly empty result usually means the upstream list itself was empty); listSchema parses but parsed.data is [].

Common situations: Fresh gateway deployment with no published models; wrong base URL (e.g. pointing at an auth or management endpoint that returns an empty data array); upstream temporarily returning empty during maintenance; MODELS override that references only retired/renamed model ids in a config branch that also disables the fallback.

Related errors

• OPENAI_BASE_URL not set
• Failed to fetch ${baseURL}/models: ${response.status} ${resp
• No endpoints configured. This build requires OpenAI-compatib
• Only 'openai' endpoint type is supported in this build
• No models available to build validation schema

AI-assisted analysis of ruvnet/ruflo@6b01dc5a68 (2026-08-12). Data as JSON: /api/errors/998d1541c8ec47cf. Report an issue: GitHub.