What causes "The model returned too much text during the provider test." in amruthpillai/reactive-resume?

Use a model that follows concise-response instructions; instruct-tuned chat models behave best.

amruthpillai/reactive-resume · warning · Error

The model returned too much text during the provider test.

Error message

The model returned too much text during the provider test.

What it means

Plain Error (not an ORPCError) thrown by testConnection when the model's finishReason is 'length' during the provider connectivity test. The test asks the model to respond with a single character; if it hit the maxOutputTokens cap (TEST_CONNECTION_MAX_OUTPUT_TOKENS) before completing, the model is considered misbehaving (verbose/looping) and the test fails with this message. Because it is a plain Error, callers that do not remap it will see it bubble up; the ai-providers test handler remaps it to BAD_GATEWAY 'Could not reach the AI provider.'.

Source

Thrown at packages/api/src/features/ai/service.ts:158

export const fileInputSchema = z.object({
	name: z.string(),
	data: z.string().max(MAX_AI_FILE_BASE64_CHARS, "File is too large. Maximum size is 10MB."),
});

type TestConnectionInput = z.infer<typeof aiCredentialsSchema>;

export async function testConnection(input: TestConnectionInput): Promise<boolean> {
	const RESPONSE_OK = "1";

	const result = await generateText({
		model: getModel(input),
		maxOutputTokens: TEST_CONNECTION_MAX_OUTPUT_TOKENS,
		temperature: 0,
		messages: [{ role: "user", content: `Respond only with the single character: ${RESPONSE_OK}` }],
	});

	if (result.text.trim() === RESPONSE_OK) return true;
	if (result.finishReason === "length") throw new Error("The model returned too much text during the provider test.");

	return false;
}

type ParsePdfInput = z.infer<typeof aiCredentialsSchema> & {
	file: z.infer<typeof fileInputSchema>;
};

type BuildResumeParsingMessagesInput = {
	userPrompt: string;
	file: z.infer<typeof fileInputSchema>;
	mediaType: string;
};

function buildResumeParsingSystemPrompt(systemPrompt: string): string {
	return `${systemPrompt}\n\nIMPORTANT: You must return ONLY raw valid JSON. Do not return markdown, do not return explanations. Just the JSON object. Use the following JSON as a template and fill in the extracted values. For arrays, you MUST use the exact key names shown in the template (e.g. use 'description' instead of 'summary', 'website' instead of 'url'):\n\n${JSON.stringify(aiExtractionTemplate, null, 2)}`;
}

View on GitHub (pinned to 3a5b12e2a4)

Solutions

Use a model that follows concise-response instructions; instruct-tuned chat models behave best.
If self-hosting, raise TEST_CONNECTION_MAX_OUTPUT_TOKENS or tune the model's system prompt to be terse.
Confirm the baseURL points to the chat/completions endpoint and not a verbose logging surface.
Switch to a better-quantized or larger model variant that respects output constraints.

Example fix

// before
// local model loops, hits token cap
model: 'ollama:llama2-uncensored'

// after
model: 'ollama:llama3.1-instruct' // follows 'respond with 1' instruction

Defensive patterns

Strategy: retry

Validate before calling

async function probeModelTerse(input) {
  const r = await generateText({ model: getModel(input), maxOutputTokens: 16, temperature: 0, messages: [{ role: 'user', content: 'Respond only with: 1' }] });
  return r.finishReason !== 'length';
}

Type guard

function isLengthFinish(error) {
  return error instanceof Error && /too much text during the provider test/i.test(error.message);
}

Try / catch

try {
  await testProvider({ id });
} catch (e) {
  if (e?.code === 'BAD_GATEWAY' || /too much text during the provider test/i.test(e.message)) {
    showToast('The model is too verbose for the connectivity test; try a different model.');
  } else throw e;
}

Prevention

Prefer instruct-tuned models that obey concise-response instructions.
For self-hosted models, raise the test token cap or tune the system prompt.
Do not use models known to emit long preambles for the provider test.

When it happens

Trigger: POST /ai-providers/{id}/test (or a direct testConnection call) where the model ignores the 'respond with 1' instruction and emits text until the token cap, setting finishReason='length'. Common with poorly-instructed local models or models that prepend long preambles.

Common situations: Self-hosted/Ollama model that does not follow short-response instructions; a model that outputs chain-of-thought despite temperature 0; an overly small maxOutputTokens constant; a misconfigured gateway that injects verbose logging into the response.

Related errors

AI-assisted analysis of amruthpillai/reactive-resume@3a5b12e2a4 (2026-08-12). Data as JSON: /api/errors/e05a537acb83bcea. Report an issue: GitHub.