mastra-ai/mastra · error · MastraError

COMPARE_INVALID_INPUT

COMPARE_INVALID_INPUT

Error message

compareExperiments requires at least 2 experiment IDs.

What it means

MastraStorage's compareExperiments() compares run results across at least two experiment IDs and enriches the comparison with per-item input/groundTruth/output data. The library throws COMPARE_INVALID_INPUT when fewer than 2 experiment IDs are passed, since a comparison needs at least two experiments to produce rows. This is a user-input error, thrown before any storage access.

Source

Thrown at packages/core/src/datasets/manager.ts:249

  async getExperiment(args: { experimentId: string; organizationId?: string; projectId?: string }) {
    const experimentsStore = await this.#getExperimentsStore();
    return experimentsStore.getExperimentById({
      id: args.experimentId,
      filters: scopeFromArgs(args),
    });
  }

  /**
   * Compare two or more experiments.
   *
   * Uses the internal `compareExperiments` function for pairwise comparison,
   * then enriches results with per-item input/groundTruth/output data.
   */
  async compareExperiments(args: { experimentIds: string[]; baselineId?: string }) {
    const { experimentIds, baselineId } = args;

    if (experimentIds.length < 2) {
      throw new MastraError({
        id: 'COMPARE_INVALID_INPUT',
        text: 'compareExperiments requires at least 2 experiment IDs.',
        domain: 'STORAGE',
        category: 'USER',
      });
    }

    const resolvedBaseline = baselineId ?? experimentIds[0]!;
    const otherExperimentId = experimentIds.find(id => id !== resolvedBaseline) ?? experimentIds[1]!;

    const internal = await compareExperimentsInternal(this.#mastra, {
      experimentIdA: resolvedBaseline,
      experimentIdB: otherExperimentId,
    });

    // Load results for both runs to get input/groundTruth/output
    const experimentsStore = await this.#getExperimentsStore();
    const [resultsA, resultsB] = await Promise.all([

View on GitHub (pinned to 75dd419e61)

Solutions

1. Ensure the caller collects and passes at least 2 distinct experiment IDs before invoking compareExperiments.
2. Guard the call site: check `experimentIds.length >= 2` and short-circuit with a friendly message otherwise.
3. If IDs are derived dynamically, validate the filtered list length before calling and surface a 'select more experiments' state in the UI.

Example fix

// before
const result = await storage.compareExperiments({ experimentIds: [selected[0]] });
// after
if (selected.length < 2) throw new Error('Select at least 2 experiments to compare');
const result = await storage.compareExperiments({ experimentIds: selected });

if (!Array.isArray(ids) || ids.length < 2) {
  throw new TypeError('compareExperiments requires at least 2 experiment IDs');
}
await storage.compareExperiments({ experimentIds: ids });

function hasAtLeastTwoIds(ids: unknown): ids is [string, string, ...string[]] {
  return Array.isArray(ids) && ids.length >= 2 && ids.every((id): id is string => typeof id === 'string');
}

try {
  const result = await storage.compareExperiments({ experimentIds: ids });
} catch (e) {
  if (e instanceof MastraError && e.id === 'COMPARE_INVALID_INPUT') {
    return { error: 'Please select at least 2 experiments to compare.' };
  }
  throw e;
}

When it happens

Trigger: Calling compareExperiments({ experimentIds }) with an empty array, a single-element array, or an array that was dynamically filtered down to fewer than 2 valid IDs.

Common situations: Building a UI that compares experiments but letting the user click 'Compare' with only one experiment selected; deriving experiment IDs from query results where most rows were filtered out; a fresh project where only one experiment has run yet.

Related errors

• DATASET_INVALID_ID
• SchemaUpdateValidationError
• DATASET_ITEM_EXTERNAL_ID_INVALID
• threadId must be a non-empty string or array of non-empty st
• page must be >= 0

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/bd581b722ee286d1. Report an issue: GitHub.