cube-js/cube · error

The maximum number of source rows ${this.options.maxSourceRo

Error message

The maximum number of source rows ${this.options.maxSourceRowLimit} was reached for ${this.preAggregation.preAggregationId}

What it means

When building a lambda union pre-aggregation, Cube downloads the source (base) table as CSV (useCsvQuery) and enforces a hard cap on the number of source rows via options.maxSourceRowLimit. downloadLambdaTable throws when the fetched rowCount exactly equals this limit, indicating the source data likely exceeded the cap and the result may be truncated, so it aborts rather than silently building an incomplete rollup.

Source

Thrown at packages/cubejs-query-orchestrator/src/orchestrator/PreAggregationPartitionRangeLoader.ts:404

    const { data } = await this.queryCache.renewQuery(
      query,
      <string[]>values,
      cacheKeyQueries,
      60 * 60,
      [query, <string[]>values],
      undefined,
      {
        requestId: this.requestId,
        skipRefreshKeyWaitForRenew: false,
        priority: this.priority(QueuePriority.Interactive),
        dataSource: this.dataSource,
        external: false,
        useCsvQuery: true,
        lambdaTypes,
      }
    );
    if (data.rowCount === this.options.maxSourceRowLimit) {
      throw new Error(`The maximum number of source rows ${this.options.maxSourceRowLimit} was reached for ${this.preAggregation.preAggregationId}`);
    }
    return {
      name: `${LAMBDA_TABLE_PREFIX}_${this.preAggregation.tableName.replace('.', '_')}`,
      columns: data.types,
      csvRows: data.csvRows,
    };
  }

  public async partitionPreAggregations(): Promise<PreAggregationDescription[]> {
    if (this.preAggregation.partitionGranularity && !this.preAggregation.expandedPartition) {
      const { buildRange, partitionRanges } = await this.partitionRanges();
      return this.compilerCacheFn(['partitions', JSON.stringify(buildRange)], () => partitionRanges.map(range => this.partitionPreAggregationDescription(range, buildRange)));
    } else {
      return [this.preAggregation];
    }
  }

  private async partitionRanges(ignoreMatchedDateRange?: boolean): Promise<PartitionRanges> {

View on GitHub (pinned to 7d981676b3)

Solutions

1. Increase maxSourceRowLimit in preAggregations options to a value above your source row count (with attention to memory).
2. Add a filter/partition to the base pre-aggregation so the source data for the lambda union is smaller.
3. Restructure to a two-stage (chained) pre-aggregation: build a base pre-aggregation first, then roll up from it instead of raw rows.
4. If the limit is being hit coincidentally (rowCount exactly equals limit), add a sanity filter and re-run.

Example fix

// before
preAggregationsOptions: { }
// after
preAggregationsOptions: { maxSourceRowLimit: 20000 }

const rowCount = await sourceDriver.query(`SELECT COUNT(*) AS c FROM ${baseTable}`);
if (rowCount[0].c >= maxSourceRowLimit) {
  throw new Error(`Source rows (${rowCount[0].c}) exceed maxSourceRowLimit (${maxSourceRowLimit})`);
}

try {
  return await loader.loadPreAggregations();
} catch (e) {
  if (String(e.message).includes('maximum number of source rows')) {
    // raise maxSourceRowLimit or filter/partition the base data
  }
  throw e;
}

When it happens

Trigger: Lambda union build where the base pre-aggregation/source query returns rowCount === maxSourceRowLimit; maxSourceRowLimit set too low for the dataset (default is small); unpartitioned lambda union over a very large source table.

Common situations: Large source tables rolled up via rollup_lambda without pre-filtering; deployments where maxSourceRowLimit was left at the default; accidental lambda (multi-stage) rollup over millions of rows instead of chaining pre-aggregations.

Related errors

• Lambda union with source data feature is supported only by e
• Unable to detect column types for pre-aggregation on empty v
• Create table failed: ${e}
• Unsupported table data passed to ${this.constructor}
• Unable to import (as rows) in Cube Store: empty columns. Mos

AI-assisted analysis of cube-js/cube@7d981676b3 (2026-09-02). Data as JSON: /api/errors/e2770ecc030c9539. Report an issue: GitHub.