cube-js/cube · error

Lambda union with source data feature is supported only by e

Error message

Lambda union with source data feature is supported only by external rollups stored in Cube Store but was invoked for '${preAggregation.preAggregationId}'

What it means

Cube's lambda (multi-stage) pre-aggregations that union over source data require the resulting rollup to be stored in Cube Store (an external store). getTableColumnTypes throws this when it is asked for column types of a table marked external=true while the skipExternalCacheAndQueue option is set, meaning the code path assumed a non-Cube-Store external rollup. It protects an internal invariant: source-data lambda unions are only valid for external rollups physically stored in Cube Store.

Source

Thrown at packages/cubejs-query-orchestrator/src/orchestrator/PreAggregationLoadCache.ts:130

  protected async getTablesQuery(preAggregation) {
    const redisKey = this.tablesCachePrefixKey(preAggregation);
    if (!this.tables[redisKey]) {
      const tables = this.preAggregations.options.skipExternalCacheAndQueue && preAggregation.external ?
        await this.fetchTablesNoCache(preAggregation) :
        await this.tablesFromCache(preAggregation);
      if (tables === undefined) {
        throw new Error('Pre-aggregation tables are undefined.');
      }
      this.tables[redisKey] = tables;
    }
    return this.tables[redisKey];
  }

  public async getTableColumnTypes(preAggregation: PreAggregationDescription, tableName: string): Promise<TableStructure> {
    const prefixKey = this.tablesCachePrefixKey(preAggregation);
    if (!this.tableColumnTypes[prefixKey]?.[tableName]) {
      if (!this.preAggregations.options.skipExternalCacheAndQueue && preAggregation.external) {
        throw new Error(`Lambda union with source data feature is supported only by external rollups stored in Cube Store but was invoked for '${preAggregation.preAggregationId}'`);
      }
      const client = await this.externalDriverFactory();
      const columnTypes = await client.tableColumnTypes(tableName);
      if (!this.tableColumnTypes[prefixKey]) {
        this.tableColumnTypes[prefixKey] = {};
      }
      this.tableColumnTypes[prefixKey][tableName] = columnTypes;
    }
    return this.tableColumnTypes[prefixKey][tableName];
  }

  private async calculateVersionEntries(preAggregation): Promise<VersionEntriesObj> {
    let versionEntries = tablesToVersionEntries(
      preAggregation.preAggregationsSchema,
      await this.getTablesQuery(preAggregation)
    );
    // It presumes strong consistency guarantees for external pre-aggregation tables ingestion
    if (!preAggregation.external) {

View on GitHub (pinned to 7d981676b3)

Solutions

  1. Store the rollup in Cube Store: make sure the lambda union pre-aggregation is built as an external rollup (external: true in the rollup config and a Cube Store external store configured).
  2. Remove or unset the skipExternalCacheAndQueue option so external cache and queue are used.
  3. Restructure the pre-aggregation so lambda union source data is not required, or build the base pre-aggregation first (pre-aggregation chaining) instead of reading source data.
  4. Check the rollup definition: source data lambda unions require a partitioned base pre-aggregation; define it and reference it instead of raw source data.

Example fix

// before
options: { skipExternalCacheAndQueue: true }
// after
options: { skipExternalCacheAndQueue: false } // rollup stored in Cube Store
Defensive patterns

Strategy: validation

Validate before calling

const opts = preAggregationsOptions || {};
if (rollupConfig.rollup_lambda && rollupConfig.external !== false && opts.skipExternalCacheAndQueue) {
  throw new Error('Lambda union with source data requires Cube Store external rollups; unset skipExternalCacheAndQueue');
}

Try / catch

try {
  await loadCache.getTableColumnTypes(preAggregation, tableName);
} catch (e) {
  if (String(e.message).includes('Lambda union with source data')) {
    // fall back to building in Cube Store or restructure rollup
  }
  throw e;
}

Prevention

When it happens

Trigger: Calling getTableColumnTypes for a preAggregation with external=true when this.preAggregations.options.skipExternalCacheAndQueue is truthy; configuring a lambda union pre-aggregation that rolls up to a non-Cube-Store external store while external caching/queue is skipped.

Common situations: Running a query whose pre-aggregation chain uses source data with a custom external driver configuration; test setups or options that set skipExternalCacheAndQueue; misconfigured rollups where externalFrom/external store does not point at Cube Store.

Related errors


AI-assisted analysis of cube-js/cube@7d981676b3 (2026-09-02). Data as JSON: /api/errors/468e7f099b0848ff. Report an issue: GitHub.