cube-js/cube · error · Error

Unable to detect number of unloaded records

Error message

Unable to detect number of unloaded records

What it means

After running UNLOAD, the driver parses the 'UNLOAD completed, N records unloaded' notice from the connection to learn how many rows were exported. If the notice text matches the prefix but contains no digits, the driver throws because it cannot determine the unloaded record count.

Source

Thrown at packages/cubejs-redshift-driver/src/RedshiftDriver.ts:464

      await this.prepareConnection(conn, {
        executionTimeout: this.config.executionTimeout ? this.config.executionTimeout * 1000 : 600000,
      });

      let unloadTotalRows: number | null = null;

      /**
       * @link https://github.com/brianc/node-postgres/blob/pg%408.6.0/packages/pg-protocol/src/messages.ts#L211
       * @link https://github.com/brianc/node-postgres/blob/pg%408.6.0/packages/pg-protocol/src/parser.ts#L357
       *
       * message: 'UNLOAD completed, 0 record(s) unloaded successfully.',
       */
      conn.addListener('notice', (e: any) => {
        if (e.message && e.message.startsWith('UNLOAD completed')) {
          const matches = e.message.match(/\d+/);
          if (matches) {
            unloadTotalRows = parseInt(matches[0], 10);
          } else {
            throw new Error('Unable to detect number of unloaded records');
          }
        }
      });

      const baseQuery = `
        UNLOAD ('SELECT ${columns} FROM ${tableName}')
        TO '${bucketType}://${bucketName}/${exportPathName}/'
      `;

      // Prefer the unloadArn if it is present
      const credentialQuery = unloadArn
        ? `iam_role '${unloadArn}'`
        : `CREDENTIALS 'aws_access_key_id=${keyId};aws_secret_access_key=${secretKey}'`;

      const unloadQuery = `${baseQuery} ${credentialQuery} ${optionsPart}`;

      // Unable to extract number of extracted rows, because it's done in protocol notice
      await conn.query({

View on GitHub (pinned to 7d981676b3)

Solutions

1. Inspect the notice message logged by the connection and confirm the driver/Node pg version renders Redshift notices intact
2. Connect directly to Redshift (bypass proxies) so the standard 'UNLOAD completed, N records' notice arrives
3. Upgrade cubejs-redshift-driver / pg client, then retry the export

try {
  return await driver.unload(tableName, options);
} catch (e) {
  if (e.message.startsWith('Unable to detect number of unloaded records')) {
    // verify S3 listing separately or retry; log the raw notice for diagnosis
  }
  throw e;
}

When it happens

Trigger: A 'notice' listener on the pg connection receives a message starting with 'UNLOAD completed' whose text yields no /\d+/ match — e.g. a truncated or non-standard notice message from a proxy or newer Redshift/psg client formatting.

Common situations: Intermediate network layers (pgbouncer-like proxies, custom pg wrappers) rewriting notices; unusual Redshift-compatible engines returning differently worded messages.

Related errors

• Unload is not configured
• Unable to UNLOAD table, there are no files in S3 storage
• Export bucket is not configured.
• Unable to retrieve list of files from S3 storage after unloa
• No CSV files were obtained from the bucket

AI-assisted analysis of cube-js/cube@7d981676b3 (2026-09-02). Data as JSON: /api/errors/09806dca2d596cff. Report an issue: GitHub.