prestodb/presto · warning · BigQueryException

BIGQUERY_TABLE_DISAPPEAR_DURING_LIST

BIGQUERY_TABLE_DISAPPEAR_DURING_LIST

Error message

Table disappeared during listing operation

What it means

Thrown by BigQueryMetadata.listTableColumns when a table that was just returned by listTables no longer exists by the time getTableMetadata fetches its schema, surfacing the BigQuery NotFoundException as BIGQUERY_TABLE_DISAPPEAR_DURING_LIST. It reflects a race between listing tables in a schema and reading each table's metadata: an external actor (or a stale/permission-limited view) removed or hid the table mid-iteration.

Source

Thrown at presto-bigquery/src/main/java/com/facebook/presto/plugin/bigquery/BigQueryMetadata.java:222

            ConnectorTableHandle tableHandle,
            ColumnHandle columnHandle)
    {
        log.debug("getColumnMetadata(session=%s, tableHandle=%s, columnHandle=%s)", session, columnHandle, columnHandle);
        return ((BigQueryColumnHandle) columnHandle).getColumnMetadata();
    }

    @Override
    public Map<SchemaTableName, List<ColumnMetadata>> listTableColumns(ConnectorSession session, SchemaTablePrefix prefix)
    {
        log.debug("listTableColumns(session=%s, prefix=%s)", session, prefix);
        requireNonNull(prefix, "prefix is null");
        ImmutableMap.Builder<SchemaTableName, List<ColumnMetadata>> columns = ImmutableMap.builder();
        for (SchemaTableName tableName : listTables(session, prefix)) {
            try {
                columns.put(tableName, getTableMetadata(session, tableName).getColumns());
            }
            catch (NotFoundException ex) {
                throw new BigQueryException(BIGQUERY_TABLE_DISAPPEAR_DURING_LIST, "Table disappeared during listing operation", ex);
            }
        }
        return columns.build();
    }

    private List<SchemaTableName> listTables(ConnectorSession session, SchemaTablePrefix prefix)
    {
        if (prefix.getTableName() == null) {
            return listTables(session, Optional.ofNullable(prefix.getSchemaName()));
        }
        SchemaTableName tableName = prefix.toSchemaTableName();
        Optional<TableInfo> tableInfo = getBigQueryTable(tableName);
        return tableInfo.isPresent() ?
                ImmutableList.of(tableName) :
                ImmutableList.of(); // table does not exist
    }

    @Override

View on GitHub (pinned to 55bb57d202)

Solutions

1. Re-run the listing query; transient races usually succeed on retry
2. Confirm no concurrent job is dropping/renaming tables in the target dataset during the query
3. Check IAM permissions on the dataset — missing permissions can surface as 404 NotFound
4. Pin the listing to specific table names instead of a schema-wide prefix to shrink the race window
5. If frequent, add retry-on-NotFound handling around listTableColumns in the connector or skip missing tables

Example fix

// before: whole-schema listing races with DROP TABLE
SELECT * FROM information_schema.columns WHERE table_schema = 'logs';
// after: target the specific surviving tables
SELECT * FROM information_schema.columns WHERE table_schema = 'logs' AND table_name = 'events';

for (SchemaTableName t : expectedTables) {
    // pre-check existence before expensive listing
    try (Table t2 = bigQuery.getTable(tableId(t))) {
        if (t2 == null) skip(t);
    }
}

try {
    columns = metadata.listTableColumns(session, prefix);
} catch (BigQueryException e) {
    if (e.getCode() == BIGQUERY_TABLE_DISAPPEAR_DURING_LIST && attempt < 3) {
        backoff(attempt); // 100ms * 2^attempt
        continue;
    }
    throw e;
}

When it happens

Trigger: During a listing operation with a SchemaTablePrefix (e.g. SHOW COLUMNS / metadata queries over all tables), getTableMetadata(session, tableName) throws com.google.api.gax.httpjson NotFoundException (HTTP 404) for one of the tables returned moments earlier by listTables.

Common situations: Concurrent DROP TABLE in the same schema while Presto lists columns; a视图 or partition-time tables being recreated; eventual consistency after dataset/table churn; IAM changes causing a 404-masquerading permission failure; hourly/daily table drops by scheduled jobs.

Related errors

• NOT_FOUND
• NOT_FOUND
• NOT_FOUND
• ICEBERG_INVALID_METADATA
• NOT_FOUND

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/6a300fb191f41f81. Report an issue: GitHub.