apache/shardingsphere · error · OverallConnectionNotEnoughException

13000

13000

Error message

Can not get %d connections one time, partition succeed connection(%d) have released. Please consider increasing the 'maxPoolSize' of the data sources or decreasing the 'max-connections-size-per-query' in properties.

What it means

OverallConnectionNotEnoughException (error code 13000) is thrown by JDBCBackendDataSource.createConnections when the proxy tries to acquire `connectionSize` connections from one storage DataSource for a single query and any acquisition fails partway. Every already-obtained connection is closed and the partial failure is rethrown with this message, so the query never runs with fewer connections than the routing plan requires. The root cause is the wrapped SQLException: typically the pool (maxPoolSize) or the backend database itself refusing the Nth simultaneous connection.

Source

Thrown at proxy/backend/core/src/main/java/org/apache/shardingsphere/proxy/backend/connector/jdbc/datasource/JDBCBackendDataSource.java:73

        if (ConnectionMode.CONNECTION_STRICTLY == connectionMode) {
            return createConnections(databaseName, dataSourceName, dataSource, connectionSize);
        }
        synchronized (dataSource) {
            return createConnections(databaseName, dataSourceName, dataSource, connectionSize);
        }
    }
    
    private List<Connection> createConnections(final String databaseName, final String dataSourceName,
                                               final DataSource dataSource, final int connectionSize) throws SQLException {
        List<Connection> result = new ArrayList<>(connectionSize);
        for (int i = 0; i < connectionSize; i++) {
            try {
                result.add(createConnection(databaseName, dataSourceName, dataSource));
            } catch (final SQLException ex) {
                for (Connection each : result) {
                    each.close();
                }
                throw new OverallConnectionNotEnoughException(connectionSize, result.size(), ex);
            }
        }
        return result;
    }
    
    private Connection createConnection(final String databaseName, final String dataSourceName, final DataSource dataSource) throws SQLException {
        TransactionRule transactionRule = ProxyContext.getInstance().getContextManager().getMetaDataContexts().getMetaData().getGlobalRuleMetaData().getSingleRule(TransactionRule.class);
        ShardingSphereDistributedTransactionManager distributedTransactionManager = transactionRule.getResource().getTransactionManager(transactionRule.getDefaultType());
        Connection result = isInDistributedTransaction(distributedTransactionManager) ? distributedTransactionManager.getConnection(databaseName, dataSourceName) : dataSource.getConnection();
        if (dataSourceName.contains(".")) {
            String catalog = dataSourceName.split("\\.")[1];
            result.setCatalog(catalog);
        }
        return result;
    }
    
    private boolean isInDistributedTransaction(final ShardingSphereDistributedTransactionManager distributedTransactionManager) {
        return null != distributedTransactionManager && distributedTransactionManager.isInTransaction();

View on GitHub (pinned to e952770a21)

Solutions

1. Increase `maxPoolSize` (and pool `maximumPoolSize` for Hikari) on the affected storage unit so maxPoolSize >= max-connections-size-per-query times expected concurrent queries
2. Lower `max-connections-size-per-query` in server.yaml props (e.g. from 8 to 2 or 1) so each query needs fewer simultaneous connections
3. Inspect the cause chain of the thrown exception (getCause()) to confirm whether it is a pool timeout, auth failure, or backend max_connections limit, and raise the backend limit if that is the ceiling
4. Check for connection leaks or long-running transactions holding pool connections, which shrink the effective pool available per query

Example fix

# before (server.yaml / storage unit)
storageUnits:
  ds_0:
    pool:
      maxPoolSize: 2
props:
  max-connections-size-per-query: 8

# after
storageUnits:
  ds_0:
    pool:
      maxPoolSize: 20
props:
  max-connections-size-per-query: 2

// before issuing a fan-out query, confirm pool capacity >= per-query demand
int perQuery = metaDataContexts.getMetaData().getProps().<Integer>getValue(ConfigurationPropertyKey.MAX_CONNECTIONS_SIZE_PER_QUERY);
HikariDataSource ds = (HikariDataSource) dataSource; // per storage unit
if (ds.getMaximumPoolSize() < perQuery) {
    throw new IllegalStateException("maxPoolSize (" + ds.getMaximumPoolSize()
            + ") < max-connections-size-per-query (" + perQuery + "); query will fail");
}

try {
    List<QueryResult> rows = backendDataSource.getConnections(databaseName, dataSourceName, connectionSize);
} catch (final OverallConnectionNotEnoughException ex) {
    log.error("Need {} connections, got {} before failure; cause: {}",
            ex.getNeeded(), ex.getAcquired(), ex.getCause()); // adjust accessors to actual API
    // capacity problem: adjust maxPoolSize / max-connections-size-per-query, then retry once after backoff
}

When it happens

Trigger: A single sharded/routed SQL statement that fans out to more connections than `max-connections-size-per-query` allows per query against a pool whose `maxPoolSize` is smaller, or when the backend DB rejects a new connection (max_connections reached, pool timeout). Concretely: createConnections() loops i < connectionSize calling dataSource.getConnection(); any SQLException on iteration i closes the previous i connections and throws OverallConnectionNotEnoughException(connectionSize, result.size(), ex).

Common situations: Default maxPoolSize (e.g. HikariCP) left low while max-connections-size-per-query is 8 or higher; a burst of concurrent proxy queries exhausting backend max_connections; a slow/unavailable storage node causing pool acquisition timeouts during high load; migration to larger sharding counts increasing fan-out per query.

Related errors

• 0
• PROXY-00001
• 5
• SPI:1
• 0

AI-assisted analysis of apache/shardingsphere@e952770a21 (2026-08-14). Data as JSON: /api/errors/ab85ea8bbf6def52. Report an issue: GitHub.