apache/shardingsphere · error · PipelineInternalException
Load meta data for schema '%s' and table '%s' failed
Error message
Load meta data for schema '%s' and table '%s' failed
What it means
StandardPipelineTableMetaDataLoader caches table metadata per (schema, table) and, on a cache miss, opens a connection and introspects the table; any SQLException from that introspection is wrapped in PipelineInternalException('Load meta data for schema X and table Y failed'). The original SQLException (carried as cause) explains the real problem: missing table, wrong schema, permission, or connection failure.
Source
Thrown at kernel/data-pipeline/core/src/main/java/org/apache/shardingsphere/data/pipeline/core/metadata/loader/StandardPipelineTableMetaDataLoader.java:72
public final class StandardPipelineTableMetaDataLoader implements PipelineTableMetaDataLoader {
private final PipelineDataSource dataSource;
private volatile IdentifierCasePolicy tableIdentifierCasePolicy;
private final Map<TableMetaDataCacheKey, PipelineTableMetaData> tableMetaDataMap = new ConcurrentHashMap<>();
@Override
public PipelineTableMetaData getTableMetaData(final String schemaName, final String tableName) {
String qualifiedSchemaName = getQualifiedSchemaName(schemaName, new DatabaseTypeRegistry(dataSource.getDatabaseType()));
PipelineTableMetaData result = findTableMetaData(qualifiedSchemaName, tableName);
if (null != result) {
return result;
}
try {
loadTableMetaData(qualifiedSchemaName, tableName);
} catch (final SQLException ex) {
throw new PipelineInternalException(String.format("Load meta data for schema '%s' and table '%s' failed", schemaName, tableName), ex);
}
result = findTableMetaData(qualifiedSchemaName, tableName);
if (null == result) {
log.warn("Can not load meta data for table '{}'", tableName);
}
return result;
}
private void loadTableMetaData(final String schemaName, final String tableName) throws SQLException {
try (Connection connection = dataSource.getConnection()) {
Map<ShardingSphereIdentifier, PipelineTableMetaData> loadedTableMetaData = loadTableMetaData(connection, schemaName, tableName);
loadedTableMetaData.forEach((key, value) -> tableMetaDataMap.put(new TableMetaDataCacheKey(schemaName, key.getValue()), value));
}
}
private Map<ShardingSphereIdentifier, PipelineTableMetaData> loadTableMetaData(final Connection connection, final String schemaName, final String tableNamePattern) throws SQLException {
Collection<String> tableNames = new LinkedList<>();
try (ResultSet resultSet = connection.getMetaData().getTables(connection.getCatalog(), schemaName, tableNamePattern, null)) {View on GitHub (pinned to e952770a21)
Solutions
- Read the cause exception to classify: unknown table vs permission vs connectivity.
- Verify the exact schema and table names (case, quoting) exist on the source with SHOW TABLES / information_schema as the pipeline user.
- Grant metadata-read privileges to the pipeline user and retry the job step.
- For transient failures, retry/restart the pipeline job after connectivity is stable.
Defensive patterns
Strategy: try-catch
Validate before calling
// preflight: confirm the table is introspectable as the pipeline user
try (Connection c = dataSource.getConnection()) {
DatabaseMetaData md = c.getMetaData();
try (ResultSet rs = md.getColumns(null, schemaName, tableName, null)) {
if (!rs.next()) {
throw new IllegalStateException("table not visible: " + schemaName + "." + tableName);
}
}
} Try / catch
try {
PipelineTableMetaData meta = loader.getTableMetaData(schemaName, tableName);
} catch (final PipelineInternalException ex) {
SQLException cause = (SQLException) ex.getCause();
// classify by cause.getErrorCode()/message: missing table, privileges, connectivity
} Prevention
- Normalize schema/table names (case, quoting) to the source database's rules in job configuration.
- Grant metadata-read privileges to the pipeline user up front.
- Validate that all tables named in the job config still exist right before job start.
When it happens
Trigger: Pipeline metadata loading (during job preparation or inventory) for a schema/table that does not exist on the source, a table name with case sensitivity mismatch, an unprivileged introspection call, or a dropped connection during getColumns/getPrimaryKeys.
Common situations: Schema/table name case mismatch on case-sensitive databases (PostgreSQL/openGauss lowercase folding); referencing a table dropped after job config was created; the pipeline user lacking metadata-reading privileges; transient network failures during job init.
Related errors
AI-assisted analysis of apache/shardingsphere@e952770a21 (2026-08-14).
Data as JSON: /api/errors/70ac1d3a6caec83a.
Report an issue: GitHub.