{"record":{"id":"65d1ead7d47d6e7d","repo":"apache/shardingsphere","slug":"source-storage-units-have-different-database-types","errorCode":null,"errorMessage":"Source storage units have different database types","messagePattern":"Source storage units have different database types","errorType":"exception","errorClass":"PipelineInvalidParameterException","httpStatus":null,"severity":"error","filePath":"kernel/data-pipeline/scenario/migration/core/src/main/java/org/apache/shardingsphere/data/pipeline/scenario/migration/api/MigrationJobAPI.java","lineNumber":146,"sourceCode":"        for (MigrationSourceTargetEntry each : new HashSet<>(sourceTargetEntries).stream()\n                .sorted(Comparator.comparing(MigrationSourceTargetEntry::getTargetTableName).thenComparing(each -> each.getSource().format())).collect(Collectors.toList())) {\n            sourceDataNodes.computeIfAbsent(each.getTargetTableName(), key -> new LinkedList<>()).add(each.getSource());\n            ShardingSpherePreconditions.checkState(1 == sourceDataNodes.get(each.getTargetTableName()).size(),\n                    () -> new PipelineInvalidParameterException(\"More than one source table for \" + each.getTargetTableName()));\n            String dataSourceName = each.getSource().getDataSourceName();\n            if (configSources.containsKey(dataSourceName)) {\n                continue;\n            }\n            ShardingSpherePreconditions.checkContainsKey(metaDataDataSource, dataSourceName,\n                    () -> new PipelineInvalidParameterException(dataSourceName + \" doesn't exist. Run `SHOW MIGRATION SOURCE STORAGE UNITS;` to verify it.\"));\n            Map<String, Object> sourceDataSourcePoolProps = dataSourceConfigSwapper.swapToMap(metaDataDataSource.get(dataSourceName));\n            StandardPipelineDataSourceConfiguration sourceDataSourceConfig = new StandardPipelineDataSourceConfiguration(sourceDataSourcePoolProps);\n            configSources.put(dataSourceName, buildYamlPipelineDataSourceConfiguration(sourceDataSourceConfig.getType(), sourceDataSourceConfig.getParameter()));\n            DatabaseType sourceDatabaseType = sourceDataSourceConfig.getDatabaseType();\n            if (null == result.getSourceDatabaseType()) {\n                result.setSourceDatabaseType(sourceDatabaseType.getType());\n            } else if (!result.getSourceDatabaseType().equals(sourceDatabaseType.getType())) {\n                throw new PipelineInvalidParameterException(\"Source storage units have different database types\");\n            }\n        }\n        result.setSources(configSources);\n        ShardingSphereDatabase targetDatabase = PipelineContextManager.getProxyContext().getMetaDataContexts().getMetaData().getDatabase(targetDatabaseName);\n        PipelineDataSourceConfiguration targetPipelineDataSourceConfig = buildTargetPipelineDataSourceConfiguration(targetDatabase);\n        result.setTarget(buildYamlPipelineDataSourceConfiguration(targetPipelineDataSourceConfig.getType(), targetPipelineDataSourceConfig.getParameter()));\n        result.setTargetDatabaseType(targetPipelineDataSourceConfig.getDatabaseType().getType());\n        List<JobDataNodeEntry> tablesFirstDataNodes = sourceDataNodes.entrySet().stream()\n                .map(entry -> new JobDataNodeEntry(entry.getKey(), entry.getValue().subList(0, 1))).collect(Collectors.toList());\n        result.setTargetTableNames(new ArrayList<>(sourceDataNodes.keySet()).stream().sorted().collect(Collectors.toList()));\n        result.setTargetTableSchemaMap(buildTargetTableSchemaMap(sourceDataNodes));\n        result.setTablesFirstDataNodes(new JobDataNodeLine(tablesFirstDataNodes).marshal());\n        result.setJobShardingDataNodes(JobDataNodeLineConvertUtils.convertDataNodesToLines(sourceDataNodes).stream().map(JobDataNodeLine::marshal).collect(Collectors.toList()));\n        result.setJobId(PipelineJobIdUtils.marshal(new MigrationJobId(contextKey, result.getJobShardingDataNodes())));\n        return result;\n    }\n    \n    private YamlPipelineDataSourceConfiguration buildYamlPipelineDataSourceConfiguration(final String type, final String param) {","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/apache/shardingsphere/blob/e952770a215630a3659c75d64369168cd3e26b82/kernel/data-pipeline/scenario/migration/core/src/main/java/org/apache/shardingsphere/data/pipeline/scenario/migration/api/MigrationJobAPI.java#L128-L164","documentation":"When building a migration job configuration, MigrationJobAPI requires every source storage unit to share one database type: the first unit fixes result.sourceDatabaseType and any later unit with a different type aborts with PipelineInvalidParameterException. Heterogeneous sources are rejected because the pipeline assumes a single source dialect.","triggerScenarios":"Running MIGRATE TABLE or a migration job build where the sourceDataNodes span multiple storage units whose JDBC URLs resolve to different database types (e.g. one PostgreSQL and one MySQL unit, or one OpenGauss vs PostgreSQL).","commonSituations":"Mixing storage units of different engines in one schema and referencing tables from both in a single migration; a unit whose URL/driver metadata resolves to an unexpected database type; renaming or misconfiguring units so the wrong one matches table names.","solutions":["Run SHOW MIGRATION SOURCE STORAGE UNITS; and confirm the database type of every unit that owns the tables being migrated.","Split the migration into separate jobs per database type, each referencing only units of one engine.","If all sources are really the same engine, check each unit's JDBC URL and stored database type metadata for a misconfiguration (e.g. wrong URL pointing at a different server product).","Remove unused/misconfigured storage units from the schema so they cannot be picked up."],"exampleFix":"-- before (t_order on MySQL unit, t_user on PostgreSQL unit)\nMIGRATE TABLE db_0.t_order INTO db_1.t_order, db_0.t_user INTO db_1.t_user;\n-- after (one job per engine)\nMIGRATE TABLE db_0.t_order INTO db_1.t_order;\nMIGRATE TABLE db_pg.t_user INTO db_1.t_user;","handlingStrategy":"validation","validationCode":"Set<String> sourceTypes = new HashSet<>();\nfor (String each : sourceDataNodes.keySet()) {\n    sourceTypes.add(getStorageUnitDatabaseType(each)); // from SHOW STORAGE UNITS metadata\n}\nShardingSpherePreconditions.checkState(sourceTypes.size() <= 1,\n    () -> new IllegalArgumentException(\"migration sources must share one database type\"));","typeGuard":null,"tryCatchPattern":"try {\n    api.migrate(...);\n} catch (final PipelineInvalidParameterException ex) {\n    if (ex.getMessage().contains(\"different database types\")) { splitJobPerEngine(); } else { throw ex; }\n}","preventionTips":["Check SHOW MIGRATION SOURCE STORAGE UNITS output for a single database type before MIGRATE TABLE.","Keep one engine per logical migration scope.","Avoid wildcard table selections spanning multiple units."],"tags":["migration","distsql","configuration","heterogeneous"],"backgroundTag":null,"analyzedSha":"e952770a215630a3659c75d64369168cd3e26b82","analyzedAt":"2026-08-14T13:54:53.392Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}