MyCATApache/Mycat-Server · error · DataMigratorException

->

Error message

{errMessage} -> {loadData}

What it means

During data migration import, MysqlDataIO runs the mysql client (load data infile) as an external process and scans its stderr line by line. Any stderr line whose lowercase text contains the substring "err" is treated as a fatal import failure and wrapped in a DataMigratorException with the offending line and the loadData command. Because the check is a naive substring match, even benign stderr output containing 'err' (e.g. 'Warning: ... error_log') aborts the migration.

Solutions

1. Run the printed loadData command manually and fix the underlying MySQL load-data error (file path, permissions, column list, separators).
2. Check mysqld's secure_file_priv and local_infile settings and ensure the data file is in an allowed location with read permission for the mysql user.
3. Verify the mysql client binary exists and is executable, and that credentials in the migration config are correct.
4. On Windows, confirm the expected GBK console output decoding matches the child process encoding so error lines are parsed correctly.

Example fix

// before
String sql = "load data infile '" + file + "' into table " + table + ";";
// after
String sql = "load data local infile '" + file.replace("\\", "/") + "' into table " + table
    + " character set utf8mb4 fields terminated by ',' enclosed by '\"' lines terminated by '\n';";

// before import
File dataFile = new File(dataPath);
if (!dataFile.isFile() || !dataFile.canRead()) throw new IllegalStateException("data file unreadable: " + dataPath);
// verify target table columns match file
try (Connection c = DriverManager.getConnection(jdbcUrl, user, pass)) {
  ResultSet rs = c.createStatement().executeQuery("SELECT COUNT(*) FROM " + table);
  rs.next();
}

try {
  dataIO.importData(...);
} catch (DataMigratorException e) {
  logger.error("load data failed: {}", e.getMessage()); // run the loadData command manually to debug
}

When it happens

Trigger: Calling importData for a table where the spawned mysql load-data process writes any stderr line containing 'err', e.g. incorrect CSV field counts, missing data file, wrong LOAD DATA syntax, or MySQL connection/auth failure of the child process.

Common situations: Data file path not readable by mysqld (secure_file_priv restrictions), column count/separator mismatch between the migrated file and target table, wrong character set (GBK vs UTF-8 on Windows), or mysql client not on PATH producing a command-not-found error on stderr.

Related errors

• ->
• global seq and share join and special…
• txIsolation
• err.errno
• Unknown Packet!

AI-assisted analysis of MyCATApache/Mycat-Server@65f8d8beb7 (2026-09-11). Data as JSON: /api/errors/e02087d741765ced. Report an issue: GitHub.

Appendix: source

Thrown at src/main/java/io/mycat/util/dataMigrator/dataIOImpl/MysqlDataIO.java:71

		String user = dn.getUserName();
		String pwd = dn.getPwd();
		String db = dn.getDb();

//		String loadData ="?mysql -h? -P? -u? -p? -D? --local-infile=1 -e \"load data local infile '?' replace into table ? CHARACTER SET '?' FIELDS TERMINATED BY ','  LINES TERMINATED BY '\\r\\n'\"";
		String loadData = "?mysql -h? -P? -u? -p? -D?  -f --default-character-set=? -e \"source ?\"";
		loadData = DataMigratorUtil.paramsAssignment(loadData,"?",mysqlBin,ip,port,user,pwd,db,charset,file.getAbsolutePath());
		LOGGER.info(table.getSchemaAndTableName()+" "+loadData);
		Process process = DataMigratorUtil.exeCmdByOs(loadData);

		//获取错误信息

		InputStreamReader in = new InputStreamReader(process.getErrorStream(),isWindows()?"GBK": Charset.defaultCharset().name());
		BufferedReader br = new BufferedReader(in);
		String errMessage = null;
        while ((errMessage = br.readLine()) != null) {
            if(errMessage.trim().toLowerCase().contains("err")){
            	System.out.println(errMessage+" -> "+loadData);
            	throw new DataMigratorException(errMessage+" -> "+loadData);
            }
        }

		process.waitFor();
	}

	@Override
	public File exportData(TableMigrateInfo table,DataNode dn, String tableName, File export, File condition) throws IOException, InterruptedException {
		String ip = dn.getIp();
		int port = dn.getPort();
		String user = dn.getUserName();
		String pwd = dn.getPwd();
		String db = dn.getDb();

//		String mysqlDump = "?mysqldump -h? -P? -u? -p? ? ?  --no-create-info --default-character-set=? "
//				+ "--add-locks=false --tab='?' --fields-terminated-by=',' --lines-terminated-by='\\r\\n' --where='? in(?)'";
		//由于mysqldump导出csv格式文件只能导出到本地，暂时替换成导出insert形式的文件
		String mysqlDump = "?mysqldump -h? -P? -u? -p? ? ?  --compact --no-create-info --default-character-set=? --add-locks=false --where=\"? in (#)\" --result-file=\"?\" ";

View on GitHub (pinned to 65f8d8beb7)