What causes "can't decode primary replication position %q: %v" in vitessio/vitess?

Inspect the quoted position string in the message to identify the flavor/format.

can't decode primary replication position %q: %v

Error message

can't decode primary replication position %q: %v

What it means

The position string returned by the primary's PrimaryPosition RPC is decoded with replication.DecodePosition. If the string is not a valid GTID position for the expected flavor, this error reports the offending string and the decode cause. It indicates the position format is unrecognized or corrupt.

Source

Thrown at go/vt/mysqlctl/builtinbackupengine.go:1845

func getPrimaryPosition(ctx context.Context, tmc tmclient.TabletManagerClient, ts *topo.Server, keyspace, shard string) (replication.Position, error) {
	si, err := ts.GetShard(ctx, keyspace, shard)
	if err != nil {
		return replication.Position{}, vterrors.Wrap(err, "can't read shard")
	}
	if topoproto.TabletAliasIsZero(si.PrimaryAlias) {
		return replication.Position{}, fmt.Errorf("shard %v/%v has no primary", keyspace, shard)
	}
	ti, err := ts.GetTablet(ctx, si.PrimaryAlias)
	if err != nil {
		return replication.Position{}, fmt.Errorf("can't get primary tablet record %v: %v", topoproto.TabletAliasString(si.PrimaryAlias), err)
	}
	posStr, err := tmc.PrimaryPosition(ctx, ti.Tablet)
	if err != nil {
		return replication.Position{}, fmt.Errorf("can't get primary replication position: %v", err)
	}
	pos, err := replication.DecodePosition(posStr)
	if err != nil {
		return replication.Position{}, fmt.Errorf("can't decode primary replication position %q: %v", posStr, err)
	}
	return pos, nil
}

func init() {
	BackupRestoreEngineMap[builtinBackupEngineName] = &BuiltinBackupEngine{}
}

// closeWithRetry does just what it says. Retrying a close operation is important as
// an error is most likely transient/ephemeral and leaving around open file descriptors
// can lead to later problems as the file may be in a sort-of uploaded state where it
// exists but has not yet been finalized (this is true for GCS). This can cause
// unexpected behavior if you retry the file while the original request is still in this
// state. Most implementations such as GCS will automatically retry operations, but close
// is one that may be left to the caller (this is true for GCS).
// We model this retry after the GCS retry implementation described here:
// https://cloud.google.com/storage/docs/retry-strategy#go
func closeWithRetry(ctx context.Context, logger logutil.Logger, file io.Closer, name string) error {

View on GitHub (pinned to 01a25a7d17)

Solutions

Inspect the quoted position string in the message to identify the flavor/format.
Confirm all tablets in the shard run the same MySQL flavor and compatible Vitess version.
Check mysqld GTID configuration on the primary (gtid_mode, binlog settings).
If caused by mixed versions, complete the rolling upgrade and retry.

Defensive patterns

Strategy: validation

Validate before calling

pos, err := replication.DecodePosition(posStr)
if err != nil {
	return fmt.Errorf("primary returned undecodable position %q; check MySQL flavor/GTID config", posStr)
}

Try / catch

if err := runBackup(...); err != nil {
	if strings.Contains(err.Error(), "can't decode primary replication position") {
		// verify flavor consistency and mysqld GTID settings, then retry
	}
}

Prevention

Run a single MySQL flavor per cluster (never mix MySQL and MariaDB).
Keep Vitess versions in sync across rolling upgrades.
Verify gtid_mode/binlog configuration on primaries.

When it happens

Trigger: replication.DecodePosition(posStr) fails because the primary returned a position in an unexpected format — e.g. MariaDB GTID string handled by MySQL-flavored parsing, an empty/garbage string from the tablet, or mixed-version components with incompatible position encodings.

Common situations: Mixed MySQL/MariaDB flavors in one cluster; tablet returning an empty position when mysqld is misconfigured; rolling upgrade where an older/newer component emits an unfamiliar position format.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/e35e30116074a3ea. Report an issue: GitHub.