What causes "invalid key:value pair" in vitessio/vitess?

Inspect the wrapped cause for the underlying RPC failure and address it (vtctld health, network, topo state)

invalid key:value pair

Error message

invalid key:value pair

What it means

After acquiring the topoReadPool, vtadmin calls vtctld's ShardReplicationPositions RPC. Any error returned by that RPC for a keyspace/shard is wrapped with context and recorded, and that shard's replication position is skipped.

Source

Thrown at go/flagutil/flagutil.go:29

distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

// Package flagutil contains flags that parse string lists and string
// maps.
package flagutil

import (
	"errors"
	"sort"
	"strings"

	"github.com/spf13/pflag"
)

var errInvalidKeyValuePair = errors.New("invalid key:value pair")

// StringListValue is a []string flag that accepts a comma separated
// list of elements. To include an element containing a comma, quote
// it with a backslash '\'.
type StringListValue []string

// Get returns the []string value of this flag.
func (value StringListValue) Get() any {
	return []string(value)
}

func parseListWithEscapes(v string, delimiter rune) (value []string) {
	var escaped, lastWasDelimiter bool

	current := make([]rune, 0, len(v))
	for _, r := range v {
		lastWasDelimiter = false
		if !escaped {

View on GitHub (pinned to 01a25a7d17)

Solutions

Inspect the wrapped cause for the underlying RPC failure and address it (vtctld health, network, topo state)
Retry the request; transient tablet dial failures are common during maintenance
Verify the shard still exists and its tablets are registered correctly in the topo server

Defensive patterns

Strategy: try-catch

Try / catch

resp, err := cluster.GetShardReplicationPositions(ctx, ks, shard)
if err != nil {
	var recerr interface{ Unwrap() error }
	if errors.As(err, &recerr) { log.Printf("underlying vtctld error: %v", recerr.Unwrap()) }
	// fall back to partial results or retry
}

Prevention

Retry transient vtctld RPC failures with backoff
Keep vtctld healthy and reachable from vtadmin
Verify shard existence in topo before per-shard queries

When it happens

Trigger: Calling Cluster.GetShardReplicationPositions when the vtctld RPC fails for a specific shard: unreachable vtctld, shard missing from topo, or tablet-level RPC errors inside vtctld.

Common situations: vtctld restarted or overloaded mid-request; shard removed between listing and querying; network partition to tablets vtctld dials; permissions/rpc auth failures.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/5c13cbb63dd28cc8. Report an issue: GitHub.