mikefarah/yq · error

CollectObject: mismatching node sizes; are you creating a ma

Error message

CollectObject: mismatching node sizes; are you creating a map with mismatching key value pairs?

What it means

This error is thrown by collectObjectOperator, which implements yq's object construction (the `{...}` expression) by 'cross-multiplying' (rotating) the contents of all candidate nodes into new map entries. It uses the FIRST candidate node's content length as the expected number of key/value slots and requires every other candidate to have at least that many children. When a later candidate has fewer children than the first, the rotation arrays would be indexed beyond what that node provides, so yq aborts with this error instead of silently producing a malformed map.

Source

Thrown at pkg/yqlib/operator_collect_object.go:39

	context := originalContext.WritableClone()

	if context.MatchingNodes.Len() == 0 {
		candidate := &CandidateNode{Kind: MappingNode, Tag: "!!map", Value: "{}"}
		log.Debugf("collectObjectOperation - starting with empty map")
		return context.SingleChildContext(candidate), nil
	}
	first := context.MatchingNodes.Front().Value.(*CandidateNode)
	var rotated = make([]*list.List, len(first.Content))

	for i := 0; i < len(first.Content); i++ {
		rotated[i] = list.New()
	}

	for el := context.MatchingNodes.Front(); el != nil; el = el.Next() {
		candidateNode := el.Value.(*CandidateNode)
		if len(candidateNode.Content) < len(first.Content) {
			return Context{}, fmt.Errorf("CollectObject: mismatching node sizes; are you creating a map with mismatching key value pairs?")
		}

		for i := 0; i < len(first.Content); i++ {
			log.Debugf("rotate[%v] = %v", i, NodeToString(candidateNode.Content[i]))
			log.Debugf("children:\n%v", NodeContentToString(candidateNode.Content[i], 0))
			rotated[i].PushBack(candidateNode.Content[i])
		}
	}
	log.Debugf("collectObjectOperation, length of rotated is %v", len(rotated))

	newObject := list.New()
	for i := 0; i < len(first.Content); i++ {
		additions, err := collect(d, context.ChildContext(list.New()), rotated[i])
		if err != nil {
			return Context{}, err
		}
		// we should reset the parents and keys of these top level nodes,
		// as they are new

View on GitHub (pinned to 8b5af0694b)

Solutions

  1. Inspect the input with `yq 'map(len)'` (or `.[] | length`) to find the node(s) with fewer entries than the first and fix or filter them out.
  2. Filter out short/empty nodes before the collect: pipe through `select(length == N)` or `select(type == "!!map")` first.
  3. Ensure each element you feed to `{...}` produces the same number of key/value pairs, e.g. by defaulting missing fields: `.name //= ""`.
  4. Split the input into homogeneous groups and run a separate `{...}` collect per group instead of one over mixed data.

Example fix

# before (fails when some docs are empty)
yq '[.[] | {name: .name, port: .port}] | .[]' input.yaml
# after (filter out docs without both fields)
yq '[.[] | select(has("name") and has("port")) | {name: .name, port: .port}] | .[]' input.yaml
Defensive patterns

Strategy: validation

Validate before calling

# validate all documents have the same number of entries before collecting:
yq -e 'all(.[]; length == (.[0] | length))' input.yaml || echo "heterogeneous input"
# or in a script:
LEN=$(yq '.[0] | length' input.yaml)
yq -e "all(.[]; length == ${LEN})" input.yaml

Type guard

// Go: guard before running a collect-object style rotation
func uniformSize(nodes []*yqlib.CandidateNode) bool {
    if len(nodes) == 0 { return true }
    n := len(nodes[0].Content)
    for _, nd := range nodes {
        if len(nd.Content) < n { return false }
    }
    return true
}

Prevention

When it happens

Trigger: Running a `{...}` collect-object expression where the piped-in results are heterogeneous: e.g. `yq '.a[] | {key: .name}'`-style expressions where one result is a map with 2 entries and a later result is a scalar or 1-entry map, so len(candidateNode.Content) < len(first.Content). Specifically triggered when any node AFTER the first in context.MatchingNodes has fewer Content entries than the first node.

Common situations: Iterating over documents/entries where some items are missing fields (e.g. some YAML docs are empty or scalars instead of maps); passing an empty map `{}` node after a non-empty map into a collect; scripts that assume all input documents share the same schema; feeding mixed JSON/YAML inputs where nulls or short arrays sneak in.

Related errors


AI-assisted analysis of mikefarah/yq@8b5af0694b (2026-09-05). Data as JSON: /api/errors/382b1c352d78ae42. Report an issue: GitHub.