JuliusBrussee/caveman · error

duplicate object key

Error message

duplicate object key %q

What it means

readJSONTOONValue tracks keys already seen in an object and rejects duplicates with "duplicate object key %q". encoding/json's Decode into a map would silently keep the last value; TOON's strict parser fails closed so lossless round-tripping is guaranteed.

Solutions

  1. Remove the duplicate key from the JSON, keeping the intended value.
  2. Normalize input with `jq -c ' .'` style preprocessing — note jq also last-wins, so decide the winner explicitly first.
  3. Fix the upstream producer to emit unique keys.
  4. If duplicates are intentional, keep JSON — TOON refuses lossy input by design.

Example fix

// before
echo '{"a":1,"a":2}' | caveman-engine toon
// after
echo '{"a":2}' | caveman-engine toon
Defensive patterns

Strategy: validation

Validate before calling

func hasDuplicateKeys(input []byte) bool {
    dec := json.NewDecoder(bytes.NewReader(input))
    // walk tokens tracking keys per object depth; return true on repeat
    return walkAndDetectDupKeys(dec)
}

Try / catch

v, err := parseJSONTOON(input)
if err != nil && strings.Contains(err.Error(), "duplicate object key") {
    return fmt.Errorf("dedupe JSON keys before TOON: %w", err)
}

Prevention

When it happens

Trigger: parseJSONTOON encounters raw JSON text containing the same key twice in one object, e.g. {"a":1,"a":2}, whether handwritten, concatenated, or produced by a template.

Common situations: Handwritten JSON with copy-pasted keys; naive string concatenation of JSON fragments; generated JSON from tools that don't dedupe keys; semantically-duplicate keys differing only in escaping.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/dfe4f6ed0d89dff8. Report an issue: GitHub.

Appendix: source

Thrown at engine/compressors/toon_encode.go:144

		return toonValue{}, err
	}
	switch t := tok.(type) {
	case json.Delim:
		switch t {
		case '{':
			var fields []toonField
			seen := make(map[string]struct{})
			for dec.More() {
				keyTok, err := dec.Token()
				if err != nil {
					return toonValue{}, err
				}
				key, ok := keyTok.(string)
				if !ok {
					return toonValue{}, fmt.Errorf("object key is %T", keyTok)
				}
				if _, duplicate := seen[key]; duplicate {
					return toonValue{}, fmt.Errorf("duplicate object key %q", key)
				}
				seen[key] = struct{}{}
				val, err := readJSONTOONValue(dec)
				if err != nil {
					return toonValue{}, err
				}
				fields = append(fields, toonField{name: key, value: val})
			}
			end, err := dec.Token()
			if err != nil {
				return toonValue{}, err
			}
			if d, ok := end.(json.Delim); !ok || d != '}' {
				return toonValue{}, fmt.Errorf("object not closed")
			}
			return toonValue{kind: toonObject, obj: fields}, nil
		case '[':
			var arr []toonValue

View on GitHub (pinned to 3ee70a1026)