malformed cache entry: could not unmarshal JSON; unable to p

Error message

malformed cache entry: could not unmarshal JSON; unable to parse: %w

What it means

When loading a memoization entry from a ConfigMap, the stored JSON string is unmarshalled into cache.Entry. If the ConfigMap data value is not valid JSON or does not match the Entry schema, the load fails with this wrapped parse error so the controller does not silently treat corrupted data as a cache hit.

Source

Thrown at workflow/controller/cache/configmap_cache.go:117

		return nil, err
	}
	err = c.validateConfigmap(ctx, cm)
	if err != nil {
		return nil, err
	}

	c.logInfo(ctx, logging.Fields{}, "config map cache loaded")
	hitTime := time.Now()
	rawEntry, ok := cm.Data[key]
	if !ok || rawEntry == "" {
		c.logInfo(ctx, logging.Fields{}, "config map cache miss: entry does not exist")
		return nil, nil
	}

	var entry Entry
	err = json.Unmarshal([]byte(rawEntry), &entry)
	if err != nil {
		return nil, fmt.Errorf("malformed cache entry: could not unmarshal JSON; unable to parse: %w", err)
	}

	entry.LastHitTimestamp = metav1.Time{Time: hitTime}
	entryJSON, err := json.Marshal(entry)
	if err != nil {
		c.logError(ctx, err, logging.Fields{"key": key}, "Unable to marshal cache entry with last hit timestamp")
		return nil, fmt.Errorf("unable to marshal cache entry with last hit timestamp: %w", err)
	}
	cm.Data[key] = string(entryJSON)

	_, err = c.kubeClient.CoreV1().ConfigMaps(c.namespace).Update(ctx, cm, metav1.UpdateOptions{})
	if err != nil {
		return nil, err
	}
	return &entry, nil
}

func (c *configMapCache) Save(ctx context.Context, key string, nodeID string, value *wfv1.Outputs) error {

View on GitHub (pinned to 35bff19146)

Solutions

kubectl get cm <cache-cm> -o jsonpath='{.data.<key>}' and validate the value parses as JSON with the Entry fields (ExitCode, NodeID, CreationTimestamp, LastHitTimestamp).
Delete the corrupted key from the ConfigMap so the memoized node re-executes and re-saves a fresh entry.
If all entries are stale-schema, delete the whole ConfigMap (memoization entries are safe to rebuild).
Verify no external tooling writes to memoization ConfigMaps (label selector: workflows.argoproj.io/workflow).
Pin/upgrade the Argo version consistently so schema matches entries on disk.

Example fix

// before: manually written entry
cm.Data["key"] = "done"
// after: valid JSON entry
import encjson "encoding/json"
b, _ := encjson.Marshal(controllercache.Entry{NodeID: "abc", ExitCode: "0", CreationTimestamp: metav1.Now(), LastHitTimestamp: metav1.Now()})
cm.Data["key"] = string(b)

Defensive patterns

Strategy: validation

Validate before calling

const raw = cm.data?.[key];
try { const e = JSON.parse(raw); if (!('nodeID' in e) && !('exitCode' in e)) throw new Error('not an Entry'); } catch { console.warn(`corrupt cache entry for key ${key}; delete it before lookup`); }

Type guard

function isCacheEntry(raw) {
  if (typeof raw !== 'string') return false;
  try { const e = JSON.parse(raw); return e && typeof e === 'object' && 'nodeID' in e; } catch { return false; }
}

Try / catch

try {
  entry, err := cache.Load(ctx, key)
  if err != nil {
    var parseErr *json.SyntaxError
    if errors.As(err, &parseErr) || strings.Contains(err.Error(), "could not unmarshal JSON") {
      // treat as cache miss: delete corrupt key, let node re-execute
    } else { return err }
  }
}

Prevention

Never hand-edit memoization ConfigMaps; use kubectl patch only to remove bad keys.
Keep Argo controller version consistent with entries on disk across upgrades.
Prevent other tools from writing to ConfigMaps matched by memoization labels.
Alert on this error early — it permanently blocks GC for that ConfigMap until fixed.

When it happens

Trigger: cm.Data[key] was written by hand or by another tool with non-JSON content; entry JSON was truncated (ConfigMap size edits, manual kubectl edit); the Entry struct changed between Argo versions so old entries no longer unmarshal.

Common situations: Manually editing the memoization ConfigMap; migrating from an older Argo version with a different entry schema; another controller or script sharing the same ConfigMap and writing raw strings; truncated entries after ConfigMap size limits.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/1b62e41645fc1343. Report an issue: GitHub.