cilium/cilium · error

failed to parse pod metrics JSON: %w

Error message

failed to parse pod metrics JSON: %w

What it means

`formatPodMetricsAsTable` unmarshals raw pod-metrics JSON (metrics.k8s.io PodMetricsList, as from `kubectl top pods`) into metricsapi.PodMetricsList. A malformed or unexpected payload yields this wrapped error instead of the pod metrics table.

Source

Thrown at cilium-cli/sysdump/sysdump.go:3789

			return "<unknown>"
		}
		return strconv.FormatUint(*v, 10)
	}

	fs, imageFs := summary.Node.Fs, summary.Node.Runtime.ImageFs
	return fmt.Sprintf("%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s",
		nodeName,
		mib(fs.UsedBytes), mib(fs.CapacityBytes), pct(fs.UsedBytes, fs.CapacityBytes),
		mib(imageFs.UsedBytes), mib(imageFs.CapacityBytes),
		count(fs.InodesFree), diskPressure)
}

// formatPodMetricsAsTable formats the raw pod metrics JSON into a table format like kubectl top pods
func (c *Collector) formatPodMetricsAsTable(rawMetrics string) (string, error) {
	// Parse the metrics JSON
	var podMetrics metricsapi.PodMetricsList
	if err := json.Unmarshal([]byte(rawMetrics), &podMetrics); err != nil {
		return "", fmt.Errorf("failed to parse pod metrics JSON: %w", err)
	}

	var sb strings.Builder
	tw := tabwriter.NewWriter(&sb, 0, 0, 2, ' ', 0)
	fmt.Fprintln(tw, "NAMESPACE\tNAME\tCPU(cores)\tMEMORY(bytes)")

	for _, metric := range podMetrics.Items {
		namespace := metric.Namespace
		name := metric.Name

		// Sum CPU and memory usage across all containers in the pod
		var totalCPUMillis int64
		var totalMemBytes int64

		for _, container := range metric.Containers {
			if cpuUsage, exists := container.Usage[corev1.ResourceCPU]; exists {
				totalCPUMillis += cpuUsage.MilliValue()
			}

View on GitHub (pinned to ac7b90affa)

Solutions

1. Inspect the raw pod metrics payload to see whether it is empty, HTML, or schema-mismatched JSON
2. Validate metrics-server works: `kubectl top pods -A` should return rows
3. Ensure the metrics-server version matches the cluster's metrics.k8s.io API version
4. Make collection fail fast so empty/garbage payloads never reach the formatter
5. Return an empty table instead of an error when there are no pod metrics

Example fix

// before
if err := json.Unmarshal([]byte(rawMetrics), &podMetrics); err != nil {
	return "", fmt.Errorf("failed to parse pod metrics JSON: %w", err)
}
// after
if strings.TrimSpace(rawMetrics) == "" {
	return "no pod metrics available\n", nil
}
if err := json.Unmarshal([]byte(rawMetrics), &podMetrics); err != nil {
	return "", fmt.Errorf("failed to parse pod metrics JSON: %w (payload head: %.100s)", err, rawMetrics)
}

raw := strings.TrimSpace(rawMetrics)
if raw == "" || raw[0] != '{' {
	return errors.New("pod metrics payload is empty or not JSON")
}

func validPodMetrics(s string) bool {
	var m metricsapi.PodMetricsList
	return json.Unmarshal([]byte(s), &m) == nil
}

var podMetrics metricsapi.PodMetricsList
if err := json.Unmarshal([]byte(rawMetrics), &podMetrics); err != nil {
	log.Printf("pod metrics unparseable, skipping table: %v", err)
	return "", nil
}

When it happens

Trigger: The raw pod metrics string collected for the sysdump is empty, truncated, non-JSON (HTML error page), or its schema does not match metricsapi.PodMetricsList (metrics API version drift).

Common situations: metrics-server unavailable or returning error bodies when the sysdump gathered pod metrics; an ingress/proxy returning a login/error page; partial capture of a large payload; older metrics API shapes.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• failed to parse metrics JSON: %w
• failed to decode features status from %s: %w
• failed to unmarshal json: %w
• error unmarshalling file %q: %w
• failed to collect Kubernetes nodes memory/cpu usage: %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/d190ae0a99b286a0. Report an issue: GitHub.