cilium/cilium · error

failed to collect hubble flows

Error message

failed to collect hubble flows: %w

What it means

The sysdump appends a task that calls c.submitHubbleFlowsTasks to gather Hubble flow logs (gops/observability output) from the Cilium agent containers in c.CiliumPods. Any failure inside that submission — pod exec setup, missing hubble container/config, or per-pod task errors — is wrapped with this prefix and returned to the Run loop. It means Hubble flow collection could not be scheduled or executed.

Solutions

  1. Check that Hubble is enabled and running (kubectl -n <ns> get pods -l k8s-app=cilium, verify hubble is configured)
  2. Retry the sysdump once agent pods are Running and exec-capable
  3. Run with a lower/sane --hubble-flows-count or omit it if Hubble is not installed
  4. Inspect the wrapped cause (%w) from the sysdump log for the underlying exec/list error

Example fix

// before: requesting flows in a cluster without Hubble
cilium-cli sysdump --hubble-flows-count 100
// after: enable Hubble first or drop the flag
cilium-cli hubble enable && cilium-cli sysdump --hubble-flows-count 100
Defensive patterns

Strategy: try-catch

Validate before calling

pods, _ := client.ListPods(ctx, ciliumNs, metav1.ListOptions{LabelSelector: "k8s-app=cilium"})
if len(pods.Items) == 0 {
    return errors.New("no Cilium pods; skip hubble flow collection")
}
if _, err := client.ListPods(ctx, hubbleNs, metav1.ListOptions{LabelSelector: "k8s-app=hubble-relay"}); err != nil {
    return errors.New("hubble not detected; run with --hubble-flows-count=0")
}

Type guard

func isHubbleFlowCollectionErr(err error) bool {
    return err != nil && strings.Contains(err.Error(), "failed to collect hubble flows")
}

Try / catch

err := sysdump.Run(ctx)
var target *tasks.TaskFailedError // or inspect message
if err != nil && strings.Contains(err.Error(), "failed to collect hubble flows") {
    log.Printf("hubble flow collection failed (is Hubble enabled?): %v", err)
    // degrade gracefully: continue without flows
} else if err != nil {
    return err
}

Prevention

When it happens

Trigger: c.Options.HubbleFlowsCount > 0 and c.submitHubbleFlowsTasks(ctx, c.CiliumPods, ciliumAgentContainerName) returns an error: typically pod exec failures against the agent container, no running Cilium pods, or Hubble not enabled in the cluster.

Common situations: Hubble not deployed/enabled while the user still requests --hubble-flows-count > 0; Cilium agent pods crashing so exec into ciliumAgentContainerName fails; RBAC/network restrictions blocking exec on agent pods; requesting flows from pods that were filtered out.

Related errors


AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/ee619fd74eb9bb09. Report an issue: GitHub.

Appendix: source

Thrown at cilium-cli/sysdump/sysdump.go:1590

				for _, pod := range c.CiliumPods {
					if pod.Status.Phase == corev1.PodRunning {
						return c.submitKVStoreTasks(ctx, pod.DeepCopy())
					}
				}
				return fmt.Errorf("could not find running Cilium Pod")
			},
		},
	}
	ciliumTasks = append(ciliumTasks, collectCiliumV2OrV2Alpha1Resource(c, "ciliumloadbalancerippools", "Cilium LoadBalancer IP Pools"))
	ciliumTasks = append(ciliumTasks, collectCiliumV2OrV2Alpha1Resource(c, "ciliumpodippools", "Cilium Pod IP Pools"))
	if c.Options.HubbleFlowsCount > 0 {
		ciliumTasks = append(ciliumTasks, Task{
			CreatesSubtasks: true,
			Description:     "Collecting Hubble flows from Cilium pods",
			Quick:           false,
			Task: func(ctx context.Context) error {
				if err := c.submitHubbleFlowsTasks(ctx, c.CiliumPods, ciliumAgentContainerName); err != nil {
					return fmt.Errorf("failed to collect hubble flows: %w", err)
				}
				return nil
			},
		})
	}

	// TODO(#2645): Ideally we would split ciliumTasks into
	// operator, agent, generic tasks, ..., etc
	// and only run then when feasible.
	if c.Options.CiliumNamespace != "" || c.Options.CiliumOperatorNamespace != "" {
		tasks = append(tasks, ciliumTasks...)

		serialTasks = append(serialTasks, Task{
			CreatesSubtasks: true,
			Description:     "Collecting profiling data from Cilium pods",
			Quick:           false,
			Task: func(_ context.Context) error {
				if !c.Options.Profiling {

View on GitHub (pinned to ac7b90affa)