cilium/cilium · error

failed to submit logs task for %q (%q): %w

Error message

failed to submit logs task for %q (%q): %w

What it means

This error is returned when the sysdump fails to SUBMIT the pooled log-collection task for a pod/container pair — i.e. c.Pool.Submit itself errored, not the log fetch. It usually indicates the worker pool is shut down or its context was cancelled while tasks were still being scheduled.

Source

Thrown at cilium-cli/sysdump/sysdump.go:2988

				previous := false
				for _, s := range p.Status.ContainerStatuses {
					if s.Name == d.Name && s.RestartCount > 0 {
						previous = true
						break
					}
				}
				if previous {
					c.logDebug("Collecting logs for restarted container %q in pod %q in namespace %q", d.Name, p.Name, p.Namespace)
					if err := c.WithFileSink(fmt.Sprintf(ciliumPreviousLogsFileName, p.Name, d.Name), func(out io.Writer) error {
						return c.Client.GetLogs(ctx, p.Namespace, p.Name, d.Name,
							corev1.PodLogOptions{LimitBytes: &limitBytes, SinceTime: &t, Previous: true, Timestamps: true}, out)
					}); err != nil {
						return fmt.Errorf("failed to collect previous logs for %q (%q) in namespace %q: %w", p.Name, d.Name, p.Namespace, err)
					}
				}
				return nil
			}); err != nil {
				return fmt.Errorf("failed to submit logs task for %q (%q): %w", p.Name, d.Name, err)
			}
		}
	}
	return nil
}

func (c *Collector) submitFlavorSpecificTasks(f k8s.Flavor) error {
	switch f.Kind {
	case k8s.KindEKS:
		if err := c.Pool.Submit(awsNodeDaemonSetName, func(ctx context.Context) error {
			// Collect the 'kube-system/aws-node' DaemonSet.
			d, err := c.Client.GetDaemonSet(ctx, awsNodeDaemonSetNamespace, awsNodeDaemonSetName, metav1.GetOptions{})
			if err != nil {
				if k8sErrors.IsNotFound(err) {
					c.logDebug("DaemonSet %q not found in namespace %q - this is expected when running in ENI mode", awsNodeDaemonSetName, awsNodeDaemonSetNamespace)
					return nil
				}
				return fmt.Errorf("failed to collect daemonset %q in namespace %q: %w", awsNodeDaemonSetName, awsNodeDaemonSetNamespace, err)

View on GitHub (pinned to ac7b90affa)

Solutions

1. Re-run the sysdump with a larger --timeout value
2. Avoid interrupting the command; wait for graceful completion
3. Reduce scope (fewer namespaces/pods via flags) so tasks finish within the window
4. Check the wrapped error to confirm whether it is context.Canceled

Example fix

// before: fail the whole sysdump on submit error
return fmt.Errorf("failed to submit logs task for %q (%q): %w", p.Name, d.Name, err)
// after
c.logWarn("failed to submit logs task for %q (%q): %v", p.Name, d.Name, err)
return nil

if err := ctx.Err(); err != nil {
    return fmt.Errorf("cannot submit logs task, sysdump context done: %w", err)
}

if errors.Is(err, context.Canceled) || errors.Is(err, context.DeadlineExceeded) {
    // pool shutdown, not a log-fetch failure
}

if err := c.Pool.Submit(name, fn); err != nil {
    c.logWarn("skipping logs task for %q: %v", p.Name, err)
    return nil // or retry with a fresh pool
}

When it happens

Trigger: c.Pool.Submit("logs-<pod>-<container>", fn) returns non-nil: the pool's context is done (user abort, global timeout) or the pool rejected the task.

Common situations: Interrupting `cilium-cli sysdump` with Ctrl-C while many log tasks are queued; sysdump exceeding its deadline during large-cluster log collection.

Related errors

• failed to create sysdump collector: %w
• failed to dump Kubernetes version: %w
• failed to write event table: %w
• failed to collect Kubernetes pods summary: %w
• failed to collect Kubernetes services: %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/d647332097ae578e. Report an issue: GitHub.