kubernetes/kops · error

reading kops-channels manifest %s: %w

Error message

reading kops-channels manifest %s: %w

What it means

After successfully parsing the VFS path, readChannelsManifest calls p.ReadFile(ctx) to fetch the kops-channels manifest bytes from the state store. Any read failure (missing object, denied credentials, network error, backend outage) is wrapped with this message, aborting the nodeup Build for master nodes.

Source

Thrown at nodeup/pkg/model/channels.go:95

	c.AddTask(&nodetasks.File{
		Path:     channelsManifestPath,
		Contents: fi.NewBytesResource(manifest),
		Type:     nodetasks.FileType_File,
	})
	return nil
}

// readChannelsManifest fetches the cloudup-built manifest and applies node-local SELinux
// decoration when needed. Otherwise it passes the bytes through unchanged.
func (b *ChannelsBuilder) readChannelsManifest(c *fi.NodeupModelBuilderContext) ([]byte, error) {
	ctx := c.Context()
	p, err := vfs.Context.BuildVfsPath(b.NodeupConfig.ChannelsManifest)
	if err != nil {
		return nil, fmt.Errorf("parsing path for kops-channels manifest %s: %w", b.NodeupConfig.ChannelsManifest, err)
	}
	data, err := p.ReadFile(ctx)
	if err != nil {
		return nil, fmt.Errorf("reading kops-channels manifest %s: %w", b.NodeupConfig.ChannelsManifest, err)
	}

	// SELinux is per-IG via containerdConfig, so the decoration can only be applied at nodeup.
	// Skip the parse/reserialize round-trip when there's nothing to add.
	if b.NodeupConfig.ContainerdConfig == nil || !b.NodeupConfig.ContainerdConfig.SeLinuxEnabled {
		return data, nil
	}
	pod := &v1.Pod{}
	if err := yaml.Unmarshal(data, pod); err != nil {
		return nil, fmt.Errorf("parsing kops-channels manifest: %w", err)
	}
	kubemanifest.AddHostPathSELinuxContext(pod, b.NodeupConfig)
	out, err := k8scodecs.ToVersionedYaml(pod)
	if err != nil {
		return nil, fmt.Errorf("re-marshaling kops-channels manifest: %w", err)
	}
	return out, nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Verify the manifest object exists at the ChannelsManifest path in the state store (aws s3 ls / gsutil ls) and re-run 'kops update cluster' if missing.
2. Check the node's cloud credentials/IAM permissions to read the state store bucket.
3. Test connectivity from the node to the backend (S3/GCS endpoint, DNS resolution).
4. Re-run nodeup once backend access is restored; wrap checks in a pre-flight read of the path.

// pre-flight: confirm the object is readable before nodeup
p, err := vfs.Context.BuildVfsPath(cfg.ChannelsManifest)
if err == nil {
	if _, err := p.ReadFile(context.Background()); err != nil {
		log.Printf("warning: channels manifest not readable: %v", err)
	}
}

if err := nodeupRun(ctx); err != nil && strings.Contains(err.Error(), "reading kops-channels manifest") {
	// transient backend/network issue: backoff and retry
	time.Sleep(30 * time.Second)
	return nodeupRun(ctx)
}

When it happens

Trigger: Build() on a master node; VFS path is valid but ReadFile fails because the manifest object does not exist, the node lacks IAM/credentials for the state store, there is a network outage, or the object is not readable.

Common situations: S3/GCS bucket deleted or object removed; instance profile lacks s3:GetObject on the state bucket; DNS or egress outage on the node; manifest never uploaded because 'kops update cluster' was interrupted.

Related errors

• parsing path for kops-channels manifest %s: %w
• keyset %q not found
• error trying to locate asset %q: %v
• error downloading file %q: %v
• error reading state store: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/ed618087535de08b. Report an issue: GitHub.