kubernetes/kops · error

rewriting channels manifest: %w

Error message

rewriting channels manifest: %w

What it means

After downloading the kops-channels static pod manifest and remapping the addons tree locally, kops rewrites the manifest so the container's bootstrap-channel URL arg points at the local file:// copy. rewriteChannelsManifestForEnroll returns an error (e.g. the manifest bytes could not be parsed as a Pod, or re-serializing failed) and this wrapper annotates it.

Source

Thrown at pkg/commands/toolbox_enroll.go:958

				return nil, fmt.Errorf("parsing configStore.base %q: %w", cluster.Spec.ConfigStore.Base, err)
			}
			bootstrapChannelURL := configBase.Join("addons", "bootstrap-channel.yaml").Path()

			addonsPath := configBase.Join("addons").Path()
			if err := remapTree(&addonsPath, path.Join(targetDir, "addons")); err != nil {
				return nil, err
			}
			localAddons := addonsPath // remapTree mutated it in place to the on-host destination
			if err := remapFile(&nodeupConfig.ChannelsManifest, targetDir); err != nil {
				return nil, err
			}
			rewritten, err := rewriteChannelsManifestForEnroll(
				bootstrapData.NodeupScriptAdditionalFiles[nodeupConfig.ChannelsManifest],
				bootstrapChannelURL,
				localAddons,
			)
			if err != nil {
				return nil, fmt.Errorf("rewriting channels manifest: %w", err)
			}
			bootstrapData.NodeupScriptAdditionalFiles[nodeupConfig.ChannelsManifest] = rewritten
		}

		if nodeupConfig.ConfigStore != nil {
			if err := remapTree(&nodeupConfig.ConfigStore.Keypairs, path.Join(targetDir, "pki/etcd")); err != nil {
				return nil, err
			}
			if err := remapTree(&nodeupConfig.ConfigStore.Secrets, path.Join(targetDir, "pki")); err != nil {
				return nil, err
			}
		}

		nodeupConfigBytes, err := yaml.Marshal(nodeupConfig)
		if err != nil {
			return nil, fmt.Errorf("error converting nodeup config to yaml: %w", err)
		}
		// Not much reason to hash this, since we're reading it from the local file system

View on GitHub (pinned to 4c8573c808)

Solutions

1. Download the manifest from s3://<base>/addons/ and inspect it — restore a valid kops-channels Pod manifest (re-run `kops update cluster --yes` / upgrade to regenerate it).
2. Verify the file parses as a Kubernetes Pod (kubectl apply --dry-run=server or yamllint + kind: Pod check).
3. If corrupted during transfer, re-run the command after fixing connectivity/KMS permissions.
4. Align kops CLI version with the cluster version that wrote the manifest, then retry.

Example fix

// before: hand-edited manifest in the state store
$ kubectl get -f kops-channels.yaml
error: unable to recognize ... no kind registered
// after: regenerate the manifest
kops update cluster <name> --yes && kops toolbox enroll ...

// Before enroll: verify the channels manifest in the state store is a valid Pod YAML
aws s3 cp s3://<bucket>/<cluster>/addons/channels/... - | \
  yq -e '.kind == "Pod" and .spec.containers' >/dev/null && echo OK

func isChannelsPod(manifest []byte) bool {
    var pod corev1.Pod
    return yaml.Unmarshal(manifest, &pod) == nil && pod.Kind == "Pod"
}

if err := runToolboxEnroll(ctx, ...); err != nil {
    if strings.Contains(err.Error(), "rewriting channels manifest") ||
       strings.Contains(err.Error(), "parsing kops-channels manifest") {
        // regenerate the manifest from a good source, then retry once
        exec.Command("kops", "update", "cluster", clusterName, "--yes").Run()
        return runToolboxEnroll(ctx, ...)
    }
    return err
}

When it happens

Trigger: GetBootstrapData on a control-plane group with an s3:// ChannelsManifest, where the downloaded channels manifest (from NodeupScriptAdditionalFiles) fails yaml.Unmarshal into corev1.Pod ("parsing kops-channels manifest"), or k8scodecs.ToVersionedYaml fails during re-serialization — e.g. the object in the state store was overwritten with non-Pod YAML/JSON, is corrupt, or was written by an incompatible kops version.

Common situations: Someone edited or replaced the s3 addons/kops-channels manifest with arbitrary YAML; partial/corrupted upload truncated the manifest; state store holds a manifest from a much older kops whose Pod spec doesn't decode into the vendored corev1 type; KMS/network corruption during read.

Related errors

• failed to parse objects: %w
• parsing kops-channels manifest: %w
• error parsing addons or manifest: %v
• failed to parse objects: %w
• failed to parse apiVersion %q

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/998fab0a003ff6cf. Report an issue: GitHub.