kubernetes/kops · error

error reading etcd manifest %s: %v

Error message

error reading etcd manifest %s: %v

What it means

After resolving the VFS path, Build reads the etcd manifest contents via p.ReadFile(ctx); any storage-level failure (missing object, bad credentials, network error to S3/GCS) is wrapped in this error with the manifest name. The manifest content is required to write the etcd task on the master.

Source

Thrown at nodeup/pkg/model/manifests.go:49

	*NodeupModelContext
}

var _ fi.NodeupModelBuilder = &ManifestsBuilder{}

// Build creates tasks for copying the manifests
func (b *ManifestsBuilder) Build(c *fi.NodeupModelBuilderContext) error {
	ctx := c.Context()

	// Write etcd manifests (currently etcd <=> master)
	if b.IsMaster {
		for _, manifest := range b.NodeupConfig.EtcdManifests {
			p, err := vfs.Context.BuildVfsPath(manifest)
			if err != nil {
				return fmt.Errorf("error parsing path for etcd manifest %s: %v", manifest, err)
			}
			data, err := p.ReadFile(ctx)
			if err != nil {
				return fmt.Errorf("error reading etcd manifest %s: %v", manifest, err)
			}

			name := p.Base()
			name = strings.TrimSuffix(name, filepath.Ext(name))

			key := "etcd-" + name

			manifestPath := "/etc/kubernetes/manifests/" + key + ".manifest"

			c.AddTask(&nodetasks.File{
				Contents: fi.NewBytesResource(data),
				Mode:     s("0440"),
				Path:     manifestPath,
				Type:     nodetasks.FileType_File,
			})
		}
	}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Verify the manifest object still exists at the URL in EtcdManifests
2. Check the instance's IAM role has read access to the state-store bucket/object
3. Test bucket reachability from the instance (aws s3 cp / gsutil cp) and check VPC endpoints/DNS
4. Re-run kops update cluster --yes to republish manifests, then restart nodeup

Example fix

// before: instance profile lacks S3 read
// after: attach policy
{"Effect":"Allow","Action":["s3:GetObject"],"Resource":["arn:aws:s3:::my-cluster-bucket/*"]}

// before running nodeup, from the node:
// aws s3 cp s3://bucket/etcd.yaml -   (or gsutil cp)
// and check the instance role:
// aws sts get-caller-identity

data, err := p.ReadFile(ctx)
if err != nil {
    if isRetryable(err) { // network/timeout
        time.Sleep(5 * time.Second)
        data, err = p.ReadFile(ctx)
    }
    if err != nil {
        return fmt.Errorf("error reading etcd manifest %s: %v", manifest, err)
    }
}

When it happens

Trigger: p.ReadFile(ctx) fails during nodeup Build on a master — the etcd manifest object was deleted/moved in the storage bucket, nodeup's IAM credentials lack s3:GetObject/gcs access, or the bucket/region is unreachable.

Common situations: State store bucket emptied or lifecycle-policy-deleted manifests; IAM role missing read permission; wrong region/network outage from the instance; VPC without S3 endpoint.

Related errors

• error parsing path for etcd manifest %s: %v
• building path for %q: %w
• unexpected path type %T for %q
• error loading NodeupConfig %q: %v
• error reading addons from %q: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/a0a5ba221969e9c5. Report an issue: GitHub.