What causes "failed to delete network %s(%s): %w" in kubernetes/kops?

Fix the underlying hcloud error: verify HCLOUD_TOKEN is valid and has write scope, and check the wrapped message for conflicts.

kubernetes/kops · error

failed to delete network %s(%s): %w

Error message

failed to delete network %s(%s): %w

What it means

Wrapped error returned by kOps' Hetzner deleteNetwork when the hcloud API client's NetworkClient.Delete call fails while tearing down the cluster network during `kops delete cluster`. The original hcloud error (auth, rate limit, conflict, network still in use) is preserved via %w so callers can errors.Is/As it.

Source

Thrown at pkg/resources/hetzner/resources.go:245

	client := c.SSHKeyClient()
	sshKey := r.Obj.(*hcloud.SSHKey)
	_, err := client.Delete(context.TODO(), sshKey)
	if err != nil {
		return fmt.Errorf("failed to delete ssh key %s(%s): %w", r.Name, r.ID, err)
	}

	return nil
}

func deleteNetwork(cloud fi.Cloud, r *resources.Resource) error {
	klog.Infof("Deleting Network: %s(%s)", r.Name, r.ID)

	c := cloud.(hetzner.HetznerCloud)
	client := c.NetworkClient()
	network := r.Obj.(*hcloud.Network)
	_, err := client.Delete(context.TODO(), network)
	if err != nil {
		return fmt.Errorf("failed to delete network %s(%s): %w", r.Name, r.ID, err)
	}

	return nil
}

func deleteFirewall(cloud fi.Cloud, r *resources.Resource) error {
	klog.Infof("Deleting Firewall: %s(%s)", r.Name, r.ID)

	c := cloud.(hetzner.HetznerCloud)
	client := c.FirewallClient()
	firewall := r.Obj.(*hcloud.Firewall)
	_, err := client.Delete(context.TODO(), firewall)
	if err != nil {
		return fmt.Errorf("failed to delete firewall %s(%s): %w", r.Name, r.ID, err)
	}

	return nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

Fix the underlying hcloud error: verify HCLOUD_TOKEN is valid and has write scope, and check the wrapped message for conflicts.
Detach all servers, load balancers and routes from the network in the Hetzner Cloud Console, then retry `kops delete cluster`.
If the resource no longer exists (not-found), rerun with `--yes` again or use `kops delete cluster` after refreshing state; kOps treats some not-found cases as done on the next pass.
Wait for hcloud rate-limit/reset windows if the wrapped error is 429, then retry.

Example fix

// before: network in use by servers
client.Delete(context.TODO(), network) // fails: server(s) still attached
// after: delete servers (deleteServer runs earlier) or detach interfaces first
// ensure resource ordering: servers/volumes deleted before network
// or manually: hcloud network remove-route / detach in console, then rerun

Defensive patterns

Strategy: retry

Validate before calling

// preflight: check token and network usage before delete
func canDeleteNetwork(ctx context.Context, c *hcloud.Client, id int) error {
    net, _, err := c.Network.Get(ctx, strconv.Itoa(id))
    if err != nil { return err }
    if net == nil { return nil } // already gone
    if len(net.Servers) > 0 || len(net.LoadBalancers) > 0 {
        return fmt.Errorf("network %d still has %d servers / %d load balancers attached", id, len(net.Servers), len(net.LoadBalancers))
    }
    return nil
}

Type guard

func isHcloudNotFound(err error) bool {
    var h_err hcloud.Error
    return errors.As(err, &h_err) && h_err.Code == hcloud.ErrorCodeNotFound
}

Try / catch

if err := deleteNetwork(...); err != nil {
    var hErr hcloud.Error
    if errors.As(err, &hErr) && hErr.Code == hcloud.ErrorCodeNotFound {
        return nil // already deleted
    }
    if errors.As(err, &hErr) && hErr.Code == hcloud.ErrorCodeConflict {
        return retryAfterDetaching(err)
    }
    return err
}

Prevention

Always let kOps delete servers, volumes and load balancers before the network (its ordering does this; avoid out-of-band console deletions).
Rotate HCLOUD_TOKEN before starting a delete and confirm write scope.
Check `hcloud network describe` for attached resources prior to deletion.
Back off on 429s instead of hammering the API.

When it happens

Trigger: client.Delete(ctx, network) returns a non-nil error: invalid/expired hcloud token, network still attached to servers/load balancers (hcloud conflict), rate limiting (429), or network already deleted (hcloud not-found).

Common situations: Deleting a cluster whose servers or load balancers were removed out-of-band but still reference the network; stale KOPS_STATE_STORE credentials after token rotation; deleting the same cluster twice concurrently.

Related errors

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/0de5ff279801104f. Report an issue: GitHub.