juanfont/headscale · critical

loading nodes for RequestTags migration: %w

Error message

loading nodes for RequestTags migration: %w

What it means

The same RequestTags migration fails at ListNodes(tx): it must load every node (with Hostinfo JSON) to decide which nodes' historical RequestTags can be merged into the new tags column. Failure is a SELECT-level error on nodes - schema mismatch (e.g. tags column missing because the preceding rename migration mis-applied), permissions, locks, or resource exhaustion during the full-table load.

Source

Thrown at hscontrol/db/db.go:627

					if err != nil {
						log.Warn().Err(err).Msg("failed to load policy, skipping RequestTags migration (tags will be validated on node reconnect)")
						return nil
					}

					if len(policyData) == 0 {
						log.Info().Msg("no policy found, skipping RequestTags migration (tags will be validated on node reconnect)")
						return nil
					}

					// 2. Load users and nodes to create PolicyManager
					users, err := ListUsers(tx, nil)
					if err != nil {
						return fmt.Errorf("loading users for RequestTags migration: %w", err)
					}

					nodes, err := ListNodes(tx)
					if err != nil {
						return fmt.Errorf("loading nodes for RequestTags migration: %w", err)
					}

					// 3. Create PolicyManager (handles HuJSON parsing, groups, nested tags, etc.)
					polMan, err := policy.NewPolicyManager(policyData, users, nodes.ViewSlice())
					if err != nil {
						log.Warn().Err(err).Msg("failed to parse policy, skipping RequestTags migration (tags will be validated on node reconnect)")
						return nil
					}

					// 4. Process each node
					for _, node := range nodes {
						if node.Hostinfo == nil {
							continue
						}

						requestTags := node.Hostinfo.RequestTags
						if len(requestTags) == 0 {
							continue

View on GitHub (pinned to 565fd254d0)

Solutions

1. Read the wrapped error: 'no such column: tags' means the earlier rename step failed - reconcile schema/history before retrying
2. Upgrade during a quiet window with no peer/admin writes to nodes
3. Raise the container's memory limit for the upgrade of very large tailnets, or prune dead nodes first (headscale nodes delete) to shrink the scan
4. Retry startup after fixing connectivity; the migration transaction is atomic and replays safely

// Pre-flight for large tailnets: estimate migration memory cost
var nodeCount int
db.QueryRow("SELECT count(*) FROM nodes").Scan(&nodeCount)
if nodeCount > 50000 {
	log.Printf("large node count (%d); ensure ample memory for the RequestTags migration", nodeCount)
}

When it happens

Trigger: nodes table schema drift (e.g. tags column missing because migration 202511131445's earlier rename step failed), lock timeout on nodes, connection loss, or extremely large node counts exhausting container memory during the full-table scan.

Common situations: Big tailnets upgrading on memory-limited containers; databases restored with mismatched schema; concurrent writers during the migration window.

Related errors

• loading users for RequestTags migration: %w
• serializing merged tags for node %d: %w
• updating tags for node %d: %w
• foreign key constraints violated
• loading policy from database: %w

AI-assisted analysis of juanfont/headscale@565fd254d0 (2026-08-15). Data as JSON: /api/errors/55891a803b5f4802. Report an issue: GitHub.