juanfont/headscale · error · ErrMemoryLimitViolations

test failed: %d %w

Error message

test failed: %d %w

What it means

Returned after a test run completes when stats collection is enabled and one or more monitored containers exceeded their configured memory limits. It wraps the `ErrMemoryLimitViolations` sentinel ('container(s) exceeded memory limits') and prefixes the violation count, so `errors.Is(err, ErrMemoryLimitViolations)` matches. This is a test-quality gate, not an infrastructure failure: the test itself may have passed.

Source

Thrown at cmd/hi/docker.go:162

		log.Printf("Warning: failed to extract artifacts from containers: %v", err)
	}

	// Always list control files regardless of test outcome
	listControlFiles(logsDir)

	// Print stats summary and check memory limits if enabled
	if config.Stats && statsCollector != nil {
		violations := statsCollector.PrintSummaryAndCheckLimits(config.HSMemoryLimit, config.TSMemoryLimit)
		if len(violations) > 0 {
			log.Printf("MEMORY LIMIT VIOLATIONS DETECTED:")
			log.Printf("=================================")

			for _, violation := range violations {
				log.Printf("Container %s exceeded memory limit: %.1f MB > %.1f MB",
					violation.ContainerName, violation.MaxMemoryMB, violation.LimitMB)
			}

			return fmt.Errorf("test failed: %d %w", len(violations), ErrMemoryLimitViolations)
		}
	}

	shouldCleanup := config.CleanAfter && (!config.KeepOnFailure || exitCode == 0)
	if shouldCleanup {
		if config.Verbose {
			log.Printf("Running post-test cleanup for run %s...", runID)
		}

		cleanErr := cleanupAfterTest(ctx, cli, resp.ID, runID)

		if cleanErr != nil && config.Verbose {
			log.Printf("Warning: post-test cleanup failed: %v", cleanErr)
		}

		// Clean up artifacts from successful tests to save disk space in CI
		if exitCode == 0 {
			if config.Verbose {

View on GitHub (pinned to 565fd254d0)

Solutions

1. Look at the printed summary table to see which container (hs vs ts) and by how much it exceeded.
2. Profile the offending component — for headscale, capture a heap profile of the test run and look at mapper/state allocations.
3. If the limit was set unrealistically low, raise HSMemoryLimit/TSMemoryLimit in the run config.
4. Re-run to rule out measurement noise from parallel Docker activity.

// before gating on it, sanity-check the limits are plausible vs the image floor
if config.HSMemoryLimit < 100 { // MB; headscale needs more than this
    return fmt.Errorf("HSMemoryLimit %v is below plausible usage", config.HSMemoryLimit)
}

if err := runDockerTest(ctx, config); err != nil {
    if errors.Is(err, ErrMemoryLimitViolations) {
        // treat as a performance regression: profile headscale heap,
        // do not retry — the run already completed and measured
    }
}

When it happens

Trigger: Running `hi run` with `--stats` (config.Stats true) plus HSMemoryLimit/TSMemoryLimit, where `statsCollector.PrintSummaryAndCheckLimits` finds max recorded memory above a limit for the headscale or tailscale containers.

Common situations: Introducing a memory leak or enlarged NodeStore snapshot in hscontrol; lowering the configured limit below real usage; noisy neighbor inflating container memory during parallel runs.

Related errors

• ensuring image availability: %w
• creating container: %w
• starting container: %w
• executing test: %w
• %w: exit code %d

AI-assisted analysis of juanfont/headscale@565fd254d0 (2026-08-15). Data as JSON: /api/errors/400655358efe93cc. Report an issue: GitHub.