kubernetes/kops · error

failed to start SSH session: %w

Error message

failed to start SSH session: %w

What it means

This error wraps a failure to create a new SSH session on an already-established SSH client connection (sshClient.NewSession()). kOps' toolbox enroll uses this to run remote commands (runScript, getHostname) on a target host. It indicates the SSH connection exists but cannot open a channel/session, typically because the connection dropped or the server refused the channel.

Source

Thrown at pkg/commands/toolbox_enroll.go:391

	scriptCommand := "/bin/bash " + scriptPath
	return s.runCommand(ctx, scriptCommand, options)
}

// CommandOutput holds the results of running a command.
type CommandOutput struct {
	Stdout bytes.Buffer
	Stderr bytes.Buffer
}

// ExecOptions holds options for running a command remotely.
type ExecOptions struct {
	Echo bool
}

func (s *SSHHost) runCommand(ctx context.Context, command string, options ExecOptions) (*CommandOutput, error) {
	session, err := s.sshClient.NewSession()
	if err != nil {
		return nil, fmt.Errorf("failed to start SSH session: %w", err)
	}
	defer session.Close()

	output := &CommandOutput{}

	session.Stdout = &output.Stdout
	session.Stderr = &output.Stderr

	if options.Echo {
		// We send both to stderr, so we don't "corrupt" stdout
		session.Stdout = io.MultiWriter(os.Stderr, session.Stdout)
		session.Stderr = io.MultiWriter(os.Stderr, session.Stderr)
	}
	if s.sudo {
		command = "sudo " + command
	}
	if err := session.Run(command); err != nil {
		return output, fmt.Errorf("error running command %q: %w", command, err)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Verify the host is reachable and SSH daemon is running (ssh user@host 'echo ok')
2. Re-establish the SSH connection and retry the command (transient failures are common)
3. Check sshd MaxSessions/MaxStartups on the target and raise if enrolling many hosts concurrently
4. Check for NAT/firewall idle timeouts and enable SSH keepalives

Example fix

// before
output, err := host.runCommand(ctx, "hostname", opts) // fails once connection dropped
// after
if err != nil && isSSHSessionError(err) {
    host.reconnect(ctx) // re-dial sshClient
    output, err = host.runCommand(ctx, "hostname", opts)
}

// before enrolling
out, err := exec.Command("ssh", "-o", "BatchMode=yes", "-o", "ConnectTimeout=5", user+"@"+host, "echo ok").CombinedOutput()
if err != nil { return fmt.Errorf("SSH unreachable for %s: %v: %s", host, err, out) }

func isSSHSessionError(err error) bool {
    return err != nil && strings.Contains(err.Error(), "failed to start SSH session")
}

out, err := host.runCommand(ctx, cmd, opts)
if isSSHSessionError(err) {
    host.reconnect(ctx)
    out, err = host.runCommand(ctx, cmd, opts)
}

When it happens

Trigger: Calling SSHHost.runCommand (directly or via runScript/getHostname) when the underlying TCP connection to the host has been reset/timed out, the SSH server has hit MaxSessions, or the server closed the connection.

Common situations: Long-running enroll workflows where an idle SSH connection was reaped by a firewall/NAT; SSH server MaxSessions exhausted; host rebooted mid-enroll; network flakiness to the node.

Related errors

• failed to SSH to %q (with user %q): %w
• connecting: %w
• unable to SSH to %q: %v
• error dialing tcp %s: %w
• error creating sftp client: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/158c5018e5769b55. Report an issue: GitHub.