gravitational/teleport · error

failed to set stdout mode: %w

Error message

failed to set stdout mode: %w

What it means

After computing the new output mode (ENABLE_VIRTUAL_TERMINAL_PROCESSING | DISABLE_NEWLINE_AUTO_RETURN), initTerminal calls winterm.SetConsoleMode on the stdout handle. If the Windows console API rejects the new mode, this wrapped error is returned. The console buffer exists but refuses the requested mode flags.

Source

Thrown at lib/client/terminal/terminal_windows.go:61

func initTerminal(input bool) (func(), error) {
	stdoutFd := int(syscall.Stdout)
	stdinFd := int(syscall.Stdin)

	oldOutMode, err := winterm.GetConsoleMode(uintptr(stdoutFd))
	if err != nil {
		return func() {}, fmt.Errorf("failed to retrieve stdout mode: %w", err)
	}

	oldInMode, err := winterm.GetConsoleMode(uintptr(stdinFd))
	if err != nil {
		return func() {}, fmt.Errorf("failed to retrieve stdout mode: %w", err)
	}

	newOutMode := oldOutMode | winterm.ENABLE_VIRTUAL_TERMINAL_PROCESSING | winterm.DISABLE_NEWLINE_AUTO_RETURN

	err = winterm.SetConsoleMode(uintptr(stdoutFd), newOutMode)
	if err != nil {
		return func() {}, fmt.Errorf("failed to set stdout mode: %w", err)
	}

	if input {
		newInMode := oldInMode
		newInMode &^= winterm.ENABLE_ECHO_INPUT
		newInMode &^= winterm.ENABLE_LINE_INPUT
		newInMode &^= winterm.ENABLE_MOUSE_INPUT
		newInMode &^= winterm.ENABLE_WINDOW_INPUT
		newInMode &^= winterm.ENABLE_PROCESSED_INPUT

		newInMode |= winterm.ENABLE_EXTENDED_FLAGS
		newInMode |= winterm.ENABLE_INSERT_MODE
		newInMode |= winterm.ENABLE_QUICK_EDIT_MODE
		newInMode |= winterm.ENABLE_VIRTUAL_TERMINAL_INPUT

		err = winterm.SetConsoleMode(uintptr(stdinFd), newInMode)
		if err != nil {
			// Attempt to reset the stdout mode before returning.

View on GitHub (pinned to 1283425b60)

Solutions

1. Update Windows to a build supporting virtual terminal processing (Windows 10 1607+), or run inside Windows Terminal.
2. Pre-check support: attempt GetConsoleMode on stdout; if SetConsoleMode with ENABLE_VIRTUAL_TERMINAL_PROCESSING fails, fall back to a non-VT rendering path or degrade gracefully instead of failing the session.
3. Ensure stdout is a genuine console buffer handle, not a wrapped/emulated handle provided by the host environment.
4. If the error persists in a specific terminal emulator, test in plain conhost to isolate emulator-specific restrictions.

Example fix

// before
cleanup, err := term.InitRaw(false)
if err != nil {
    return trace.Wrap(err)
}

// after
cleanup, err := term.InitRaw(false)
if err != nil {
    log.Warnf("VT output mode unavailable (%v); continuing without raw output", err)
    cleanup = func() {}
}

// Probe VT support before entering a real session:
mode, err := winterm.GetConsoleMode(uintptr(syscall.Stdout))
if err != nil {
    return errors.New("no console on stdout; VT output unavailable")
}
if err := winterm.SetConsoleMode(uintptr(syscall.Stdout), mode|winterm.ENABLE_VIRTUAL_TERMINAL_PROCESSING); err != nil {
    return errors.New("console host does not support VT processing")
}
winterm.SetConsoleMode(uintptr(syscall.Stdout), mode) // restore

func vtOutputSupported() bool {
    mode, err := winterm.GetConsoleMode(uintptr(syscall.Stdout))
    if err != nil {
        return false
    }
    return winterm.SetConsoleMode(uintptr(syscall.Stdout), mode|winterm.ENABLE_VIRTUAL_TERMINAL_PROCESSING) == nil
}

cleanup, err := t.InitRaw(input)
if err != nil && strings.Contains(err.Error(), "failed to set stdout mode") {
    log.Warn("VT output mode rejected; continuing without raw output")
    cleanup = func() {}
    return nil
}

When it happens

Trigger: Calling Terminal.InitRaw on Windows when SetConsoleMode(stdout, newOutMode) fails: the console host does not support ENABLE_VIRTUAL_TERMINAL_PROCESSING (legacy conhost pre-Windows-10-1607), the stdout handle is a console in an incompatible state, or the handle lacks permission to change its mode.

Common situations: Running on old Windows 10 builds (VT processing was opt-in before 1607) or legacy terminals where the flag is unsupported; running inside environments emulating a console (some CI agents, remote shells) that reject the flag; group-policy or terminal-software restricting console mode changes.

Related errors

• failed to retrieve stdout mode: %w
• failed to set stdin mode: %w
• a tncon session is already active
• unexpected nil response from GetAssertion
• unexpected nil response from MakeCredential

AI-assisted analysis of gravitational/teleport@1283425b60 (2026-09-02). Data as JSON: /api/errors/584898d7b4ea3903. Report an issue: GitHub.