hashicorp/nomad · error

failed to receive initial message: %v

Error message

failed to receive initial message: %v

What it means

ExecTaskStreaming is the plugin-server side handler for the Driver_ExecTaskStreaming RPC. The first gRPC message from the client must be received to learn the task/exec setup; this error wraps any failure of that initial server.Recv() call (transport closed, stream canceled, decode failure).

Source

Thrown at plugins/drivers/server.go:319

	}

	result, err := b.impl.ExecTask(req.TaskId, req.Command, timeout)
	if err != nil {
		return nil, err
	}
	resp := &proto.ExecTaskResponse{
		Stdout: result.Stdout,
		Stderr: result.Stderr,
		Result: exitResultToProto(result.ExitResult),
	}

	return resp, nil
}

func (b *driverPluginServer) ExecTaskStreaming(server proto.Driver_ExecTaskStreamingServer) error {
	msg, err := server.Recv()
	if err != nil {
		return fmt.Errorf("failed to receive initial message: %v", err)
	}

	if msg.Setup == nil {
		return fmt.Errorf("first message should always be setup")
	}

	if impl, ok := b.impl.(ExecTaskStreamingRawDriver); ok {
		return impl.ExecTaskStreamingRaw(server.Context(),
			msg.Setup.TaskId, msg.Setup.Command, msg.Setup.Tty,
			server)
	}

	d, ok := b.impl.(ExecTaskStreamingDriver)
	if !ok {
		return fmt.Errorf("driver does not support exec")
	}

	execOpts, errCh := StreamToExecOptions(server.Context(),

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Check the wrapped %v error: if it is context canceled/deadline exceeded, verify the caller's exec timeout and stream cancellation logic.
2. Ensure the client always sends the initial ExecTaskStreamingRequest with a Setup message immediately after opening the stream.
3. Verify the driver plugin binary is compatible with the Nomad client's plugin protocol version and is not crashing (check plugin stderr/logs).
4. Retry the exec request; transient gRPC disconnects between client and plugin are often temporary.

Example fix

// before
msg, err := server.Recv()
if err != nil {
	return fmt.Errorf("failed to receive initial message: %v", err)
}
// after
msg, err := server.Recv()
if err != nil {
	if server.Context().Err() != nil {
		return fmt.Errorf("exec stream canceled before initial message: %w", err)
	}
	return fmt.Errorf("failed to receive initial message: %w", err)
}

// Before initiating exec, ensure the plugin client is healthy
if err := clientPing(drvPlugin); err != nil {
	return fmt.Errorf("driver plugin unreachable before exec: %w", err)
}

err := srv.ExecTaskStreaming(ctx)
if err != nil && strings.Contains(err.Error(), "failed to receive initial message") {
	// inspect wrapped cause; if context.Canceled/DeadlineExceeded, retry with longer deadline
}

When it happens

Trigger: Calling driverPluginServer.ExecTaskStreaming when the client closes/cancels the stream before sending any message, the gRPC connection drops, or the initial ExecTaskStreamingRequest fails to deserialize.

Common situations: Client task/plugin crashes right after starting an exec stream; context cancellation or RPC deadline expiring before the first message; network interruption between Nomad client and driver plugin process; mismatched plugin protocol versions causing undecodable messages.

Related errors

• failed to receive initial message: %v
• first message should always be setup
• task driver does not support exec
• failed executing plugin %q for secret %q: %w
• CSI.ControllerListVolumes: plugin returned an invalid entry

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/b1136c141fcc3c90. Report an issue: GitHub.