Nvidia has little popularity among Linux users. It’d feel weird to not quote famous words by Linus Torvalds that it “has been the single worst company [the Linux developers have] ever dealt with”. While I’m not a kernel developer, my experience with their GPU driver as a user is so bad that I consider it to be the main cause of me switching to NixOS - because of easy rollbacks.
My AMD Radeon RX 580 was 4 years old last year. I never had a single issue with its driver as it is an in-tree kernel module and didn’t require installing any proprietary blob. My desktop setup is GNOME using Wayland session1 running on a HiDPI display.
To make a long story a bit shorter: at some point I needed a new GPU and I needed it fast. GeForce series 40 and Radeon RX 7000 series were just about to release their cards. I enjoyed my experience with AMD but was in a rush, Nvidia was releasing their products sooner, and I ordered RTX 4080 on a Black Friday sale.
Driver issues #
I hoped that Nvidia support for Linux has improved in recent years. This is true, to some extent. Generic Buffer Management has been supported since driver version 495 which made it possible to use any Wayland compositor. They have released an open-source kernel module (that still runs proprietary code). The gaming performance is almost on pair with Windows.
But then I encountered the following issues:
- No support for HiDPI TTY framebuffer as there is no fbdriver support. This also affects KMSCON.
- Modesetting via
nvidia_drm.modeset=1screwed up my flicker-free boot process (plymouth). - GNOME Display Manager (GDM) wouldn’t show an option to run Wayland session without the open-source kernel module that is considered to be less stable than the proprietary driver.
- GAMMA_LUT support is missing which disables shifting screen colors to red. This is known in GNOME as “night light” and in iOS as “night shift”.
- Many applications do not support Nvidia hardware video decoding (NVDEC).
Another thing that negatively surprised me is that Proton does not support features like ray-tracing or DLSS out of the box. Fortunately, this was easy to fix with a few flags2.
After reading Reddit for a few minutes hours it was obvious to me that
there is no easy fix for all of the above. Moreover, I enjoy using Wayland and
was saddened by a fact that wlroots, a highly popular dependency for writing
Wayland window managers, does not work well with Nvidia. And it seems that
work that aims to improve that is moving slow.
Solution #
Ten years ago one of major sources of pain for Linux users was a new feature used in laptops called “Nvidia Optimus”. It improved battery life of a computer by running most graphic computations on GPU integrated with CPU (iGPU) and offloading processing demanding applications, usually games, to a dedicated Nvidia GPU (dGPU). That is, the computer would run desktop environment on an iGPU but The Witcher 2 would render on dGPU with output image being sent back to iGPU memory so that it could be presented on a display.
Most people that I knew back then that used Linux as their primary operating system decided to disable their dedicated GPUs on BIOS level. Some, such as me, would use third-party software (“bumblebee”) to run selected software manually on dGPU. Such a setup would break very often. It was hell. Now it’s a new hope for PC desktop users.
Since 2016 Nvidia driver supports Linux PRIME offloading which is a successor to Optimus that supports various GPU vendors and has a much better support. I know it may not sound very intuitive, but it’s actually a great solution for the problems above.
Let’s run Linux on integrated GPU that has very good driver quality and offload gaming to the Nvidia card. I’ve tried this and the results are above expectations. There is no performance penalty that I’m aware of. All the issues mentioned before are mitigated. I’m almost certain that my power consumption is now on a bit lower level.
Here is a relevant set of settings that I use on my NixOS machine. Part of it is taken from NixOS wiki. I’m sure it’s not difficult to apply this on any other Linux OS.
1{ config, pkgs, ... }: {
2 # Enable Intel iGPU early in the boot process
3 boot.initrd.kernelModules = [ "i915" ];
4
5 environment.systemPackages =
6 # Running `nvidia-offload vlc` would run VLC with dGPU
7 let nvidia-offload = pkgs.writeShellScriptBin "nvidia-offload" ''
8 export __NV_PRIME_RENDER_OFFLOAD=1
9 export __NV_PRIME_RENDER_OFFLOAD_PROVIDER=NVIDIA-G0
10 export __GLX_VENDOR_LIBRARY_NAME=nvidia
11 export __VK_LAYER_NV_optimus=NVIDIA_only
12
13 exec "$@"
14 '';
15 in [ nvidia-offload ];
16
17 hardware.nvidia = {
18 # Drivers must be at verion 525 or newer
19 package = config.boot.kernelPackages.nvidiaPackages.beta;
20 prime = {
21 offload.enable = true; # Enable PRIME offloading
22 intelBusId = "PCI:0:2:0"; # lspci | grep VGA | grep Intel
23 nvidiaBusId = "PCI:1:0:0"; # lspci | grep VGA | grep NVIDIA
24 };
25 };
26
27 home-manager.users.jupblb = { lib, pkgs, ... }: {
28 # Overwrite steam.desktop shortcut so that is uses PRIME
29 # offloading for Steam and all its games
30 home.activation.steam = lib.hm.dag.entryAfter ["writeBoundary"] ''
31 $DRY_RUN_CMD sed 's/^Exec=/&nvidia-offload /' \
32 ${pkgs.steam}/share/applications/steam.desktop \
33 > ~/.local/share/applications/steam.desktop
34 $DRY_RUN_CMD chmod +x ~/.local/share/applications/steam.desktop
35 '';
36 };
37
38 services.xserver.videoDrivers = [ "nvidia" ];
39}
There are two disadvantages to this solution:
- An app must be running with
nvidia-offloadin order to utilise dGPU. - Number and resolution of displays connected to the PC is limited by the motherboard I/O.
Personally, I use a single 4k display and need the dedicated GPU for games only. If you are also so lucky and suffer from similar issues with Nvidia drivers then this is the way.