NVIDIA and Wayland

· That blog you have just entered

TL;DR: NVIDIA Linux drivers still suck and don’t work well with Wayland. This can be mitigated by using iGPU with dGPU offloading.

Nvidia has little popularity among Linux users. It’d feel weird to not quote famous words by Linus Torvalds that it “has been the single worst company [the Linux developers have] ever dealt with”. While I’m not a kernel developer, my experience with their GPU driver as a user is so bad that I consider it to be the main cause of me switching to NixOS - because of easy rollbacks.

My AMD Radeon RX 580 was 4 years old last year. I never had a single issue with its driver as it is an in-tree kernel module and didn’t require installing any proprietary blob. My desktop setup is GNOME using Wayland session1 running on a HiDPI display.

To make a long story a bit shorter: at some point I needed a new GPU and I needed it fast. GeForce series 40 and Radeon RX 7000 series were just about to release their cards. I enjoyed my experience with AMD but was in a rush, Nvidia was releasing their products sooner, and I ordered RTX 4080 on a Black Friday sale.

# Driver issues

I hoped that Nvidia support for Linux has improved in recent years. This is true, to some extent. Generic Buffer Management has been supported since driver version 495 which made it possible to use any Wayland compositor. They have released an open-source kernel module (that still runs proprietary code). The gaming performance is almost on pair with Windows.

But then I encountered the following issues:

Another thing that negatively surprised me is that Proton does not support features like ray-tracing or DLSS out of the box. Fortunately, this was easy to fix with a few flags2.

After reading Reddit for a few minutes hours it was obvious to me that there is no easy fix for all of the above. Moreover, I enjoy using Wayland and was saddened by a fact that wlroots, a highly popular dependency for writing Wayland window managers, does not work well with Nvidia. And it seems that work that aims to improve that is moving slow.

# Solution

Ten years ago one of major sources of pain for Linux users was a new feature used in laptops called “Nvidia Optimus”. It improved battery life of a computer by running most graphic computations on GPU integrated with CPU (iGPU) and offloading processing demanding applications, usually games, to a dedicated Nvidia GPU (dGPU). That is, the computer would run desktop environment on an iGPU but The Witcher 2 would render on dGPU with output image being sent back to iGPU memory so that it could be presented on a display.

Most people that I knew back then that used Linux as their primary operating system decided to disable their dedicated GPUs on BIOS level. Some, such as me, would use third-party software (“bumblebee”) to run selected software manually on dGPU. Such a setup would break very often. It was hell. Now it’s a new hope for PC desktop users.

Since 2016 Nvidia driver supports Linux PRIME offloading which is a successor to Optimus that supports various GPU vendors and has a much better support. I know it may not sound very intuitive, but it’s actually a great solution for the problems above.

Let’s run Linux on integrated GPU that has very good driver quality and offload gaming to the Nvidia card. I’ve tried this and the results are above expectations. There is no performance penalty that I’m aware of. All the issues mentioned before are mitigated. I’m almost certain that my power consumption is now on a bit lower level.

Here is a relevant set of settings that I use on my NixOS machine. Part of it is taken from NixOS wiki. I’m sure it’s not difficult to apply this on any other Linux OS.

 1{ config, pkgs, ... }: {
 2  # Enable Intel iGPU early in the boot process
 3  boot.initrd.kernelModules = [ "i915" ];
 5  environment.systemPackages =
 6    # Running `nvidia-offload vlc` would run VLC with dGPU
 7    let nvidia-offload = pkgs.writeShellScriptBin "nvidia-offload" ''
 8      export __NV_PRIME_RENDER_OFFLOAD=1
10      export __GLX_VENDOR_LIBRARY_NAME=nvidia
11      export __VK_LAYER_NV_optimus=NVIDIA_only
13      exec "$@"
14    '';
15    in [ nvidia-offload ];
17  hardware.nvidia = {
18    # Drivers must be at verion 525 or newer
19    package = config.boot.kernelPackages.nvidiaPackages.beta;
20    prime   = {
21      offload.enable = true;        # Enable PRIME offloading
22      intelBusId     = "PCI:0:2:0"; # lspci | grep VGA | grep Intel
23      nvidiaBusId    = "PCI:1:0:0"; # lspci | grep VGA | grep NVIDIA
24    };
25  };
27  home-manager.users.jupblb = { lib, pkgs, ... }: {
28    # Overwrite steam.desktop shortcut so that is uses PRIME
29    # offloading for Steam and all its games
30    home.activation.steam = lib.hm.dag.entryAfter ["writeBoundary"] ''
31      $DRY_RUN_CMD sed 's/^Exec=/&nvidia-offload /' \
32        ${pkgs.steam}/share/applications/steam.desktop \
33        > ~/.local/share/applications/steam.desktop
34      $DRY_RUN_CMD chmod +x ~/.local/share/applications/steam.desktop
35    '';
36  };
38  services.xserver.videoDrivers = [ "nvidia" ];

There are two disadvantages to this solution:

  1. An app must be running with nvidia-offload in order to utilise dGPU.
  2. Number and resolution of displays connected to the PC is limited by the motherboard I/O.

Personally, I use a single 4k display and need the dedicated GPU for games only. If you are also so lucky and suffer from similar issues with Nvidia drivers then this is the way.

  1. It’s better than X11 because of the tear-free experience. ↩︎

  2. My Steam launch options secret sauce is PROTON_ENABLE_NVAPI=1 VKD3D_CONFIG=dxr PROTON_HIDE_NVIDIA_GPU=0↩︎

Have a question? Email me! :)