[MIR] nvidia-modprobe

Setuid binary that loads a kernel module... Security team will definitely have to weigh in on this. :)

See LP: #1361207

I'm surprised /etc/modules and udev rules aren't sufficient. Why don't the standard mechanisms for loading modules and creating device nodes work for nvidia drivers?

Thanks

nvidia-modprobe creates the /dev/nvidia-uvm device node and loads the nvidia_uvm module for a normal user, on demand.

Using clinfo as an example since it is in the archive, small, and wasn't compiled against anything Nvidia (it is built against ocl-icd-libopencl1). See LP: #1499996 for the relationship between ocl-icd-libopencl1 (an OpenCL ICD loader) and nvidia-opencl-icd-352 (an OpenCL ICD).

Without nvidia-modprobe installed, clinfo reports no devices:

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 17:41 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 17:41 /dev/nvidiactl

$ lsmod | grep nvidia
nvidia 10563584 41
drm 356352 3 nvidia

$ clinfo
Number of platforms 0

Without nvidia-modprobe installed, but running as root, clinfo finds the device:

$ sudo clinfo
Number of platforms 1
Platform Name NVIDIA CUDA
<snip>

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 17:41 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 17:41 /dev/nvidiactl
crw-rw-rw- 1 root root 247, 0 Oct 12 17:45 /dev/nvidia-uvm

$ lsmod | grep nvidia
nvidia_uvm 36864 0
nvidia 10563584 42 nvidia_uvm
drm 356352 3 nvidia

So the Nvidia OpenCL ICD is also capable of creating the device node and loading the nvidia_uvm module, provided it is run as root on its first run.

With nvidia-modprobe installed, it "just works" for the user:

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 16:59 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 16:59 /dev/nvidiactl

$ lsmod | grep nvidia
nvidia 10563584 41
drm 356352 3 nvidia

$ clinfo
Number of platforms 1
Platform Name NVIDIA CUDA
<snip>

$ ls -l /dev/nv*
crw-rw-rw- 1 root root 195, 0 Oct 12 16:59 /dev/nvidia0
crw-rw-rw- 1 root root 195, 255 Oct 12 16:59 /dev/nvidiactl
crw-rw-rw- 1 root root 247, 0 Oct 12 17:38 /dev/nvidia-uvm

$ lsmod | grep nvidia
nvidia_uvm 36864 0
nvidia 10563584 42 nvidia_uvm
drm 356352 3 nvidia

Running strace on clinfo shows that the Nvidia OpenCL ICD does the following:
- check if the nvidia_uvm module is loaded, if not, run '/sbin/modprobe' if we are root, otherwise run '/usr/bin/nvidia-modprobe'.
- check if /dev/nvidia-uvm exists, if it does not, try to create it, and if that fails run '/usr/bin/nvidia-modprobe' again.

The same is true of CUDA applications that are linked to libcuda or libcudart.

Alberto has already written udev rules, and they work, however, as far I can tell, they still require the user to 'sudo modprobe nvidia_uvm' before running any OpenCL or CUDA applications, or manually add nvidia_uvm to /etc/modules.

So the question is, do we want to try to load nvidia_uvm and create /dev/nvidia-uvm on demand, or should it just be done for all nvidia devices?

For desktops, nvidia_uvm could be loaded when nvidia is loaded, but for servers nvidia_uvm would need to be added to /etc/modules.

Just for reference, the nvidia_uvm module does not seem to be needed for normal desktop and gaming use.
With nvidia-modprobe installed, I opened Steam and the game Portal 2. Afterwards, nvidia_uvm had not been loaded, and /dev/nvidia-uvm had not been created.

Status changed to 'Confirmed' because the bug affects multiple users.

Graham, thanks for investigating and reporting back. If you're up for some legwork, I'm curious what SUSE and Red Hat do here; I don't expect them to want to use a setuid binary executable for this either and I wonder if they've got the module loading done differently or better.

Thanks

I can certainly have a look what SUSE do.

I've tried installing the NVIDIA drivers from the SUSE repository not the NVIDIA website in SLED 12.

nvidia-modprobe is installed, but without setuid being set:

~> which nvidia-modprobe
/usr/bin/nvidia-modprobe

~> ls -l /usr/bin/nvidia-modprobe
-rwxr-xr-x 1 root root 25432 Sep 3 10:57 /usr/bin/nvidia-modprobe

The devices are created and the modules are loaded on startup, not on demand:

~> ls -l /dev/nv*
crw-rw----+ 1 root video 195, 0 Nov 17 11:43 /dev/nvidia0
crw-rw----+ 1 root video 195, 255 Nov 17 11:43 /dev/nvidiactl
crw-rw----+ 1 root video 247, 0 Nov 17 11:43 /dev/nvidia-uvm
crw------- 1 root root 10, 144 Nov 17 11:43 /dev/nvram

~> lsmod | grep nvidia
nvidia_uvm 39162 0
nvidia 10573501 40 nvidia_uvm
drm 322623 2 nvidia

They have the following in /etc/modeprobe.d/50-nvidia.conf:

options nvidia NVreg_DeviceFileUID=0 NVreg_DeviceFileGID=33 NVreg_DeviceFileMode=0660
install nvidia PATH=$PATH:/bin:/usr/bin; /sbin/modprobe --ignore-install nvidia; /sbin/modprobe nvidia_uvm; test -c /dev/nvidia-uvm || mknod -m 660 /dev/nvidia-uvm c $(cat /proc/devices | while read major device; do if [ "$device" == "nvidia-uvm" ]; then echo $major; break; fi ; done) 0 && chown :video /dev/nvidia-uvm; test -c /dev/nvidiactl || mknod -m 660 /dev/nvidiactl c 195 255 && chown :video /dev/nvidiactl; devid=-1; for dev in $(ls -d /sys/bus/pci/devices/*); do vendorid=$(cat $dev/vendor); if [ "$vendorid" == "0x10de" ]; then class=$(cat $dev/class); classid=${class%%00}; if [ "$classid" == "0x0300" -o "$classid" == "0x0302" ]; then devid=$((devid+1)); test -c /dev/nvidia${devid} || mknod -m 660 /dev/nvidia${devid} c 195 ${devid} && chown :video /dev/nvidia${devid}; fi; fi; done

It certainly makes sense to simply set up things when the nvidia module is loaded, although I plan to do it with a udev rule. This will make nvidia-modprobe redundant on the desktop (i.e. where X is started by default).

On the server (and on the desktop with Mir), however, X is not started, and there is nothing that loads the nvidia module (with root privileges). A udev rule should probably try to load the driver when the hardware is available.

This should make nvidia-modprobe largely redundant in both use cases.

Marking this 'Won't Fix', we can track the udev rule in LP: #1361207.

Thanks for the update! Unassigning and unsubscribing the Ubuntu Security Team based on comment #13.

Graham, thanks for doing the legwork to confirm that a better solution is possible.

Alberto, thanks for making the better solution a reality.

Good work all around :)