Installing nvidia-driver-510-server removes nvidia-cuda-dev

On Ubuntu 22.04, installing first `nvidia-cuda-dev` and then `nvidia-driver-510-server` leads to `nvidia-cuda-dev` being removed (or at least most packages it installed). The other way the same happens too - installing `nvidia-driver-510-server` first and then `nvidia-cuda-dev` removes stuff from the driver. I guess this is related to https://bugs.launchpad.net/ubuntu/+source/nvidia-cuda-toolkit/+bug/1963976 but I'm not sure if it's the same.

Is there any workaround for this? I need both packages - `nvidia-cuda-dev` for `ffmpeg` to run (which is compiled with nvidia support) and `nvidia-driver-510-server` so that the card is actually used.