Official tensorflow-gpu binaries (the one downloaded by pip or conda) are built with cuda 9.0, cudnn 7 since TF 1.5, and cuda 10.0, cudnn 7 since TF 1.13. These are written in the release notes. You have to use the matching version of cuda if using the official binaries.
If you don't like to change your Nvidia software, you can:
(1) Use a different version of TensorFlow
(2) Use non-official binaries built by others. e.g.: https://github.com/mind/wheels/releases, https://github.com/hadim/docker-tensorflow-builder#builds,
https://github.com/inoryy/tensorflow-optimized-wheels
(3) Build the binaries by yourself from source with your version of Nvidia software.