深度学习环境搭建(Ubuntu16.04+GTX1080Ti+CUDA8.0+Cudnn6.0+TensorFlow+Caffe2(Pytorch))

2022-12-01,,,,

OS System:Ubuntu16.04

GPU Device:GTX1080Ti

Softwares:CUDA8.0、Cudnn6.0、TensorFlow(1.4.0)、Caffe2(1.0.0)


一、win10下安装Ubuntu16.04(双系统)

1、Linux分区方案

(Lagency+MBR)
/boot 512M
swap 16GB(本机物理内存为32GB)
/ 30GB or 35GB
/home 余下的(越多越好) (UEFI+GPT)
efi 512M
swap 16GB(本机物理内存为32GB)
/ 30GB or 35GB
/home 余下的(越多越好) swap大小设置参考:
4GB of RAM requires a minimum of 2GB of swap space
4GB to 16GB RAM requires a minimum of 4GB of swap space
16GB to 64GB of RAM requires a minimum of 8GB of swap space
64GB to 256GB of RAM requires a minimum of 16GB of swap space

2、系统引导

Lagency+MBR:win10下使用EasyBCD添加Ubuntu引导(Grub2)

UEFI+GPT:开机按快捷键选择BOOT Menu;使用Ubuntu的Grub引导;win10下使用rEFInd引导多系统

* win10下使用rEFInd步骤:

1. 官网下载zip包

2. 打开管理员命令行

3. 输入 mountvol x: /s (挂载ESP分区到x盘)

4. 把压缩包内refind文件夹复制到 x:\EFI 目录下

5. 把x:\EFI\refind\refind.conf-sample重命名为refind.conf

6. 命令行输入 bcdedit /set {bootmgr} path \EFI\refind\refind_x64.efi

7. 重启电脑

ref: https://blog.csdn.net/qf0129/article/details/78143749

二、显卡驱动、CUDA及Cudnn安装

1、安装GTX1080Ti显卡驱动

0)到NVIDIA官网查询适配的显卡驱动版本信息或下载

1)禁用自带显卡驱动nouveau

sudo gedit /etc/modprobe.d/blacklist.conf
add in the last line: blacklist nouveau
sudo update-initranfs –u
reboot
lsmod | grep nouveau //make sure nouveau is disabled, nothing will be printed out

2)run文件安装 或 命令行安装

#1 run文件安装

sudo apt-get remove --purge nvidia-*
cd Downloads
sudo service lightdm stop
ctrl + alt + f1, login by name and passwd
sudo chmod a+x NVIDIA-Linux-x86_64-375.26.run
sudo ./NVIDIA-Linux-x86_64-375.26.run --no-x-check --no-nouveau-check --no-opengl-files
//
–no-opengl-files 只安装驱动文件,不安装OpenGL文件,不加会导致循环登录
–no-check 安装驱动时不检查X服务,可省略
–no-nouveau-check 安装驱动时不检查nouveau,可省略
sudo service lightdm start

#2 命令行安装(推荐)

sudo apt-get remove --purge nvidia-*
sudo service lightdm stop
ctrl + alt + f1, login by name and passwd
sudo add-apt-repository ppa:graphics-drivers
sudo apt-get update
sudo apt-get install nvidia-375
sudo service lightdm start

2、安装CUDA8.0及Cudnn6.0

ref: https://www.cnblogs.com/wmxfd/p/installation_of_nvidia_graphics_driver_and_cuda8_and_cudnn6.html

三、TensorFlow安装

1、使用virtualenv虚拟环境安装,避免影响系统自带Python环境,使用Python3

/home目录下:
sudo apt update
sudo apt install python3-dev python3-pip
sudo pip3 install -U virtualenv # system-wide install
virtualenv --system-site-packages -p python3 ./venv_tf_p3
source ./venv_tf_p3/bin/activate
pip install --upgrade pip
pip install tensorflow-gpu==1.4
python -c "import tensorflow as tf; print(tf.__version__)" //for test
# pip安装速度慢时需要修改pip源
在home/用户名/目录下创建.pip文件夹
cd .pip
创建pip.conf文件,并输入以下内容:
[global]
timeout = 6000
index-url = http://mirrors.aliyun.com/pypi/simple/
trusted-host = mirrors.aliyun.com

2、安装jupyter notebook并添加virtualenv运行环境

source ./venv_tf_p3/bin/activate
1、安装jupyter notebook
pip install jupyter
pip install ipykernel
2、为jupyter添加kernel
python -m ipykernel install --user --name=venv_tf_p3
3、运行
jupyter notebook

四、Caffe2安装

1、使用virtualenv虚拟环境安装,避免影响系统自带Python环境,使用Python2

/home目录下:
sudo apt update
sudo apt install python-dev python-pip
sudo pip install –U virtualenv # system-wide install //安装依赖
sudo apt-get install -y --no-install-recommends \
build-essential \
git \
libgoogle-glog-dev \
libgtest-dev \
libiomp-dev \
libleveldb-dev \
liblmdb-dev \
libopencv-dev \
libopenmpi-dev \
libsnappy-dev \
libprotobuf-dev \
openmpi-bin \
openmpi-doc \
protobuf-compiler
sudo apt-get install -y --no-install-recommends \
libgflags-dev \
cmake sudo apt install graphviz python-tk virtualenv --system-site-packages -p python2.7 ./venv_cf_p2
source ./venv_cf_p2/bin/activate
pip install --upgrade pip //安装依赖
pip install --user \
future \
numpy \
protobuf \
typing \
hypothesis \
pyyaml \
pydot
//可选库安装
pip install --user \
flask \
requests \
scikit-image \
scipy \
tornado
pip install --user matplotlib==2.0.2 //这里需装旧版本的matplotlib,否则在导入matplotlib时出现 :ImportError: No module named functools_lru_cache git clone https://github.com/pytorch/pytorch.git && cd pytorch
git submodule update --init --recursive
export USE_LMDB=1 //声明环境变量,编译LMDB,MNIST例程用到
export USE_OPENCV=1 //声明环境变量,编译OpenCV
python setup.py install //环境变量设置
export PYTHONPATH=/usr/local:$PYTHONPATH
export PYTHONPATH=$home/pytorch/build:$PYTHONPATH //change $home to you home path, such as "/home/john"
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
add in ~/.bashrc
source ~/.bashrc //测试是否安装成功
cd ~ && python -c 'from caffe2.python import core' 2>/dev/null && echo "Success" || echo "Failure"
终端输出Success即可

[ 在pytorch目录下运行另一个测试命令:python caffe2/python/operator_test/activation_ops_test.py,没有输出期望结果,但对实际的GPU调用没有影响 ]

2、安装jupyter notebook并添加virtualenv运行环境

[ 若按照(三、2、)方法安装会出现以下错误,估计是jupyter对Python2的兼容问题 ]

[ 解决:直接指定版本安装 ]

source ./venv_cf_p2/bin/activate
1、安装jupyter notebook
pip install jupyter-console==5.2.0 jupyter-client==5.2.1 jupyter-core==4.4.0 jupyter==1.0.0 ipython==5.2.0 ipykernel==4.10.0
2、为jupyter添加kernel
python -m ipykernel install --user --name=venv_cf_p2
3、运行
jupyter notebook

参考:

Caffe2官方:https://caffe2.ai/docs/getting-started.html?platform=ubuntu&configuration=compile

更新时间:

2018/10/23

深度学习环境搭建(Ubuntu16.04+GTX1080Ti+CUDA8.0+Cudnn6.0+TensorFlow+Caffe2(Pytorch))的相关教程结束。

《深度学习环境搭建(Ubuntu16.04+GTX1080Ti+CUDA8.0+Cudnn6.0+TensorFlow+Caffe2(Pytorch)).doc》

下载本文的Word格式文档,以方便收藏与打印。