Installing TensorFlow GPU from Source on Ubuntu

In this tutorial I will be going through the process of building the latest TensorFlow from sources for Ubuntu 16.04 using Python3.

In order to use TensorFlow with GPU support you must have a Nvidia graphic card with a minimum compute capability of 3.0.

This is going to be a long procedure! So let’s brace ourselves and dive into it!

TensorFlow Meme

I am going to assume you know some of the basics of using a terminal in Linux (even if you don’t know, it isn’t that hard just closely follow each step).

TensorFlow Python and other Dependencies Installation

Running the following will take care of all of the dependencies:

$ sudo apt-get install python3-numpy python3-dev python3-pip python3-wheel python3-virtualenv libcurl3-dev libcupti-dev openjdk-8-jdk git

Install Nvidia Drivers

1. Determine the latest version of Nvidia driver available for your graphics card

a. Visit the graphics drivers PPA homepage here and determine the latest versions of Nvidia drivers available which is ‘nvidia-384’ at the time of writing this blog.
b. Verify that your graphics card is capable of running the latest drivers. You can search on this link to determine if your graphics card is supported by a driver version. Don’t be so particular about the version part after the dot (after nvidia-384.xxx), just make sure you’re supported on the main version 384.

2. Remove older Nvidia driver

​ If your graphic is supported, you can go ahead and remove all previously installed Nvidia drivers on your system. Enter the following command in terminal.

$ sudo apt-get purge nvidia*

3. Add the graphics drivers PPA

$ sudo add-apt-repository ppa:graphics-drivers

$ sudo apt-get update

4. Install (and activate) the latest Nvidia graphics drivers. Enter the following command to install the version of Nvidia graphics supported by your graphics card

$ sudo apt-get install nvidia-384

5. Reboot your computer for the new driver to kick-in.

You can check your installation status with the following command

$ lsmod | grep nvidia

If there is no output, then your installation has probably failed. It is also possible that the driver is not available in your system’s driver database. You can run the following command to check if your system is running on the open source driver nouveau. If the output is negative for nouveau, then all is well with your installation.

$ lsmod | grep nouveau

6. You should prevent automatic updates that might break the drivers (This has caused a lot of problems for me in the past).

$ sudo apt-mark hold nvidia-384

Install Nvidia CUDA Toolkit 8.0

To install the Nvidia Toolkit  download base installation .run file from Nvidia website.  MAKE SURE YOU SAY NO TO INSTALLING NVIDIA DRIVERS! (As you have already installed in the NVIDIA drivers) Also make sure you select yes to creating a symbolic link to your cuda directory.

Go to your download directory or the place you downloaded to:
$ cd ~/Downloads

Install it:
$ sudo sh cuda_8.0.61_375.26_linux.run --override --silent --toolkit

This will install cuda into: /usr/local/cuda

Install CudNN v6.0 for Cuda 8.0

To install CudNN download cudNN v6.0 for Cuda 8.0 from Nvidia website and extract into /usr/local/cuda via:
$ tar -xzvf cudnn-8.0-linux-x64-v6.0.tgz
$ sudo cp cuda/include/cudnn.h /usr/local/cuda/include
$ sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

Then update your bash file:
$ gedit ~/.bashrc

This will open your bash file in a text editor which you will scroll to the bottom and add these lines:

export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/extras/CUPTI/lib64"
export CUDA_HOME=/usr/local/cuda

Once you save and close the text file you can return to your original terminal and type this command to reload your .bashrc file:
$ source ~/.bashrc

Note for those using zsh: You will have to appropriately make the above changes to the .zshrc file.

Install Bazel

Now all of the NVIDIA stuff has been done! You’re half way there! Good job!

Now time to install Bazel:

$ echo "deb [arch=amd64] http://storage.googleapis.com/bazel-apt stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list
$ curl https://bazel.build/bazel-release.pub.gpg | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install bazel
$ sudo apt-get upgrade bazel

Clone Tensorflow

$ cd ~
$ git clone https://github.com/tensorflow/tensorflow

Creating a Virtual Environment

Before moving forward it is highly recommended that you create a virtual environment. TensorFlow will be installed in this virtual environment!

First we need to create a directory to contain all the environments.

$ sudo mkdir ~/virtualenvs

Now by using the virtualenv command, the virtual environment can be created:

$ sudo virtualenv --system-site-packages -p python3 ~/virtualenvs/tensorflow

Environment Activation

Up to now, the virtual environment named tensorflow has been created. For environment activation, the following must be done:

$ source ~/virtualenvs/tensorflow/bin/activate
However the command is too verbose!

Alias

The solution is to use an alias to make life easy! Let’s execute the following command:
$ echo 'alias tensorflow="source $HOME/virtualenvs/tensorflow/bin/activate"' >> ~/.bash_aliases
$ bash
After running the previous command, please close and open terminal again. Now by running the following simple script, the tensorflow environment will be activated.
$ tensorflow

For environment deactivation, the following must be executed:

$ deactivate

Note for zsh users: Add the appropriate alias (alias tensorflow=”source $HOME/virtualenvs/tensorflow/bin/activate”) to the .zshrc file.  

Configure TensorFlow Installation

$ cd ~/tensorflow
I highly recommend checking-out to the latest branch rather than master.
$ git checkout r1.6

Now run:
$ ./configure

WARNING:

The virtual environment must be activated before running the ./configure script

The flags of the configuration are of great importance because they determine how well and compatible the TensorFlow will be installed!!

To see what flags to enter after executing the above command please see this link.

EXCEPT FOR THE FOLLOWING:

Please specify the location of python. [Default is /usr/bin/python]:

/usr/bin/python3.5

Please input the desired Python library path to use. Default is [/usr/local/lib/python2.7/dist-packages]:

/usr/local/lib/python3.5/dist-packages

Do you wish to build TensorFlow with CUDA support?

Y

Please specify the Cuda SDK version you want to use, e.g. 7.0. [Leave empty to use system default]:

8.0

Please specify the location where CUDA 8.0 toolkit is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

/usr/local/cuda-8.0

Please specify the cuDNN version you want to use. [Leave empty to use system default]:

6

Please specify the location where cuDNN 6 library is installed. Refer to README.md for more details. [Default is /usr/local/cuda]:

/usr/local/cuda-8.0

Build TensorFlow

After configuration of the setup, the pip package needs to be built by the Bazel. To build a TensorFlow package with GPU support, execute the following command:

$ bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package --action_env="LD_LIBRARY_PATH=${LD_LIBRARY_PATH}"

The above will take a really long time to finish and is also resource intensive (will consume a lot of RAM). It took 30 minutes on my computer (and my computer has decent specs). In the meanwhile read this short story I wrote, watch a movie or anime. If you’re feeling tired you might as well take a quick nap. This process is seriously going to take a long time!!!!

Build & Install Pip Package

You are nearly there!

Hopefully, you were successfully able to build TensorFlow. If some problems occurred please feel free to comment and I will try my best to help you. You can also, search for a solution on Google (or whatever search engine you use); you’re bound to find something helpful!

Great! Let’s move on.

The bazel build command builds a script named build_pip_package. Running the following script will build a .whl file within the ~/tensorflow_package directory:

$ bazel-bin/tensorflow/tools/pip_package/build_pip_package ~/tensorflow_package

Now time for the installation!

WARNING:

Since sudo mkdir ~/virtualenvs is used for creating of the virtual environment, using the pip3 installreturns permission error. In this case the permission level of the environment directory must be change using the sudo chmod -R 777 ~/virtualenvs command.

Run the following command:

$ pip3 install ~/tensorflow_package/file_name.whl

where file_name.whl is the name of the .whl file generated. In my case it was:  tensorflow-1.6.0rc1-cp35-cp35m-linux_x86_64.whl

The file_name.whl is specific to system configuration and may vary in different scenarios.

Test Your Installation

Close all your terminals and open a new terminal to test.  Also make sure your Terminal is not in the ‘tensorflow’ directory.

Activate your virtual environment.

Start Python:

$ python

Run the following in the python shell:

import tensorflow as tf


hello = tf.constant(‘Hello, TensorFlow!’)


sess = tf.Session()


print(sess.run(hello))

Make sure the above runs correckctly without any errors.

Your output should look something like this:

Output
Output:  b’Hello, TensorFlow!’

YOU’RE DONE!

Phew! This was long! I hope installing TensorFlow wasn’t too tough!

Now you have one of the best tools in your bag to embark on the fun and intriguing adventure of Deep Learning and Artificial Intelligence!

Good Luck!

Mohit Jain

Edit: If on running sess = tf.Session(), you get the following error: failed call to cuInit: CUDA_ERROR_UNKNOWN, then run the following command and it’ll be fixed:

$ sudo apt-get install nvidia-modprobe

References:

http://www.linuxandubuntu.com/home/how-to-install-latest-nvidia-drivers-in-linux

https://www.tensorflow.org/install/install_sources

https://alliseesolutions.wordpress.com/2016/09/08/install-gpu-tensorflow-from-sources-w-ubuntu-16-04-and-cuda-8-0/

http://www.machinelearninguru.com/deep_learning/tensorflow/installation/install_from_the_source.html

https://stackoverflow.com/questions/47080760/tensorflow-fails-to-compile/47295278#47295278

https://stackoverflow.com/questions/41293077/how-to-compile-tensorflow-with-sse4-2-and-avx-instructions

https://docs.bazel.build/versions/master/install.html

https://github.com/tensorflow/tensorflow/issues/394

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.