【Tensorflow2.0】Tensorflow2.x installation tutorial

Article directory

  • Anaconda can make the installation of tensorflow easier
    • 1. First install anaconda
    • 2. Create an environment
    • 3. Activate the created environment
    • 4. Test whether the installation is successful

Anaconda can make the installation of tensorflow easier

The tensorflow developer conference just ended yesterday. The latest updates and first releases of product ecosystems such as TensorFlow 2.0, TensorFlow Lite, TensorFlow.js, Swift for TensorFlow, and TFX were released at the conference. They will still be supported in 2019 tensorflow1.x, but we believe that the upgrade of the version will improve the ease of use and performance, especially the confusion of tensorflow1.x development API, including slim, tf.layers, tf.contrib later version has tf.keras, There is also the way of reading data, which is very troublesome, so this article provides an installation method based on the linux system (this tutorial can also be used after installing anaconda on the Windows system).

Interlude: I am honored that this article was included in the Tensorflow official community on November 5, 2020. Welcome to the community

1. First install anaconda

This can be downloaded directly from the official website. You can install anaconda or miniconda. I choose miniconda, so that I will not install a large number of python packages that are not needed, but install it later by myself according to the needs.
Download miniconda, basically press Enter all the way to complete the installation, and then configure the download source to use the domestic mirror to accelerate the download:
( Times are changing, society is developing, anaconda no longer has domestic Accelerate mirroring, so do not execute the following command to add Tsinghua source!!, it can also be used with the default official source, changed on July 2, 2019) Tsinghua source can be used again

#Tsinghua conda image is used for priority naming
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/
conda config --add channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/
conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/fastai/
conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/pytorch/
conda config --append channels https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/

# Display channel address when searching
condaconfig --set show_channel_urls yes

2. Create an environment

Used to install tensorflow2.0 and related python packages.

#Add cudatoolkit and cudnn so that no matter whether your machine is originally installed properly, you can use tensorflow2.0gpu version
conda create -n tf2 python=3.6

3. Activate the created environment

Install tensorflow2.0

#Activate environment
source activate tf2
#For the installation of the GPU version, in order to ensure that it can be installed no matter what the original linux environment is, if the original system is already in compliance, the following commands do not need to be executed
conda install cudatoolkit=10 cudnn=7.6 #currently tf2.0 does not support cuda10.1 (corresponding to cudnn7.6), the official website requires cuda=10, cudnn>7.4
#For the CPU version, the above command does not need to be executed
# Next, there are many situations to install tensorflow
conda install tensorflow #Install tensorflow cpu stable version
conda install tensorflow-gpu#Install the stable version of tensorflow gpu
#Before using this, you can use conda serach tensorflo or conda search tensorflow-gpu to query those versions
#The specified version can be installed initially conda install tensorflow=1.5 tensorflow 1.5 version
#If the installation is wrong, you can use conda uninstall tensorflow to uninstall and then reinstall
#The next step is to install the new tensorflow version, and the conda environment uses pip to take effect
pip install tf-nightly #cpu version
pip install tf-nightly-gpu#GPU version
#I checked the specific version online
pip install tf-nightly-2.0-preview#Install tf 2.0 preview CPU version
pip install tf-nightly-gpu-2.0-preview#Install tf 2.0 preview GPU version

About conda installing cuda and cudnn, when nvidia releases new cuda and cudnn versions, conda can’t install newer versions yet, only old ones. The solution at this time is:

conda search cudatoolkit -c conda-forge
conda search cudnn -c conda-forge
#Select conda-forge to search, you will get newer versions of cuda and cudnn


The same is true for cudnn.
Updated on October 24, 2022
Recently tensorflow2 has been updated to version 2.10, here is an update on the installation method.
For pip installation method, see: https://github.com/tensorflow/tensorflow/issues/57679
Conda installation method:

conda create -n tf python=3.9
conda install cudatoolkit=11.3 cudnn=8.4 -c conda-forge
conda install -c nvidia cuda-nvcc
#conda install -c nvidia cuda-nvcc_linux-64
conda install tensorflow-gpu=2.10 -c conda-forge

The cudatoolkit installed from conda-forge lacks ptxas, we must install cuda-nvcc from the nvidia channel
The installation is completed above, the test codes are:

import tensorflow as tf
print('tf.version:',tf.__version__)
print('list devices')
print(tf.config.list_physical_devices())
print("test lstm:")
a = tf.random.normal([1,6,3])
l = tf.keras.layers.LSTM(9)
o = l(a)
print("test cnn")
b = tf.random.normal([1,9,9,3])
c = tf.keras.layers.Conv2D(10,3)
d=c(b)

Execution result:

So a feasible installation is:
cudatoolkit= 11.3.1, cudnn=8.4.1.50, cuda-nvcc=11.8.89, and python=3.9
The same operation cudatoolkit=11.2 is unsuccessful; for cuda-nvcc, you canSolve the error:
Couldn’t get ptxas version string: INTERNAL: Couldn’t invoke ptxas –version

This update is complete.

The following is updated on July 2, 2019. Recently, tensorflow2.0 has changed from alpha to beat version, so update the latest installation method (the construction of the basic environment remains unchanged)

pip install --upgrade pip
pip install --upgrade tensorflow==2.0.0-beta1 #just the CPU version
pip install --upgrade tensorflow-gpu==2.0.0-beta1#GPU version
#The test about the success of the test installation is the same as the original
# will output 2.0.0-beta1

It was updated on August 24, 2019, and tensorflow was updated again this morning:

pip install tensorflow==2.0.0rc0 # cpu version
pip install tensorflow-gpu==2.0.0rc0 #gpu version
#Recently there is an update rc1 version update, the date is not recorded
pip install tensorflow==2.0.0rc1 # cpu version
pip install tensorflow-gpu==2.0.0rc1 #gpu version
The #tensorflow rc2 version was updated on September 25, 2019, and it feels like the official version is coming soon
#The installation method is the same as above, see https://github.com/tensorflow/tensorflow/releases/tag/v2.0.0-rc2
# Uninstall the old version
pip uninstall tensorflow-gpu
pip install tensorflow-gpu==2.0.0rc2

The meaning of rc’s release candidate is basically the last version available for production, but it is still a test. I feel that many features have been updated. For specific update content, please refer to the official documentation on github
The installation of tensorflow2.0 is completed above. Method:

pip install --upgrade pip
pip install tensorflow-gpu==2.0.0

Updated on October 25, 2019, starting today, conda supports the installation of tensorflow2.0, which is a little comfortable.

conda search tensorflow #search cpu version
conda search tensorflow-gpu#search GPU version
#Everyone can install on demand
conda install tensorflow-gpu=2.0.0 #All kinds of necessary packages are automatically installed,

It is best to use conda to install. You can judge the compatibility between the versions of all installation packages. If they can be installed, they must be compatible.
The requirements of cuda and cudnn on the official website are as follows:

The driver on linux must be greater than 410.x, the cuda version can only be 10, and the cudnn version must be greater than 7.4.

Updated on January 10, 2020, tensorflow2.1 is officially released, there are a few updates:

  • Pip installs the GPU version by default, that is to say, pip install tensorflow is the same as pip install tensorflow-gpu
  • tensorflow2.1 supports cuda 10.1 and cudnn 7.6, that is to say, it can be used in the same Install tensorflow(2.1), pytorch(1.3.1), mxnet(1.5.1) in one environment at the same time, because these three deep learning libraries all support cuda 10.1.

    Specific installation method:
conda create -n dl python=3.6
conda activate dl
conda install cudatoolkit=10.1 cudnn=7.6
pip install tensorflow-gpu #install tensorflow
pip install torch===1.3.1 torchvision===0.4.2 -f https://download.pytorch.org/whl/torch_stable.html
pip install mxnet-cu101#Three deep learning libraries are initially installed successfully
pip install jupyterlab pandas matplotlib pydot scikit-learn seaborn scikit-image opencv-contrib-python

I found a problem. Tensorflow2.1 installed with pip needs a libnvinfer.so file. This is tensorrt. If tensorrt is not installed, an error will be reported:

Follow the official tutorial to install it, I am Installed with the tar package, version tensorrt6. Among them
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH: This command can be written in the current user .bashrc, Then source it (if you use the deb package to install, you can save the work of changing environment variables).
Enter the dl environment again, all the content installed by pip does not need to be added to sudo.
Here, there is no problem in using these three deep learning libraries.

import tensorflow as tf
import torch
import mxnet as mx
from mxnet.runtime import feature_list

print(mx.test_utils.list_gpus())
print(feature_list())
# will output: [✔ CUDA, ✔ CUDNN, ✔NCCL, ✔ CUDA_RTC, ✖ TENSORRT, ✔ CPU_SSE, ✔ CPU_SSE2, ✔ CPU_SSE3, ✔ CPU_SSE4_1, ✔ CPU_SSE4_2, ✖ CPU_SSE4A, ✔ CPU_AVX, ✖ CPU_AVX2, ✔ OPENMP, ✖ SSE, ✔ F16C , ✖ JEMALLOC, ✔ BLAS_OPEN, ✖ BLAS_ATLAS, ✖ BLAS_MKL, ✖ BLAS_APPLE, ✔ LAPACK, ✔ MKLDNN, ✔ OPENCV, ✖ CAFFE, ✖ PROFILER, ✔ DIST_KVSTORE, ✖ CXX14, ✖ INT64_TENSOR_SIZE, ✔ SIGNAL_HANDLER, ✖ DEBUG, ✖ TVM_OP]
print(torch.cuda.is_available())
print(torch.backends.cudnn.version())
print(tf.test.is_gpu_available())


As you can see, all three deep learning libraries can use the GPU, and one of the marked red is the package we installed in advance.

# Verify in tensorflow2.1 whether there is gpu input, the system prompts deprecated
tf.test.is_gpu_available()#No need
tf.test.is_built_with_cuda()#Whether to support NVIdia GPU
tf.test.is_built_with_rocm()#Whether to support AMD graphics card
tf.test.is_built_with_gpu()#NVIDIA or AMD isIs there any support


In addition, installing tensorrt requires pycuda, so several additional packages must be installed, so Only in the conda environment can you use pip to install pycuda.

conda install -c nvidia nvcc_linux-64=10.1
conda install -c conda-forge cudatoolkit-dev=10.1

Execute these two commands to install pycuda

pip install pycuda

If cudatoolkit is downloaded from Nvidia official website and installed as described here, the above two commands are unnecessary.
Supplement again
I have encountered some other problems recently. There is no error when installing cuda10, but tensorflow cannot be used. This is because the system graphics card driver is too low and does not match the newer cuda10. The only All that needs to be done is to update the graphics card driver. Each version of cuda has the minimum requirements for the graphics card driver, as shown in the following table (refer to the official website):

As you can see, the minimum version requirement of cuda10 is 41 0.48. The way to view the graphics card driver of the original system is to enter in the shell: nvidia-smi (this command is available only if the original system has cuda toolkit installed)

The current version on my computer is 384.98, and the highest cuda version that can be installed is 9.0. And cuda9 supports up to tensorflow1.12.
The corresponding relationship between cuda version and tensorflow version is as follows (refer to the official website description):

graphics driver installation or update and cuda tool See the blog for the system completion method of kitt and cudnn.
Updated on May 7, 2020
Today, the official version of tensorflow2.2 was released. Please refer to github for the updated content. In addition, the Keras official website has been revised, with a new interface, closer to tf2, You can study happily.
tensorflow2.2 installation method:

conda create -n dl python=3.7
conda activate dlconda install cudatoolkit=10.1 cudnn=7.6.5
The installation of #tensorflow is also convenient and there are no warnings
pip install tensorflow-gpu -i https://pypi.douban.com/simple
#Up to now (May 7, 2020) the latest pytorch gpu version can also be installed
pip install torch==1.5.0+cu101 torchvision==0.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html -i https://pypi.douban.com/simple

Updated on May 25, 2021
With the gradual release of tf2. Supporting cuda version update, cudnn update, python version requirements (it is recommended not to choose the latest one, unless it is installed from source code), according to the content of this article, it is not difficult to find the relevant version requirements and installation methods, so there are no major changes , this article will remain stable for the time being. It will be updated when there are major changes in the future.

4. Test whether the installation is successful

There are two ways

  • Method 1:
import tensorflow as tf
print(tf.__version__)
# output '2.0.0-alpha0'
print(tf.test.is_gpu_available())
# will output True, it proves that the installation is successful

Updated June 26, 2022

#The new version of tf puts tf.test.is_gpu_available() is replaced by the following command
tf.config.list_physical_devices('GPU')

  • Method 2:
import tensorflow as tf
with tf.device('/GPU:0'):
    a = tf.constant(3)

  • Method Three:
#Enter python to enter the python environment
import tensorflow as tf
#View tensorflow version
print(tf.__version__)
# output '2.0.0-alpha0'
#Test whether the GPU can be called, first check the usage of the graphics card
import os
os.system("nvidia-smi")
# call the graphics card
@tf.function
def f():
pass
f()
#At this time, a lot of logs will be printed, and there is still a warning on my computer, which does not feel affected
#Query the graphics card again
os.system("nvidia-smi")
It is possible to compare the usage of two

It is recommended to try all three testing methods. Sometimes the first method is successful, but an error will be reported when the model is actually built for training. It is best to try all of them.
OK, the process of testing and using is also completed, friends can play it
By the way, here are the resources used for tensorflow learning

  • There will be a series of courses on Wu Enda’s course Cousera, Will gradually complete TensorFlow: From Basics to Mastery in 2019
  • Udacity’s class is based on Tensorflow’s deep learning Intro to TensorFlow for Deep Learning, which can also be accessed on the Internet
  • Coursera Introduction to tensorflow2 from Imperial College London, this course will be released around January 2020
  • Tensorflow2 in-depth course from Imperial College London on Cousera, this course will be released around March 2020
  • Simple and rude TensorFlow 2, this is tensorflow The officially designated Chinese tutorial is still good.

Citing Articles
1 https://docs.anaconda.com/anaconda/user-guide/tasks/tensorflow/
2 https://medium.com/tensorflow/test-drive-tensorflow-2-0-alpha-b6dd1e522b01 Need to go online
3 https://docs .nvidia.com/cuda/cuda-toolkit-release-notes/index.html
4 https://www.tensorflow.org/install/source#install_python_and_the_tensorflow_package_dependencies

Leave a Reply

Your email address will not be published. Required fields are marked *