CentOS安裝caffe-ssd算法

CentOS安裝caffe

1. 下載源碼

su guxiaotu # 切換到用戶guxiaotu
# 切換到當前用戶主目錄下
cd $HOME
sudo yum install -y git # 安裝git
git clone https://github.com/weiliu89/caffe.git caffe-ssd #下載代碼而且重命名爲caff-ssd
# 進入caffe-ssd源代碼目錄
cd caffe-ssd
# checkout出ssd算法源碼
git checkout ssd

2. 設置環境變量

# 設置環境變量
echo 'export CAFFE_ROOT=$HOME/caffe-ssd' >> ~/.bashrc	# 配置$CAFFE_ROOT
# 將/usr/lib/python2.7/dist-packages和$CAFFE_ROOT/python追加到$PYTHONPATH.
echo 'export PYTHONPATH=$PYTHONPATH:/usr/lib/python2.7/dist-packages:$CAFFE_ROOT/python'>>~/.bashrc
# 將$CAFFE_ROOT/build/tool命令工具追加到$PATH中
echo 'export PATH=$PATH:$CAFFE_ROOT/build/tool' >> ~/.bashrc
# 使環境變量生效
source ~/.bashrc

3. 安裝第三方庫

sudo yum install -y epel-release \
wget \
zip \
gcc-c++ \
cmake \
protobuf-devel \
leveldb-devel \
snappy-devel \
boost-devel \
hdf5-devel \
gflags-devel \
glog-devel \
lmdb-devel \
openblas-devel \
python-devel \
liblas-devel \
atlas-devel	\
libopenblas-dev \
python-matplotlib \
numpy

# 清除緩存包
sudo yum clean all
sudo rm -rf /var/cache/yum

centos中opencv-devel默認爲2.4.5，會提示"warning: GStreamer: unable to query position of stream (/builddir/build/BUILD/opencv-2.4.5/modules/highgui/src/cap_gstreamer.cpp:660)"，國外論壇2013年討論過，源代碼有問題。因此選擇手動安裝opencv

sudo wget -O /opt/opencv2.4.13.6.zip https://github.com/opencv/opencv/archive/2.4.13.6.zip
sudo unzip /opt/opencv2.4.13.6.zip -d /opt
cd /opt/opencv-2.4.13.6/ && sudo mkdir release/ && cd release
sudo cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
sudo make && sudo make install

# glog
sudo wget https://storage.googleapis.com/google-code-archive-downloads/v2/code.google.com/google-glog/glog-0.3.3.tar.gz -P /opt
tar zxvf /opt/glog-0.3.3.tar.gz -C /opt
cd /opt/glog-0.3.3
sudo ./configure
sudo make && sudo make install
# gflags
sudo wget https://codeload.github.com/gflags/gflags/zip/v2.0 -O /opt/gflags-2.0.zip
sudo unzip /opt/gflags-2.0.zip -d /opt
cd /opt/gflags-2.0
sudo ./configure
sudo make && sudo make install
# lmdb
git clone https://github.com/LMDB/lmdb /opt/lmdb
cd /opt/lmdb/libraries/liblmdb
sudo make && sudo make instal

提醒：

每次從新make編譯源代碼前，須要進入以前源代碼包make clean清除下編譯

Ø 出現問題:若是gflags高於2.0版本會出現如下問題 /usr/bin/ld: /usr/local/lib/libgflags.a(gflags.cc.o): relocation R_X86_64_32S against `.rodata' can not be used when making a shared object; recompile with -fPIC /usr/local/lib/: could not read symbols: Bad value collect2: ld returned 1 exit status make: [libglog.la] Error 1

Ø 分析緣由： Glog Need to be compiled into shared library.

4. 安裝Python以及第三方庫

sudo yum install -y python-pip
sudo pip install --upgrade pip # 升級pip到10.0.1版本
# 臨時設置阿里雲的pip源加快Python庫的下載速度
sudo pip install -i https://mirrors.aliyun.com/pypi/simple ansible 
# 安裝Python第三方庫
pip install Cython \
numpy \
scipy  \
scikit-image  \
matplotlib==1.5.3 \
ipython \
h5py \
leveldb \
networkx \
nose \
pandas \
python-dateutil \
protobuf \
python-gflags \
pyyaml \
Pillow \
mkl \
pyldap \
six --user

# 也能夠按照$CAFFE_ROOT/python/requirements.txt中指定具體版本安裝
pip install Cython==0.19.2 \
numpy==1.7.1 \
scipy==0.13.2 \
scikit-image==0.9.3 \
matplotlib==1.3.1 \
ipython==3.0.0 \
h5py==2.2.0 \
leveldb==0.191 \
networkx==1.8.1 \
nose==1.3.0 \
pandas==0.12.0 \
python-dateutil==2.6.0 \
protobuf==2.5.0 \
python-gflags==2.0 \
pyyaml==3.10 \
Pillow==2.3.0 \
six==1.1.0 --user

注意 matplotlib==1.5.3，1.5.3是當前1.0版本中最高版本，超過版本2.0.0以後，會提示「ImportError: cannot import name cbook」

5. 編譯caffe環境

# 設置行號
echo 'set number' >> /etc/vimrc
# 進入caffe-ssd目錄
cd $CAFFE_ROOT
cp Makefile.config.example Makefile.config
sudo vim Makefile.config
# 修改如下內容
8 CPU_ONLY := 1	# 第8行，將前面#取消，啓用只使用CPU模式
89 WITH_PYTHON_LAYER := 1 # 第89行，取消註釋表示使用Python編寫layer
# 第91~92行後面追加配置hdf5路徑
91 INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
92 LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib/usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial

# 編譯caffe
make all
# 編譯pycaffe，前提確保$CAFFE_ROOT/python添加到環境變量PYTHONPATH中（詳細請看2. 設置環境變量）
make pycaffe
make test
# 可選
make runtest -j8

Ø 出現問題: ./include/caffe/util/cudnn.hpp:8:34:致命錯誤：caffe/proto/caffe.pb.h：沒有那個文件或目錄

Ø 分析緣由： 應該是版本比較低。 pip install protobuf --upgrade -i http://pypi.douban.com/simple --trusted-host pypi.douban.com --user pip install pillow --upgrade -i http://pypi.douban.com/simple --trusted-host pypi.douban.com --user

6. 準備模型以及數據集

6.1下載做者訓練的模型數據，存放在$CAFFE_ROOT/models/VGGNet/

# 若是使用做者已經訓練好的模型數據，請下載到$CAFFE_ROOT/model
sudo wget -P $CAFFE_ROOT/model http://www.cs.unc.edu/%7Ewliu/projects/SSD/models_VGGNet_VOC0712_SSD_300x300.tar.gz
# 解壓到制定目錄
tar -zxvf $CAFFE_ROOT/model/models_VGGNet_VOC0712_SSD_300x300.tar.gz -C $CAFFE_ROOT/model

6.2 下載VOC2007 and VOC2012數據集， 存放在默認目錄$HOME/data/中

# 用戶主目錄下建立data目錄後進入
mkdir $HOME/data
# 下載數據集
sudo wget -P $HOME/data http://host.robots.ox.ac.uk/pascal/VOC/voc2012/VOCtrainval_11-May-2012.tar
sudo wget -P $HOME/data http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtrainval_06-Nov-2007.tar
sudo wget -P $HOME/data http://host.robots.ox.ac.uk/pascal/VOC/voc2007/VOCtest_06-Nov-2007.tar
# 解壓到指定目錄（必須按照如下順序解壓，不能顛倒）
tar -xvf $HOME/data/VOCtrainval_11-May-2012.tar -C $HOME/data
tar -xvf $HOME/data/VOCtrainval_06-Nov-2007.tar -C $HOME/data
tar -xvf $HOME/data/VOCtest_06-Nov-2007.tar -C $HOME/data

注意：三個壓縮文件解壓順序必定不能打亂

6.3 建立LMDB文件

cd $CAFFE_ROOT # 必須保證在$CAFFE_ROOT中執行
sudo vim /etc/ld.so.conf.d/usr-libs.conf
# 添加如下內容
/usr/local/lib
# 在data/VOC0712/中建立trainval.txt, test.txt, and test_name_size.txt 
./data/VOC0712/create_list.sh
# You can modify the parameters in create_data.sh if needed.
# It will create lmdb files for trainval and test with encoded original image:
#   - $HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_trainval_lmdb
#   - $HOME/data/VOCdevkit/VOC0712/lmdb/VOC0712_test_lmdb
# and make soft links at examples/VOC0712/
sudo vim data/VOC0712/create_data.sh
# 修改一下如下值
root_dir=$CAFFE_ROOT
# 執行sh腳本生成lmdb文件
./data/VOC0712/create_data.sh

注意：若是提示缺乏某個model，說明缺乏對應Python第三方庫或者版本太低，使用sudo pip install --upgrade 具體包名安裝，也能夠制定具體版本安裝 提示缺乏sci沒法使用pip install -U命令安裝scikit-image，提示「Cannot uninstall 'pyparsing'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.」

# 因爲其餘庫以來pyparsing，因此選擇忽略它
pip install scikit-image  --ignore-installed pyparsing --user

Ø 出現問題: error while loading shared libraries: libgflags.so.2: cannot open shared object file: No such file or directory Ø 分析緣由： 緣由是程序沒有找到相應的依賴庫，解決方法：

1. 將全部的用戶須要用到的庫放到/usr/loca/lib；
2. 在/etc/ld.so.conf.d/目錄下新建文件usr-libs.conf，內容是：/usr/local/lib
3. sudo ldconfig

7. 訓練/計算

7.1 訓練模型

# It will create model definition files and save snapshot models in:
#   - $CAFFE_ROOT/models/VGGNet/VOC0712/SSD_300x300/
# and job file, log file, and the python script in:
#   - $CAFFE_ROOT/jobs/VGGNet/VOC0712/SSD_300x300/
# and save temporary evaluation results in:
#   - $HOME/data/VOCdevkit/results/VOC2007/SSD_300x300/
# It should reach 77.* mAP at 120k iterations.
python examples/ssd/ssd_pascal.py

7.2 圖片數據集上測試

# If you would like to test a model you trained, you can do:
python examples/ssd/score_ssd_pascal.py

7.3 視頻數據測試$CAFFE_ROOT/examples/videos

cd $CAFFE_ROOT
# 測試示例視頻
sudo vim $CAFFE_ROOT/examples/ssd/ssd_pascal_video.py 
# 第99~100行修改模式爲CPU，P.Solver.GPU修改成P.Solver.CPU
99 # Use GPU or CPU
100 solver_mode = P.Solver.CPU
# 第77~76行修改視頻文件路徑$CAFFE_ROOT/examples/videos
75 # The video file path
76 video_file = "examples/videos/ILSVRC2015_train_00755001.mp4"

7.4 攝像頭測試

# 攝像頭測試
sudo vim $CAFFE_ROOT/examples/ssd/ssd_pascal_webcam.py 
# 第100~101行修改模式爲CPU，P.Solver.GPU修改成P.Solver.CPU
99 # Use GPU or CPU
102 solver_mode = P.Solver.CPU
# If you would like to attach a webcam to a model you trained, you can do:
python examples/ssd/ssd_pascal_webcam.py