深度學習框架-caffe安裝-環境[Mac OSX 10.12]

深度學習框架-caffe安裝
[Mac OSX 10.12]

【參考資源】
1.英文原文：（使用GPU）
[http://hoondy.com/2015/04/03/how-to-install-caffe-on-mac-os-x-10-10-for-dummies-like-me/]
2.基於1的兩篇中文博客：
[http://ylzhao.blogspot.kr/2015/04/mac-os-x-1010caffe.html]
[http://www.jianshu.com/p/8795b882ea67]
3.無GPU，僅使用CPU的狀況下的配置
[http://blog.csdn.net/u014696921/article/details/52156552]
[http://www.phperz.com/article/16/1006/298567.html]

---

個人電腦配置

系統：MacBook Pro OS X Sierra 版本10.12.2
CPU：2.7 GHz Intel Core i5
顯卡：Intel Iris Graphics 6100 1536 MB

若是顯卡是NVIDIA的，可使用GPU，須要安裝cuda，cuda driver和cuDNN GPU庫，而且在Makefile配置成使用GPU。參考資源中【1】【2】是有NVIDIA顯卡的因此安裝了cuda，cuda driver和cuDNN GPU庫，最後的caffe的Makefile.config文件中配置成使用GPU。

因爲我電腦配置的不是NVIDIA顯卡，因此不能使用cuda加速了，因此只能安裝個CPU模式。能夠忽略安裝cuda，cuda driver和cuDNN的安裝步驟，最後的caffe的Makefile.config文件中配置成僅使用CPU。

詳細安裝步驟

Homebrew

1. 根據 http://brew.sh/ 上面的說明安裝Homebrew包管理

Anaconda Python

1. 從https://store.continuum.io/cshop/anaconda/下載和安裝Anaconda Python包（其中包括Caffe框架用到的hdf5）
2. export PATH=~/anaconda/bin:$PATH

BLAS - Intel MKL

1. 因爲Mac OS X操做系統自帶的BLAS庫存在一些不穩定的問題，所以我選擇安裝Intel MKL庫。若是你是在校大學生，可使用學校郵箱從https://software.intel.com/en-us/qualify-for-free-software/student頁面申請Intel Parallel Studio XE 2017安裝包（後面不要忘記在Makefile.config中設置BLAS:=MKL）
2. 確保在安裝Intel Parallel XE時選擇每個組件（由於缺省狀況下不會安裝MKL組件）
3. cd /opt/intel/mkl/lib/
4. sudo ln -s . /opt/intel/mkl/lib/intel64（由於在編譯Caffe時Caffe會從MKL的intel64目錄中去搜索mkl的庫，可是在安裝MKL後，MKL的lib目錄下並無intel64這個目錄，因此須要創建一個intel64目錄到lib目錄的軟連接）

經過Homebrew安裝依賴項

brew edit opencv 在自動打開的vim編輯器中將下面兩行
args << "-DPYTHON#{py_ver}_LIBRARY=#{py_lib}/libpython2.7.#{dylib}"
args << "-DPYTHON#{py_ver}_INCLUDE_DIR=#{py_prefix}/include/python2.7"
替換爲
args << "-DPYTHON_LIBRARY=#{py_prefix}/lib/libpython2.7.dylib"
args << "-DPYTHON_INCLUDE_DIR=#{py_prefix}/include/python2.7"

vim中具體操做是：
i 從當前光標處進入插入模式，開始修改內容，esc 退出插入模式，:wq 保存修改並退出。

brew install --fresh -vd snappy leveldb gflags glog szip lmdb homebrew/science/opencv
brew install --build-from-source --with-python --fresh -vd protobuf
brew install --build-from-source --fresh -vd boost boost-python

從Github上面克隆Caffe的代碼

git clone https://github.com/BVLC/caffe.git
cd caffe
cp Makefile.config.example Makefile.config

配置Makefile.config

1. 設置BLAS := mkl（BLAS (使用intel mkl仍是OpenBLAS)）
2. 取消USE_CUDNN := 1註釋
3. 檢查並設置Python路徑
- 首先修改文件權限：chmod g+w Makefile.config
- 打開文件進行修改：sudo vim Makefile.config     ；按「i」鍵開始修改，修改 ：將# CPU_ONLY = 1前面的#去掉（ 因爲我沒有NVIDIA的顯卡，就沒有安裝CUDA，所以須要打開這個選項） 並按「tab」鍵，（默認從tab處執行），設置BLAS := mkl，檢查並設置python路徑，修改結束後按esc鍵，鍵入「：wq」保存並退出；

如下是個人Makefile.config中的全部配置：（能夠先在命令行中驗證一下本身的文件路徑，必定要根據本身路徑進行設置！）

Refer to http://caffe.berkeleyvision.org/installation.html

# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
# USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
    CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#   You should not set this flag if you will be reading LMDBs with any
#   possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
        -gencode arch=compute_20,code=sm_21 \
        -gencode arch=compute_30,code=sm_30 \
        -gencode arch=compute_35,code=sm_35 \
        -gencode arch=compute_50,code=sm_50 \
        -gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := mkl
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda
PYTHON_INCLUDE :=  $(ANACONDA_HOME)/include/python2.7 \
           $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
           $(ANACONDA_HOME)/include \

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
# PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

設置環境變量

1. export DYLD_FALLBACK_LIBRARY_PATH=/usr/local/cuda/lib:$HOME/anaconda/lib:/usr/local/lib:/usr/lib:/opt/intel/composer_xe_2015.2.132/compiler/lib:/opt/intel/composer_xe_2015.2.132/mkl/lib

• 必須手動查看本身的文件路徑！根據本身的路徑添加環境變量，個人路徑以下：

export DYLD_FALLBACK_LIBRARY_PATH=$HOME/caffe/.build_release/lib:/usr/local/cuda/lib:$HOME/anaconda/lib:/usr/local/lib:/usr/lib:/opt/intel/compilers_and_libraries_2017.1.126/mac/compiler/lib:/opt/intel/compilers_and_libraries_2017.1.126/mac/mkl/lib/

編譯Caffe

make clean
make all
make test
make runtest
make pycaffe
make distribute

• make all的時候注意庫的連接路徑，make runtest注意，會有這樣的一個問題DYLD_FALLBACK_LIBRARY_PATH is cleared by the new System Integrity Protection ，因此要把System Integrity Protection禁止掉：具體操做：電腦從新開機同時按住command+r，進入恢復模式，而後打開終端，輸入csrutil disable，就關閉SIP了，從新啓動電腦便可。