hadoop2.7.3編譯，支持snappy、bzip2本地壓縮

軟件包：

apache-ant-1.9.9-bin.tar.gz

apache-maven-3.3.9-bin.tar.gz

apache-tomcat-6.0.44.tar.gz

CentOS-6.9-x86_64-minimal.iso

findbugs-3.0.1.tar.gz

hadoop-2.7.3-src.tar.gz

jdk-8u121-linux-x64.tar.gz

protobuf-2.5.0.tar.gz

scala-2.11.11.tgz

snappy-1.1.3.tar.gz

 

 

環境變量：

 

export JAVA_HOME=/root/jdk1.8.0_121
export SCALA_HOME=/root/scala-2.11.11
export ANT_HOME=/root/apache-ant-1.9.9
export FINDBUGS_HOME=/root/findbugs-3.0.1
export M2_HOME=/root/apache-maven-3.3.9
export PATH=.:$PATH:$JAVA_HOME/bin:$SCALA_HOME/bin:$M2_HOME/bin:$ANT_HOME/bin:$FINDBUGS_HOME/bin
export CLASSPATH=.:$JAVA_HOME/jre/lib/rt.jar:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

 

安裝依賴包：

yum -y install  lzo-devel  zlib-devel  gcc gcc-c++ autoconf automake libtool openssl-devel fuse-devel cmake

yum -y install build-essential zlib1g-dev pkg-config libssl-dev

yum -y install libprotobuf-dev protobuf-compiler

yum -y install snappy libsnappy-dev

yum -y install bzip2 libbz2-dev

yum -y install libjansson-dev

yum -y  install fuse libfuse-dev

 

bzip2壓縮支持

yum install -y  bzip2-devel

 

四.配置\安裝protobuf  

進入protobuf所在目錄，而後按下面順序操做

./configure

make

make check

sudo make install

[root@node1 snappy-1.1.3]# protoc --version

libprotoc 2.5.0

 

安裝snappy

# tar xf snappy-1.1.1.tar.gz
# cd snappy-1.1.1
# ./configure
# make && make install

查看snappy是否安裝完成

# ll /usr/local/lib/ | grep snappy

下載太卡的問題：

http://archive.apache.org/dist/tomcat/tomcat-6/v6.0.44/bin/apache-tomcat-6.0.44.tar.gz

放到如下兩個目錄中

hadoop-2.7.3-src/hadoop-common-project/hadoop-kms/downloads/

hadoop-2.7.3-src/hadoop-hdfs-project/hadoop-hdfs-httpfs/downloads

 

修改apache-maven-3.3.9/conf/settings.xml：

<mirrors>
    <!-- mirror
     | Specifies a repository mirror site to use instead of a given repository. The repository that
     | this mirror serves has an ID that matches the mirrorOf element of this mirror. IDs are used
     | for inheritance and direct lookup purposes, and must be unique across the set of mirrors.
     |-->
    <!-- 阿里雲倉庫 -->
        <mirror>
            <id>alimaven</id>
            <mirrorOf>central</mirrorOf>
            <name>aliyun maven</name>
            <url>http://maven.aliyun.com/nexus/content/repositories/central/</url>
        </mirror>
</mirrors>

mvn package -DskipTests -Pdist,native -Dtar -Drequire.snappy -e -X

最終會生成：

/root/hadoop-2.7.3-src/hadoop-dist/target/hadoop-2.7.3.tar.gz

 

解壓並查看本地支持：