HBase學習之路 （二）HBase集羣安裝

前提

一、HBase 依賴於 HDFS 作底層的數據存儲

二、HBase 依賴於 MapReduce 作數據計算

三、HBase 依賴於 ZooKeeper 作服務協調

四、HBase源碼是java編寫的，安裝須要依賴JDK

版本選擇

打開官方的版本說明http://hbase.apache.org/1.2/book.html

JDK的選擇

Hadoop的選擇

 

此處咱們的hadoop版本用的的是2.7.5，HBase選擇的版本是1.2.6

安裝

一、zookeeper的安裝

參考http://www.cnblogs.com/qingyunzong/p/8619184.html

二、Hadoopd的安裝

參考http://www.cnblogs.com/qingyunzong/p/8634335.html

三、下載安裝包

找到官網下載 hbase 安裝包 hbase-1.2.6-bin.tar.gz，這裏給你們提供一個下載地址： http://mirrors.hust.edu.cn/apache/hbase/

四、上傳服務器並解壓縮到指定目錄

[hadoop@hadoop1 ~]$ ls
apps  data  hbase-1.2.6-bin.tar.gz  hello.txt  log  zookeeper.out
[hadoop@hadoop1 ~]$ tar -zxvf hbase-1.2.6-bin.tar.gz -C apps/

五、修改配置文件

配置文件目錄在安裝包的conf文件夾中

（1）修改hbase-env.sh 

[hadoop@hadoop1 conf]$ vi hbase-env.sh

export JAVA_HOME=/usr/local/jdk1.8.0_73
export HBASE_MANAGES_ZK=false

（2）修改hbase-site.xml

[hadoop@hadoop1 conf]$ vi hbase-site.xml

<configuration>

        <property>
                <!-- 指定 hbase 在 HDFS 上存儲的路徑 -->
                <name>hbase.rootdir</name>
                <value>hdfs://myha01/hbase126</value>
        </property>
        <property>
                <!-- 指定 hbase 是分佈式的 -->
                <name>hbase.cluster.distributed</name>
                <value>true</value>
        </property>
        <property>
                <!-- 指定 zk 的地址，多個用「,」分割 -->
                <name>hbase.zookeeper.quorum</name>
                <value>hadoop1:2181,hadoop2:2181,hadoop3:2181,hadoop4:2181</value>
        </property>

</configuration>

（3）修改regionservers 

[hadoop@hadoop1 conf]$ vi regionservers 

hadoop1
hadoop2
hadoop3
hadoop4

（4）修改backup-masters

該文件是不存在的，先自行建立

[hadoop@hadoop1 conf]$ vi backup-masters

hadoop4

（5）修改hdfs-site.xml 和 core-site.xml 

最重要一步，要把 hadoop 的 hdfs-site.xml 和 core-site.xml 放到 hbase-1.2.6/conf 下

[hadoop@hadoop1 conf]$ cd ~/apps/hadoop-2.7.5/etc/hadoop/
[hadoop@hadoop1 hadoop]$ cp core-site.xml hdfs-site.xml ~/apps/hbase-1.2.6/conf/

六、將HBase安裝包分發到其餘節點

分發以前先刪除HBase目錄下的docs文件夾，

[hadoop@hadoop1 hbase-1.2.6]$ rm -rf docs/

在進行分發

[hadoop@hadoop1 apps]$ scp -r hbase-1.2.6/ hadoop2:$PWD
[hadoop@hadoop1 apps]$ scp -r hbase-1.2.6/ hadoop3:$PWD
[hadoop@hadoop1 apps]$ scp -r hbase-1.2.6/ hadoop4:$PWD

七、 同步時間

HBase 集羣對於時間的同步要求的比 HDFS 嚴格，因此，集羣啓動以前千萬記住要進行 時間同步，要求相差不要超過 30s

八、配置環境變量

全部服務器都有進行配置

[hadoop@hadoop1 apps]$ vi ~/.bashrc 

#HBase
export HBASE_HOME=/home/hadoop/apps/hbase-1.2.6
export PATH=$PATH:$HBASE_HOME/bin

使環境變量當即生效

[hadoop@hadoop1 apps]$ source ~/.bashrc 

啓動HBase集羣

嚴格按照啓動順序進行

一、啓動zookeeper集羣

每一個zookeeper節點都要執行如下命令

[hadoop@hadoop1 apps]$ zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /home/hadoop/apps/zookeeper-3.4.10/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop1 apps]$ 

二、啓動HDFS集羣及YARN集羣

若是須要運行MapReduce程序則啓動yarn集羣，不然不須要啓動

[hadoop@hadoop1 apps]$ start-dfs.sh
Starting namenodes on [hadoop1 hadoop2]
hadoop2: starting namenode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-namenode-hadoop2.out
hadoop1: starting namenode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-namenode-hadoop1.out
hadoop3: starting datanode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop3.out
hadoop4: starting datanode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop4.out
hadoop2: starting datanode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop2.out
hadoop1: starting datanode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-datanode-hadoop1.out
Starting journal nodes [hadoop1 hadoop2 hadoop3]
hadoop3: starting journalnode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-journalnode-hadoop3.out
hadoop2: starting journalnode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-journalnode-hadoop2.out
hadoop1: starting journalnode, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-journalnode-hadoop1.out
Starting ZK Failover Controllers on NN hosts [hadoop1 hadoop2]
hadoop2: starting zkfc, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-zkfc-hadoop2.out
hadoop1: starting zkfc, logging to /home/hadoop/apps/hadoop-2.7.5/logs/hadoop-hadoop-zkfc-hadoop1.out
[hadoop@hadoop1 apps]$ 

啓動完成以後檢查如下namenode的狀態

[hadoop@hadoop1 apps]$ hdfs haadmin -getServiceState nn1
standby
[hadoop@hadoop1 apps]$ hdfs haadmin -getServiceState nn2
active
[hadoop@hadoop1 apps]$ 

三、啓動HBase

保證 ZooKeeper 集羣和 HDFS 集羣啓動正常的狀況下啓動 HBase 集羣 啓動命令：start-hbase.sh，在哪臺節點上執行此命令，哪一個節點就是主節點

[hadoop@hadoop1 conf]$ start-hbase.sh
starting master, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-master-hadoop1.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
hadoop3: starting regionserver, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-regionserver-hadoop3.out
hadoop4: starting regionserver, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-regionserver-hadoop4.out
hadoop2: starting regionserver, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-regionserver-hadoop2.out
hadoop3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
hadoop3: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
hadoop4: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
hadoop4: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
hadoop2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
hadoop2: Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
hadoop1: starting regionserver, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-regionserver-hadoop1.out
hadoop4: starting master, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-master-hadoop4.out
[hadoop@hadoop1 conf]$ 

觀看啓動日誌能夠看到：

（1）首先在命令執行節點啓動 master

（2）而後分別在 hadoop02,hadoop03,hadoop04,hadoop05 啓動 regionserver

（3）而後在 backup-masters 文件中配置的備節點上再啓動一個 master 主進程

驗證啓動是否正常

一、檢查各進程是否啓動正常

 主節點和備用節點都啓動 hmaster 進程

 各從節點都啓動 hregionserver 進程

按照對應的配置信息各個節點應該要啓動的進程如上圖所示

二、經過訪問瀏覽器頁面

hadoop1

hadop4

從圖中能夠看出hadoop4是備用節點

三、驗證高可用

幹掉hadoop1上的hbase進程，觀察備用節點是否啓用

[hadoop@hadoop1 conf]$ jps
4960 HMaster
2960 QuorumPeerMain
3169 NameNode
3699 DFSZKFailoverController
3285 DataNode
5098 HRegionServer
5471 Jps
3487 JournalNode
[hadoop@hadoop1 conf]$ kill -9 4960

 hadoop1界面訪問不了

hadoop4變成主節點

四、若是有節點相應的進程沒有啓動，那麼能夠手動啓動

啓動HMaster進程

[hadoop@hadoop3 conf]$ jps
3360 Jps
2833 JournalNode
2633 QuorumPeerMain
3179 HRegionServer
2732 DataNode
[hadoop@hadoop3 conf]$ hbase-daemon.sh start master
starting master, logging to /home/hadoop/apps/hbase-1.2.6/logs/hbase-hadoop-master-hadoop3.out
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; support was removed in 8.0
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; support was removed in 8.0
[hadoop@hadoop3 conf]$ jps
2833 JournalNode
3510 Jps
3432 HMaster
2633 QuorumPeerMain
3179 HRegionServer
2732 DataNode
[hadoop@hadoop3 conf]$ 

 

啓動HRegionServer進程

[hadoop@hadoop3 conf]$ hbase-daemon.sh start regionserver