Hive遠程模式安裝(1.00)

hiveserver2和metastore分離部署，元數據存儲採用mysql，mysql與metastore分離部署。

1. mysql: 部署在ctrl節點
2. hiveserver2: 部署在ctrl和data01節點
3. metastore: 部署在data02和data03節點
4. beeline: 部署在其餘任意一臺機器

部署圖：

首先參考《Hive單節點安裝使用(1.00)》文檔，安裝好單節點的hive軟件，而後將整個目錄拷貝到其餘節點，再按以下步驟配置各節點。

安裝配置

• hiveserver2節點配置

hive 1.0中已經再也不提供hiveserver，取而代之的是hiveserver2。
hiveserver2已經再也不須要hive.metastore.local這個配置項(hive.metastore.uris爲空,則表示是metastore在本地,不然就是遠程)，直接配置hive.metastore.uris便可。

1 2 3 4 5 6

<property> <name>hive.metastore.uris</name> <value>thrift://data02:9083,thrift://data03:9083</value> <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description> </property>

hiveserver2不鏈接mysql數據庫，不須要配置鏈接mysql的配置項。

• metastore節點配置

須要配置鏈接mysql的配置項：

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

<property> <name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://ctrl:3306/hive?createDatabaseIfNotExist=true</value> <description>JDBC connect string for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> <description>Driver class name for a JDBC metastore</description> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> <description>username to use against metastore database</description> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>hive</value> <description>password to use against metastore database</description> </property>

其餘配置：

hive.metastore.warehouse.dir：hive數據在HDFS中的目錄
hive.exec.scratchdir：hive在HDFS中的臨時目錄
hive.exec.local.scratchdir：hive的本地臨時目錄，/tmp/hive
hive.downloaded.resources.dir: hive下載的本地臨時目錄, /tmp/hive

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25

<property> <name>hive.metastore.warehouse.dir</name> <value>/user/hive/warehouse</value> <description>location of default database for the warehouse</description> </property> <property> <name>hive.exec.scratchdir</name> <value>/tmp/hive</value> <description>HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/< username> is created, with ${hive.scratch.dir.permission}.</description> </property> <property> <name>hive.exec.local.scratchdir</name> <value>/tmp/hive</value> <description>Local scratch space for Hive jobs</description> </property> <property> <name>hive.downloaded.resources.dir</name> <value>/tmp/hive</value> <description>Temporary local directory for added resources in the remote file system.</description> </property>

• beeline客戶端安裝

beeline已經集成在hive軟件包中，不須要額外安裝。 下載hive軟件包，而後配置hadoop的主目錄便可：

1 2 3 4 5 6

cd /opt wget http://apache.fayea.com/hive/stable/apache-hive-1.0.0-bin.tar.gz tar -zxvf apache-hive-1.0.0-bin.tar.gz mv apache-hive-1.0.0-bin hive cd /opt/hive/conf mv hive-env.sh.template hive-env.sh

在hive-env.sh中增長：

1 2 3

export HADOOP_HOME=/opt/hadoop/client/hadoop-2.4.1 export HIVE_HOME=/opt/hive export HIVE_CONF_DIR=/opt/hadoop/client/hive/conf

啓動

• 啓動hiveserver2

[hadoop@ctrl bin]$ hive --service hiveserver2 &
[hadoop@data01 bin]$ hive --service hiveserver2 &

或者：

[hadoop@ctrl bin]$ hiveserver2 &
[hadoop@data01 bin]$ hiveserver2 &

• 啓動metastore

[hadoop@data02 bin]$ hive --service metastore &
[hadoop@data03 bin]$ hive --service metastore &

• 使用beeline鏈接

[root@cheyo conf]# beeline --color=true --fastConnect=true -u jdbc:hive2://192.168.99.107:10000
[root@cheyo conf]# ../bin/beeline --color=true --fastConnect=true 
Beeline version 1.0.0 by Apache Hive
beeline> !connect jdbc:hive2://ctrl:10000
scan complete in 17ms
Connecting to jdbc:hive2://ctrl:10000
Enter username for jdbc:hive2://ctrl:10000:hadoop
Enter password for jdbc:hive2://ctrl:10000:
Connected to: Apache Hive (version 1.0.0)
Driver: Hive JDBC (version 1.0.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://ctrl:10000> show tables;
+-----------+--+
| tab_name  |
+-----------+--+
| person    |
| t_hive    |
+-----------+--+
2 rows selected (0.404 seconds)
0: jdbc:hive2://ctrl:10000> select * from person;
+--------------+-------------+--+
| person.name  | person.age  |
+--------------+-------------+--+
| cheyo        | 25          |
| yahoo        | 30          |
| people       | 27          |
+--------------+-------------+--+
3 rows selected (0.413 seconds)
0: jdbc:hive2://ctrl:10000>

啓動hwi(Web界面)

待保/opt/hive/lib/目錄下有hive-hwi-1.0.0.war文件。hwi只須要在一臺HiveServer2上啓動便可。

• 啓動hwi服務

[hadoop@ctrl bin]$ hive --service hwi &

• 訪問Web

http://ip:9999/hwi

注意

• 使用beeline鏈接時，填寫的username將用於權限管理。填寫錯誤可能會致使運行HiveQL命令時指示沒有權限。

日誌

Hive的日誌默認在/tmp/{username}/hive.log中。能夠經過配置文件修改。
好比將日誌路徑修改成:/opt/hive/log/

1 2 3 4 5 6 7 8 9 10 11 12 13

cd /opt/hive mkdir -p /opt/hive/log cd /opt/hive/conf mv hive-log4j.properties.template hive-log4j.properties vi hive-log4j.properties #修改以下一行： hive.log.dir=/opt/hive/log mv hive-exec-log4j.properties.template hive-exec-log4j.properties vi hive-exec-log4j.properties #修改以下一行： hive.log.dir=/opt/hive/log