mac OSX 上 brew install hive

本文介紹brew install hive並修改默認的metastore存儲方案，改Derby數據庫爲mysql的方法以及可能遇到的問題的解決方案。

1. 經過homebrew安裝hive

1	brew install hive

2. 添加hadoop和hive的環境變量

1 2 3 4 5 6

sudo vim ~/.bash_profile   export HADOOP_HOME= /usr/local/Cellar/hadoop/hadoop .version.no export HIVE_HOME= /usr/local/Cellar/hive/hive .version.no /libexec   source ~/.bash_profile

3. 下載mysql connector

1 2 3

curl -L 'http://www.mysql.com/get/Downloads/Connector-J/mysql-connector-java-5.1.22.tar.gz/from/http://mysql.he.net/' | tar xz   sudo cp mysql-connector-java-5.1.15 /mysql-connector-java-5 .1.22-bin.jar /usr/local/Cellar/hive/hive .version.no /libexec/lib/

4. 建立mysql metastore

1 2 3 4

mysql> CREATE DATABASE metastore; mysql> USE metastore; mysql> CREATE USER 'hiveuser' @ 'localhost' IDENTIFIED BY 'password' ; mysql> GRANT SELECT , INSERT , UPDATE , DELETE , ALTER , CREATE ON metastore.* TO 'hiveuser' @ 'localhost' ;

5. 配置hive的配置文件

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

cd /usr/local/Cellar/hive/hive .version.no /libexec/conf cp hive-default.xml.template hive-site.xml   #添加或者編輯以下內容 <property>    <name>javax.jdo.option.ConnectionURL< /name >    <value>jdbc:mysql: //localhost/metastore < /value > < /property > <property>    <name>javax.jdo.option.ConnectionDriverName< /name >    <value>com.mysql.jdbc.Driver< /value > < /property > <property>    <name>javax.jdo.option.ConnectionUserName< /name >    <value>hiveuser< /value > < /property > <property>    <name>javax.jdo.option.ConnectionPassword< /name >    <value>password< /value > < /property > <property>    <name>datanucleus.fixedDatastore< /name >    <value> false < /value > < /property >

6. 測試hive是否工做

1 2 3

$ hive; hive > show tables; hive> create table temp_table temp_col string;

7. Revoke few permissions on the mysql metastore

1 2	$ mysql mysql> REVOKE ALTER,CREATE ON metastore.* FROM 'hiveuser' @ 'localhost' ;

9. Further troubleshooting : 
(a) If you get a bin log error saying statement format is not support. Login to your mysql console as root

$ mysql -uroot
mysql > SET GLOBAL binlog_format = 'ROW';

(b) You could also try reading the logs as follows. Logs can be emitted to the bash prompt while running hive by setting hive.root.logger to INFO,console.

$ hive -hiveconf hive.root.logger=INFO,console

(c)You could also read the raw hive logs which is usually located at /tmp/user_name/hive.log

 

(d)If you still have any errors, feel free to comment.

 

 

在配置完成後，可能遇到的問題解決方案：

一，Reference error

解決方案：

        使用sequel pro， 修改hive用戶的權限，添加Reference的global權限。

 

二，啓動hive時遇到的" Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D "

 

解決方案：

 

修改hive-site.xml 配置中的如下key value便可：

 

<name>hive.exec.scratchdir</name>

<value>/tmp/hive-${user.name}</value>

 

 <name>hive.exec.local.scratchdir</name>

 <value>/tmp/${user.name}</value>

 

<name>hive.downloaded.resources.dir</name>

<value>/tmp/${user.name}_resources</value>

 

<name>hive.scratch.dir.permission</name>

    <value>733</value>

 

restart hive metastore and hiveserver2