Hive學習筆記——安裝和內部表CRUD

1.首先須要安裝Hadoop和Hive

安裝的時候參考 http://blog.csdn.net/jdplus/article/details/46493553

安裝的版本是apache-hive-2.1.1-bin.tar.gz,解壓到/usr/local目錄下

而後在/etc/profile文件中添加

export HIVE_HOME=/usr/local/hive
export PATH=$PATH:$HIVE_HOME/bin

2.修改配置文件

在bin/hive-config.sh文件中添加

export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_121
export HIVE_HOME=/usr/local/hive
export HADOOP_HOME=/usr/local/hadoop

添加hive-env.sh文件

cp hive-env.sh.template hive-env.sh

修改conf目錄下的hive-site.xml的內容,該模式是本地模式,且使用JDBC鏈接元數據,本地模式能夠查看Hive編程指南P24-27

實際數據仍是存放在HDFS中,MySQL中存放的是元數據表,即schema信息

<configuration>
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://localhost:3306/hive</value>
    <description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>Username to use against metastore database</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>XXXX</value>
    <description>password to use against metastore database</description>
</property>
#若是不配置下面的部分會產生錯誤1.
<property>
    <name>hive.exec.local.scratchdir</name>
    <value>/usr/local/hive/tmp/local</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
<property>
    <name>hive.downloaded.resources.dir</name>
    <value>/usr/local/hive/tmp/downloaded</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
</property>
<property>
    <name>hive.querylog.location</name>
    <value>/usr/local/hive/tmp/location</value>
    <description>Location of Hive run time structured log file</description>
</property>
<property>
   <name>hive.server2.logging.operation.log.location</name>
    <value>/usr/local/hive/tmp/operation_logs</value>
    <description>Top level directory where operation logs are stored if logging functionality is enabled</description>
</property>
</configuration>

注意/usr/local/hive/tmp/local , /usr/local/hive/tmp/downloaded , /usr/local/hive/tmp/location , /usr/local/hive/tmp/operation_logs 這四個文件夾須要本身建立

修改hive-log4j.properties

#cp hive-log4j.properties.template hive-log4j.properties
#vim hive-log4j.properties
hive.log.dir=自定義目錄/log/

在HDFS上創建/tmp和/user/hive/warehouse目錄，並賦予組用戶寫權限

注意這裏面的/user/hive/warehouse是由hive-site.xml中的${hive.metastore.warehouse.dir}指定的數據倉庫的目錄

hadoop fs -mkdir       /tmp
hadoop fs -mkdir       /user/hive/warehouse
hadoop fs -chmod g+w   /tmp
hadoop fs -chmod g+w   /user/hive/warehouse

 Mysql配置

#建立數據庫
mysql> create database hive;
#賦予訪問權限
mysql> grant all privileges on hive.* to root@localhost identified by '密碼' with grant option;
mysql> flush privileges;
#將JDBC複製到Hive庫目錄用於java程序與mysql的鏈接
cp mysql-connector-java-5.1.35/mysql-connector-java-5.1.35-bin.jar /usr/local/apache-hive-1.1.0-bin/lib/

在hive的bin目錄下,初始化元數據

./schematool -initSchema -dbType mysql

 

若是想使用hive的web界面的話,參考 http://blog.csdn.net/yyywyr/article/details/51416721

 

Hadoop和Hive的own和grp都是Hadoop

Hive的安裝很簡單,在清華鏡像站下載Hive的二進制文件,而後解壓到/usr/local目錄下

修改own和grp就行

2.啓動hive以前須要先啓動Hadoop,啓動後顯示

hive> 

3.查看有沒有表

show tables

4.建立表,多個字段的時候須要指定用什麼來分隔

create table test(id int,name string)row format delimited fields terminated by '\t';

create table test(id int,name string)row format delimited fields terminated by ',';

5.插入數據

insert into table test values (1,'row1'),(2,'row2');

也能夠select任意一張空表來insert

insert into table default.example_table (select 1L,'xiaodou',array(1L,2L,3L) from default.test limit 1);

6.加載數據,也能夠使用Hadoop fs -put命令直接上傳文件,注意文件中的分隔符須要和建立表的時候指定分隔符保持一致

load data local inpath 'XXXX' into table XXXX;

若是是加載分區的數據的話

load data local inpath 'xxx/role_id=1' into table XXX PARTITION(role_id=1);

7.查看數據

select * from XXXX;

8.刪除表

drop table XXXX;