hadoop 筆記（hive)

 

//**********************************
//安裝配置
1. 修改配置文件

1.1 在conf文件夾下

touch hive-site.xml

<configuration>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://localhost:3306/hivedb?createDatabaseIfNotExist=true</value>
<description>JDBC connect string for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>root</value>
<description>password to use against metastore database</description>
</property>
</configuration>

2. 拷貝mysql驅動包 至 hive/lib下

3. 修改環境變量 ~/.bashrc

export HIVE_HOME=/data/apache-hive-2.3.2-bin
export HADOOP_HOME=/data/hadoop/hadoop-3.0.0
export PATH=$PATH:$HIVE_HOME/bin

注意HADOOP_HOME也是必需要設置的

4. 初始化數據庫

schematool -dbType mysql -initSchema

備註：當前採用mysql做爲hive的元數據庫

1. 出現問題：rg.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.

解決方法進入mysql:GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'root';
flush privileges;

//**********************************
//基本操做

1. 啓動客戶端
hive --service cli

2. 建立並使用數據庫
create database testdb;
use testdb;

備註：出現錯誤：Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
緣由未出生化數據庫

3. 建立一張表

create table users(username string, password string, age int) row format delimited fields terminated by ",";

4. 插入數據

4.1 建立一個數據note.txt

'fred','fred', 30
'sinny', 'sinny', 29

4.2 執行load命令
load data local inpath "/data/data/note.txt" into table users;

遇到問題：
1. There are 0 datanode(s) running and no node(s) are excluded in this operation.
hadoop datanode未啓動

解決方案：刪除/tmp下hadoop相關文件；刪除/usr/local/hadoop/tmp文件 重啓

2. 提示hadoop處於安全模式

./hdfs dfsadmin -safemode leave #無用

bin/hadoop fsck / #無用

在hdfs-site.xml中設置安全閥值屬性，屬性值默認爲0.999f，若是設爲1則不進行安全檢查 # 能夠

5. 查詢命令

select * from users;