HIVE操做常見問題彙總

一、Terminal initialization failed; falling back to unsupported

[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
        at jline.TerminalFactory.create(TerminalFactory.java:101)
        at jline.TerminalFactory.get(TerminalFactory.java:158)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)

        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

解決方案：

引發該問題的主要緣由是由於Hadoop裏面yarn的lib中含有和hive的lib目錄中版本不一致的jline的jar包，只須要把hive中的jline包同步過去就能夠了,先刪除yarn的lib中的jline包，再複製

如：cp /u01/apache-hive-1.2.1-bin/lib/jline-2.12.jar /u01/hadoop-2.6.1/share/hadoop/yarn/lib

二、hive使用derby做爲元數據庫找不到所建立表的緣由

不少初學者安裝hive使用的是默認元數據庫derby，這裏面容易遇到下面問題：

安裝好了hive 環境，建表後關閉鏈接窗口，再從新進表就不見了,這是哪裏配置的不對？
1.命令行鍵入 hive後建立表，
2.在show tables; 顯示錶列表，上面顯示剛纔建立的表
3.把窗口關閉，再從新開個窗口鏈接服務器，鍵入hive ，而後show table，表看不到了

爲何會找不到建立的表，這個並無消失，而是你的位置不對。

A.若是咱們在home下面進入hive，那麼咱們的metastore_db,就會產生在home目錄下。
B.若是咱們在/usr目錄下進入hive，那麼咱們的metastore_db,就會產生在usr目錄下。

因此若是你更換了目錄啓動hive，那麼以前的建立的表你是找不到的。

什麼是不一樣路徑下進入hive,例如

實驗一：目錄下建立表，查看錶，不切換目錄操做

查看當前目錄，這裏是/home/bdata/data目錄

[bdata@bdata4 data]$ pwd

/home/bdata/data

查看內容，目前只有IY02_C.txt文件

[bdata@bdata4 data]$ ls

IY02_C.txt

執行hive命令，進入客戶端

[bdata@bdata4 data]$ hive

Logging initialized using configuration in jar:file:/u01/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.properties
hive> show tables;
OK

Time taken: 0.64 seconds

#建立一個表結構，根據|分割

hive> CREATE TABLE YHJHK_IY02(AAB301 String,AAE043 String,AAB001 String,AAC001 String,AAC003 String,AAC004 String,AAC002 String,AAC005 String,AAC006 String,AAC009 String,AAC012 String,AAC028 String,AAC064 String,AAC087 String,AAC007 String,AAC032 String,AIC001 int, AIC090 int, AAC031 String,AAE160 String,AIC104 int, AIC164 int, AIC020 int, AAC095 int, AAC096 int, AAC097 int, AAC098 int, AAC099 int, AAC103 int, AAC104 int, AIC268 int, AIC115 int, AIC116 int, AIC117 int, AAB000 int, SUBJOB_ID int, VZE105 String, VZE003 String) ROW FORMAT DELIMITED FIELDS TERMINATED BY '|' STORED AS TEXTFILE;

#加載HDFS中的文件，這裏/data/IY02_C.txt是咱們提早存放到hdfs文件系統下的

hive> load data inpath '/data/IY02_C.txt' into table YHJHK_IY02;

#再次查看錶，肯定存在表yhjhk_iy02

hive> show tables;
OK
yhjhk_iy02

Time taken: 0.686 seconds, Fetched: 1 row(s)

#退出

hive> quit;

再次查看該目錄，則多了兩個文件metastore_db, derby.log

[bdata@bdata4 data]$ ls

derby.log  IY02_C.txt  metastore_db

實驗二：切換目錄，進入hive，查看錶信息

回到主目錄

[bdata@bdata4 data]$ cd

查看當前路徑，確認目錄已經切換

[bdata@bdata4 ~]$ pwd

/home/bdata

查看當前目錄，確認是新目錄，沒有metastore_db和derby.log，只有一個data目錄

[bdata@bdata4 ~]$ ls

data

進入hive客戶端

[bdata@bdata4 ~]$ hive

Logging initialized using configuration in jar:file:/u01/apache-hive-1.2.1-bin/lib/hive-common-1.2.1.jar!/hive-log4j.properties

#查看錶信息，這裏查詢出沒有任何表信息

hive> show tables;
OK

Time taken: 0.64 seconds

hive> quit;

查看當前目錄，已經生成了metastore_db和derby.log文件

[bdata@bdata4 ~]$ ls

data metastore_db derby.log

經過以上方式肯定，咱們在用derby的時候，若是忽然執行hive查詢表時，沒有查詢到數據，此時，不用擔憂，表數據並未消失，只須要切換到以前進行hive建表的目錄執行客戶端操做便可,或者提早切換爲MYSQL來存儲原信息則不會出現該問題。

三、執行建表命令報錯：FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections, we don't support retries at the client level.)

這個是因爲個人mysql再也不本地(默認使用本地數據庫)，這裏須要配置遠端元數據服務器，而後在hive服務端啓動元數據存儲服務，metastore用於和mysql之間的表結構建立或更新時通信

首先在配置文件配置metastore的配置，以下

<property> 

     <name>hive.metastore.uris</name> 

     <value>thrift://192.168.10.34:9083</value>

     <description>jdbc/odbc connection hive,if mysql must set </description>  

</property>

配置好後，啓動通信服務：

[bdata@bdata4 bin]$ ./hive --service metastore -hiveconf hbase.zookeeper.quorum=bdata1,bdata2,bdata3 -hiveconf hbase.zookeeper.property.clientPort=2181  &

四、執行HIVE命令報錯：Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
        ... 8 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
        ... 14 more
Caused by: MetaException(message:Could not connect to meta store using any of the URIs provided. Most recent failure: org.apache.thrift.transport.TTransportException: java.net.ConnectException: Connection refused
        at org.apache.thrift.transport.TSocket.open(TSocket.java:187)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:420)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:86)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3005)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3024)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:503)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
        at java.net.Socket.connect(Socket.java:579)
        at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
        ... 22 more
)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.open(HiveMetaStoreClient.java:466)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:236)
        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:74)

        ... 19 more

解決辦法

同問題3解決辦法一致

五、建表時，報錯：FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: An exception was thrown while adding/validating class(es) : Specified key was too long; max key length is 767 bytes

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes

解決辦法：

修改數據庫的字符集

登陸數據庫，進入HIVE數據庫，刪除全部的表

執行查詢命令：

alter database hive character set latin1;

修改爲功後，重啓metastore，從新進入hive客戶端執行建立表的操做便可成功

6.hiveserver2啓動後，beeline不能鏈接的涉及的問題：
緣由：權限問題
解決：
/user/hive/warehouse
/tmp
/history (若是配置了jobserver 那麼/history也須要調整)
這三個目錄，hive在運行時要讀取寫入目錄裏的內容，因此把權限放開，設置權限：

hadoop fs -chmod -R 777 /hive/tmp

hadoop fs -chmod -R 777 /hive/warehouse

7.beeline 連接拒絕報錯信息
緣由：官方的一個bug
解決：
hive.server2.long.polling.timeout

hive.server2.thrift.bind.host 注意把host改爲本身的host

八、java.lang.IllegalArgumentException: java.net.UnknownHostException: dfscluster

解決辦法：

找不到hdfs集羣名字dfscluster,這個文件在HADOOP的etc/hadoop下面，有個文件hdfs-site.xml，複製到hive的conf下，重啓便可

[bdata@bdata4 hadoop]$ cp  hdfs-site.xml /u01/apache-hive-1.2.1-bin/conf/