Ubuntu下Python訪問Hive——thrift安裝及測試

一、下載thrift：

wget http://mirror.bjtu.edu.cn/apache//thrift/0.8.0/thrift-0.9.0.tar.gz

二、安裝依賴：

sudo apt-get install build-essential
sudo apt-get install bison flex
sudo apt-get install libboost-dev python-dev
sudo apt-get install autoconf automake libtool pkg-config
sudo apt-get install git

三、安裝thrift：

tar -xzvf thrift-0.9.0.tar.gz
cd thrift-0.9.0
sudo ./configure --prefix=/host/java/thrift
sudo make && sudo make install

四、啓動Hive Thrift Server  (>>hive --service  hiveserver)

要先啓動hadoop 設置hdfs安全模式爲leave

>>start-all.sh

>>hadoop dfsadmin -safemode leave

啓動時報錯……不要緊，運行python時候纔會再有反應

Starting Hive Thrift Server
13/07/29 13:24:13 WARN conf.HiveConf: DEPRECATED: Configuration property hive.metastore.local no longer has any effect. Make sure to provide a valid value for hive.metastore.uris if you are connecting to a remote metastore.

解決辦法：

由於在0.10裏面，hive.metastore.local參數已經取消，都使用hive.metastore.uris參數，若是hive.metastore.uris爲空，就默認爲local模式。

把hive.metastore.local參數刪掉就行了。

五、測試代碼編寫以下：

testHive.py:

#!/usr/bin/env python

import sys

sys.path.append('/usr/local/hadoop/hive/lib/py')#'/usr/local/hadoop/hive是你的hive安裝路徑

from hive_service import ThriftHive

from hive_service.ttypes import HiveServerException

from thrift import Thrift

from thrift.transport import TSocket

from thrift.transport import TTransport

from thrift.protocol import TBinaryProtocol

 

def hiveExe(sql):

 

    try:

        transport = TSocket.TSocket('127.0.0.1', 10000) 

        transport = TTransport.TBufferedTransport(transport)

        protocol = TBinaryProtocol.TBinaryProtocol(transport)

        client = ThriftHive.Client(protocol)

        transport.open()

 

 

        client.execute(sql)

 

        print "The return value is : " 

        print client.fetchAll()

        print "............"

        transport.close()

    except Thrift.TException, tx:

        print '%s' % (tx.message)

 

if __name__ == '__main__':

    hiveExe("select * from testParttion")#t_afan_test是你的表名

六、執行

>>python testHive.py

七、結果

The return value is:

['[12,23,23,34]\t["what","are","this"]','[34,45,34,23,12]\t["who","am","i","are"]']