Hiveserver2配置及優化

先簡單介紹下HS1和HS2的主要區別:異步

           HiveServer1:async

                      能夠看到HS的進程和MetaStore的進程是在一個虛擬機裏面的,並且從圖中能夠看出,一個HS服務同時只能提供一個訪問鏈接。tcp

 

         HiveServer2:this

                             HS2的主要改進是把MetaStoreServer從Hiveserver中剝離出來了,造成一個單獨的進程運行,並且hiveserver和metastore server能夠同時服務         於多個客戶端(Beeline CLI,Hive CLI,HCatalog等)。spa

 

 

配置Hiveserver2的訪問協議,http或者tcp線程

<property>code

  <name>hive.server2.transport.mode</name>server

  <value>binary</value>隊列

  <description>Server transport mode. "binary" or "http".</description>進程

</property>

 

對應http協議的訪問端口

<property>

  <name>hive.server2.thrift.http.port</name>

  <value>10001</value>

  <description>Port number when in HTTP mode.</description>

</property>

 

對應tcp協議的訪問端口

<property>

  <name>hive.server2.thrift.port</name>

  <value>10001</value>

  <description>Port number when in TCP mode.</description>

</property>

 

對應在http協議下線程池的線程數

<property>

  <name>hive.server2.thrift.http.min.worker.threads</name>

  <value>5</value>

  <description>Minimum number of worker threads when in HTTP mode.</description>

</property>

<property>

  <name>hive.server2.thrift.http.max.worker.threads</name>

  <value>500</value>

  <description>Maximum number of worker threads when in HTTP mode.</description>

</property>

 

對應在tcp協議下的線程池的線程數:

<property>

  <name>hive.server2.thrift.min.worker.threads</name>

  <value>5</value>

  <description>Minimum number of worker threads when in TCP mode.</description>

</property>

<property>

  <name>hive.server2.thrift.max.worker.threads</name>

  <value>500</value>

  <description>Maximum number of worker threads when in TCP mode.</description>

</property>

 

配置metastore server的地址

<property>

  <name>hive.metastore.uris</name>

  <value></value>

  <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>

</property>

 

配置異步線程池的線程數

<property>

    <name>hive.server2.async.exec.threads</name>

    <value>100</value>

    <description>Number of threads in the async thread pool for HiveServer2</description>

</property>

 

配置異步線程結束的超時時間(超過這個時間HS會退出):

<property>

  <name>hive.server2.async.exec.shutdown.timeout</name>

  <value>10</value>

  <description>Time (in seconds) for which HiveServer2 shutdown will wait for async

  threads to terminate</description>

</property>

 

配置異步線程的等待超時時間(超過這個值線程會被回收)

<property>

  <name>hive.server2.async.exec.keepalive.time</name>

  <value>10</value>

  <description>Time (in seconds) that an idle HiveServer2 async thread (from the thread pool) will wait

  for new task to arrive before terminating</description>

</property>

 

<property>

  <name>hive.server2.long.polling.timeout</name>

  <value>5000L</value>

  <description>Time in milliseconds that HiveServer2 will wait, before responding to asynchronous calls that use long polling</description>

</property>

 

配置請求緩衝隊列的長度

<property>

  <name>hive.server2.async.exec.wait.queue.size</name>

  <value>100</value>

  <description>Size of the wait queue for async thread pool in HiveServer2.

  After hitting this limit, the async thread pool will reject new requests.</description>

</property>

相關文章
相關標籤/搜索