hive2.1.0 安裝

 

參數配置地址

https://cwiki.apache.org/confluence/display/Hive/AdminManual+Configuration

 

一、配置HIVE_HOME 和PATH

HIVE_HOME=/opt/apache-hive-2.1.0-bin

PATH=$JAVA_BIN:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin:$PATH
export JAVA_HOME JAVA_BIN HADOOP_HOME HIVE_HOME  PATH CLASSPATH

二、拷貝hive-default.xml.template 爲hive-site.xml

<property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
    <description>數據庫鏈接密碼</description>
  </property>
<property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.8.145:3306/hive?createDatabaseIfNotExist=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
    <description>驅動</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>用戶名</description>
  </property>

  <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/hive/local</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hive/resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

 

三、 拷貝hive-env.sh.template 爲hive-env.sh

修改

# Set HADOOP_HOME to point to a specific hadoop install directory
 HADOOP_HOME=/opt/hadoop-2.7.2

# Hive Configuration Directory can be controlled by:
 export HIVE_CONF_DIR=/opt/apache-hive-2.1.0-bin/conf

四、下載 mysql-connector-java-5.1.39.jar 到hive 的lib包中

 

 

5 初始化數據庫

[root@master opt]# schematool -initSchema -dbType mysql
which: no hbase in (/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/opt/apache-hive-2.1.0-bin/bin:/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:      	 jdbc:mysql://192.168.8.145:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :  	 com.mysql.jdbc.Driver
Metastore connection User:     	 root
Tue Aug 23 18:09:10 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.mysql.sql
Tue Aug 23 18:09:11 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Initialization script completed
Tue Aug 23 18:09:15 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
schemaTool completed

 

六、出現如下錯誤

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
       	at org.apache.hadoop.fs.Path.initialize(Path.java:205)
       	at org.apache.hadoop.fs.Path.<init>(Path.java:171)
       	at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:631)
       	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:550)
       	at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)
       	at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
       	at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
       	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
       	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
       	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
       	at java.lang.reflect.Method.invoke(Method.java:606)
       	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
       	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
       	at java.net.URI.checkPath(URI.java:1804)
       	at java.net.URI.<init>(URI.java:752)
       	at org.apache.hadoop.fs.Path.initialize(Path.java:202)
       	... 12 more

 

須要修改

<property>
    <name>hive.exec.local.scratchdir</name>
    <value>${system:java.io.tmpdir}/${system:user.name}</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
 <property>
    <name>hive.downloaded.resources.dir</name>
    <value>${system:java.io.tmpdir}/${hive.session.id}_resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

改爲

<property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/hive/local</value>
    <description>Local scratch space for Hive jobs</description>
  </property>
  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hive/resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

輸入hive

 

[root@master conf]# hive
which: no hbase in (/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/opt/apache-hive-2.1.0-bin/bin:/opt/jdk1.7.0_79/bin:/opt/hadoop-2.7.2/bin:/opt/hadoop-2.7.2/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/hadoop-2.7.2/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/opt/apache-hive-2.1.0-bin/lib/hive-common-2.1.0.jar!/hive-log4j2.properties Async: true
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:52 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Tue Aug 23 18:37:55 EDT 2016 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. tez, spark) or using Hive 1.X releases.
hive> show databases;
OK
default
Time taken: 1.16 seconds, Fetched: 1 row(s)
hive>

 在dnfs中建立目錄數據倉庫目錄

#hadoop fs -mkdir -r user/hive/warehouse

<property>

<name>hive.querylog.location</name>

<value>/usr/hive/log</value>

</property>

這個是用於存放hive相關日誌的目錄

其他的不用修改。

-------------------------------------------finish hive-site.xml

cp hive-log4j.properties.template  hive-log4j.proprties

vi hive-log4j.properties

hive.log.dir=

這個是當hive運行時，相應的日誌文檔存儲到什麼地方

（mine：hive.log.dir=/usr/hive/log/${user.name}）

hive.log.file=hive.log