spark_spark鏈接hive config

1 環境

hadoop2.7.3

apache-hive-2.1.1-bin

spark-2.1.0-bin-hadoop2.6

jdk1.8

 

2 配置文件

在hive-site.xml中配置mysql數據庫鏈接。

cp apache-hive-2.1.1-bin/conf/hive-site.xml  ./spark-2.1.0-bin-hadoop2.6/conf/

cp apache-hive-2.1.1-bin/lib/mysql-connector-java-5.1.40-bin.jar ./spark-2.1.0-bin-hadoop2.6/jars

 

3 啓動

啓動hadoop : ./hadoop-2.7.3/sbin/start-all.sh

啓動mysql :  service mysql start

啓動hive :  ./apache-hive-2.1.1-bin/bin/hive

啓動spark : ./spark-2.1.0-bin-hadoop2.6/bin/spark-sql 驗證是否正常鏈接hive,查詢語法同hive一致。 (i.e. show tables;)

　　　　　　或者 ./spark-2.1.0-bin-hadoop2.6/bin/spark-shell 運行scala程序