經過hiveserver遠程服務構建hive web查詢分析工具

時間 2019-12-05

標籤經過 hiveserver 遠程服務構建 hive web 查詢分析工具欄目 Hadoop 简体版

原文原文鏈接

（1）hive 三種啓動方式及用途，本文主要關注經過hiveserver（可jdbc鏈接）的方式啓動 php

1， hive 命令行模式，直接輸入/hive/bin/hive的執行程序，或者輸入 hive --service cli html

用於linux平臺命令行查詢，查詢語句基本跟mysql查詢語句相似 java

2， hive web界面的啓動方式，hive --service hwi mysql

用於經過瀏覽器來訪問hive，提供基本的基於web的hive查詢服務，能夠看做是hive數據平臺的demo， linux

具體用法可見：http://www.cnblogs.com/gpcuster/archive/2010/02/25/1673480.html 使用HIVE的WEB界面：HWI 程序員

3， hive 遠程服務 (端口號10000) 啓動方式，nohup ./hive --service hiveserver >/dev/null 2>/dev/null & web

用java等程序實現經過jdbc等驅動的方式訪問hive就用這種起動方式了，這個是程序員最須要的方式了。 sql

開源工具phphiveadmin就採用的這種方式，這種方式其實啓動了一個 Hive Thrift Server ，容許你使用任意語言 apache

與hive server通訊，因此若是你不會java，語言將不會成爲問題。瀏覽器

（2）給出一個基於hiveserver的demo，這個demo能夠擴展成一個基於web操做hive的離線分析工具，相似phphiveadmin。

一、demo部署路徑以下：

注：開發環境：myeclipse 8.5, tomcat 6.0

二、code：

2.1 HiveTestCase.java

         importjava.sql.Connection; 
       
         importjava.sql.DriverManager; 
       
         importjava.sql.ResultSet; 
       
         importjava.sql.SQLException; 
       
         importjava.sql.Statement; 
       
         publicclassHiveTestCase { 
       
             publicstaticvoidmain(String[] args)throwsException { 
       
                 String querySQL ="SELECT a.name, a.id, a.sex FROM com58 a where a.id > 100 and a.id < 110 and sex='male'"; 
       
                 hive2Txt(querySQL); 
       
             } 
       
             privatestaticvoidhive2Txt(String querySQL) 
       
                     throwsClassNotFoundException, SQLException { 
       
                 Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver"); 
       
                 // String dropSQL = "drop table com58"; 
       
                 // 1 male 29 3d649ecc 3d649ecc@qq.com 20110304 20110402 
       
                 // 1.id 2.sex 3.age 4.name 5.mail 6.sDate 7.eDate 
       
                 // String createSQL = 
       
                 // "create table com58 (id int, sex string, age int, name string, mail string, sDate bigint, eDate bigint) row format delimited fields terminated by ' '"; 
       
                 // hive插入數據支持兩種方式一種：load文件，令一種是 CTAS（create table as select... 
       
                 // 從另外一個表中查詢進行插入） 
       
                 // hive是不支持insert into...values(....)這種操做的 
       
                 // String insterSQL = 
       
                 // "LOAD DATA LOCAL INPATH '/home/june/hadoop/hadoop-0.20.203.0/tmp/1.txt' OVERWRITE INTO TABLE com58"; 
       
                 Connection con = DriverManager.getConnection( 
       
                         "jdbc:hive://localhost:10000/default","",""); 
       
                 Statement stmt = con.createStatement(); 
       
                 // stmt.executeQuery(dropSQL); // 執行刪除語句 
       
                 // stmt.executeQuery(createSQL); // 執行建表語句 
       
                 // stmt.executeQuery(insterSQL); // 執行插入語句 
       
                 ResultSet res = stmt.executeQuery(querySQL);// 執行查詢語句 
       
                 while(res.next()) { 
       
                     System.out.println("name:\t"+ res.getString(1) +"\tid:\t" 
       
                             + res.getString(2) +"\tsex:\t"+ res.getString(3)); 
       
                 } 
       
             } 
       
         }

2.2 hiveSelect.jsp

         <%@ page language="java"contentType="text/html; charset=UTF-8" 
       
             pageEncoding="UTF-8"%> 
       
         <!DOCTYPE html PUBLIC"-//W3C//DTD HTML 4.01 Transitional//EN""http://www.w3.org/TR/html4/loose.dtd"> 
       
         <%@pageimport="java.util.*"%> 
       
         <% 
        @page 
       
             import="java.io.File, 
       
             java.io.FileWriter, 
       
             java.io.IOException, 
       
             java.sql.Connection, 
       
             java.sql.DriverManager, 
       
             java.sql.ResultSet, 
       
             java.sql.SQLException, 
       
             java.sql.Statement"%> 
       
         <% 
       
             Class.forName("org.apache.hadoop.hive.jdbc.HiveDriver"); 
       
             String table = request.getParameter("table"); 
       
             String querySQL ="select * from "+ table +" limit 10"; 
       
             Connection con = DriverManager.getConnection( 
       
                     "jdbc:hive://localhost:10000/default","",""); 
       
             Statement stmt = con.createStatement(); 
       
             //      stmt.executeQuery(dropSQL); // 執行刪除語句 
       
             //      stmt.executeQuery(createSQL); // 執行建表語句 
       
             //      stmt.executeQuery(insterSQL); // 執行插入語句 
       
             ResultSet res = stmt.executeQuery(querySQL);// 執行查詢語句 
       
             FileWriter fw =newFileWriter("/home/june/a.txt"); 
       
             while(res.next()) { 
       
                 System.out.println("name:\t"+ res.getString(1) +"\tid:\t" 
       
                         + res.getString(2) +"\tsex:\t"+ res.getString(3)); 
       
                 fw.write("name:\t"+ res.getString(1) +"\tid:\t" 
       
                         + res.getString(2) +"\tsex:\t"+ res.getString(3) 
       
                         +"\n"); 
       
             } 
       
             fw.flush(); 
       
             fw.close(); 
       
         %> 
       
         <html> 
       
             <head> 
       
                 <meta http-equiv="Content-Type"content="text/html; charset=UTF-8"> 
       
                 <title>Insert title here</title> 
       
             </head> 
       
             <body> 
       
             </body> 
       
         </html>

2.3 select.html

         <!DOCTYPEhtml PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"> 
       
         <html> 
       
             <head> 
       
                 <metahttp-equiv="Content-Type"content="text/html; charset=UTF-8"> 
       
                 <title>Insert title here</title> 
       
             </head> 
       
             <html> 
       
                 <head> 
       
                     <title>Hive Web Interface-Create a Hive Session</title> 
       
                 </head> 
       
                 <body> 
       
                     <table> 
       
                         <tr> 
       
                             <tdvalign="top"> 
       
                                 <h2> 
       
                                     select hive table to file. 
       
                                 </h2> 
       
                                 <formaction="hiveSelect.jsp"> 
       
                                     <tableborder="1"> 
       
                                         <tr> 
       
                                             <td> 
       
                                                 Session Name 
       
                                             </td> 
       
                                             <td> 
       
                                                 <inputtype="text"name="table"value="table"> 
       
                                             </td> 
       
                                         </tr> 
       
                                         <tr> 
       
                                             <tdcolSpan="2"> 
       
                                                 <inputtype="submit"> 
       
                                             </td> 
       
                                         </tr> 
       
                                     </table> 
       
                                 </form> 
       
                             </td> 
       
                         </tr> 
       
                     </table> 
       
                 </body> 
       
             </html> 
       
         </html>