Hive lateral view 和 explode 詳解

1. 建表語句結構 

create table if not exists employees ( 
name         string, 
salary       float, 
subordinates array<string>, 
deductions   map<string, float>, 
address      struct<street:string, city:string, state:string, zip:int> 
) 
row format delimited 
fields terminated by '\001'  
collection items terminated by '\002'  
map keys terminated by '\003' 
lines terminated by '\n' 
stored as textfile;

2. 表裏 name 和 subordinates 的數據結構

3. 使用 lateral view 和 explode 查詢

select name,subordinate from employees lateral view explode(subordinates) subordinates_table as subordinate;

總結: explode就是將hive一行中複雜的 array 或者 map 結構拆分紅多行。

下面就作個小例子, 建立 hive 表 doc, 表裏只有一列 text 類型爲 string, 將 hadoop 目錄下的 README.txt 導入該表, 並寫出 sql 求出 wordcount

create table if not exists doc(text string) row format delimited lines terminated by '\n';

load data local inpath '/opt/hadoop-2.7.4/README.txt' overwrite into table doc;

select word, count(*) from doc lateral view explode(split(text,' ')) ITable as word group by word;