Hive 內部表和外部表的區別

內部表和外部表區別

• 未被 external 修飾的表是內部表(managed table ),被 external 修改的是外部表(external table );
• 內部表的數據由Hive自身管理,外部表由HDFS管理；
• 內部表的數據存儲位置是hive.metastore.warehouse.dir(默認:/user/hive/warehouse),外部表數據的存儲位置由本身指定(若是沒有 LOCATION),Hive將在HDFS上的 /user/hive/warehouse/文件夾下之外部表的表名建立一個文件夾,並將屬於這個表的數據存放在這裏);
• 刪除內部表會直接刪除元數據(metadata)及存儲數據,刪除外部表僅僅會刪除元數據,HDFS上的文件並不會刪除,Hive默認建立的是內部表。
• 對內部表的修改會將修改直接同步給元數據,而對外部表的表結構和分區進行修改,則修改修復。

MSCK REPAIR TABLE table_name

建立內部表

create table test.t1(
   id    int
  ,name  string
  ,hobby array<string>
  ,add   map<string,string>
)
row format delimited
fields terminated by ','
collection items terminated by '_'
map keys terminated by ":";


Time taken: 0.112 seconds
hive> 
    > 
    > 
    > create table test.t1(
    >    id    int
    >   ,name  string
    >   ,hobby array<string>
    >   ,add   map<string,string>
    > )
    > row format delimited
    > fields terminated by ','
    > collection items terminated by '_'
    > map keys terminated by ":";
OK
Time taken: 0.056 seconds
hive>

查看錶的描述

Time taken: 0.056 seconds
hive> desc test.t1;
OK
col_name        data_type       comment
id                      int                                         
name                    string                                      
hobby                   array<string>                               
add                     map<string,string>                          
Time taken: 0.048 seconds, Fetched: 4 row(s)
hive>

裝載數據

注意：通常不多用insert(不是insert overwrite) 語句，由於就算是插入一條語句，也會調用MapReduce，這裏咱們選擇Load Data 的方式。

原始數據
1,xiaoming,book-TV-code,beijing:chaoyang-shagnhai:pudong
2,lilei,book-code,nanjing:jiangning-taiwan:taibei
3,lihua,music-book,heilongjiang:haerbin

加載數據
load data local inpath '/opt/software/gouyang/5.txt' overwrite into table test.t1;
                        
Time taken: 0.048 seconds, Fetched: 4 row(s)
hive> load data local inpath '/opt/software/gouyang/5.txt' overwrite into table test.t1;
Loading data to table test.t1
Table test.t1 stats: [numFiles=1, numRows=0, totalSize=147, rawDataSize=0]
OK
Time taken: 0.253 seconds
hive> select * from test.t1;
OK
t1.id   t1.name t1.hobby        t1.add
1       xiaoming        ["book-TV-code"]        {"beijing":"chaoyang-shagnhai:pudong"}
2       lilei   ["book-code"]   {"nanjing":"jiangning-taiwan:taibei"}
3       lihua   ["music-book"]  {"heilongjiang":"haerbin"}
Time taken: 0.058 seconds, Fetched: 3 row(s)
hive>