0028-如何在CDH未啓用認證的狀況下安裝及使用Sentry

1.文檔編寫目的node


CDH平臺中的安全,認證(Kerberos/LDAP)是第一步,受權(Sentry)是第二步。若是要啓用受權,必須先啓用認證。但在CDH平臺中給出了一種測試模式,即不啓用認證而只啓用Sentry受權。但強烈不建議在生產系統中這樣使用,由於若是沒有用戶認證,受權沒有任何意義形同虛設,用戶能夠隨意使用任何超級用戶登陸HiveServer2或者Impala,並不會作密碼校驗。注:本文檔僅適用於測試環境。mysql

本文檔主要描述如何在CDH未啓用認證的狀況下安裝,配置及使用Sentry。git

  • 內容概述

1.如何安裝Sentry服務sql

2.Hive/Impala/Hue/HDFS服務如何與Sentry集成shell

3.Sentry測試數據庫

  • 測試環境

1.操做系統爲CentOS6.5安全

2.CM和CDH版本爲5.11.1oop

3.採用root用戶操做測試

  • 前置條件

1.CDH集羣運行正常ui

2.集羣未啓用認證服務(如Kerberos或LDAP)

2.Sentry安裝


1.在MySQL中建立sentry數據庫

建表語句:

create database sentry default character set utf8;

CREATE USER 'sentry'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON sentry. * TO 'sentry'@'%';

FLUSH PRIVILEGES;

命令行操做:

[root@ip-172-31-6-148 527-hive-HIVEMETASTORE]# mysql -uroot -p
Enter password: 
...
mysql> create database sentry default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> CREATE USER 'sentry'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL PRIVILEGES ON sentry.* TO 'sentry'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql>

2.進入Cloudera Manager控制檯點擊「添加服務」

3.進入服務添加界面

4.選擇Sentry服務,點擊「繼續」

5.選擇Sentry Server及Gateway的安裝節點,點擊「繼續「,注意需提早在MySQL中創建好相關用戶和數據庫

6.輸入Sentry服務的數據庫信息,點擊測試,測試經過,點擊「繼續」

7.等待服務安裝成功,點擊「繼續」

8.點擊「完成」,Sentry服務到此安裝完成。

3.Sentry配置

3.1Hive配置


1.配置Hive使用Sentry服務

2.關閉Hive的用戶模擬功能

3.集羣未啓用安全認證環境下,須要配置如下參數

<property>
    <name>sentry.hive.testing.mode</name>
    <value>true</value>
</property>

3.2Impala配置


配置Impala與Sentry集成

3.3Hue配置


配置Hue與Sentry集成

3.4HDFS配置


配置HDFS開啓ACLs與Sentry權限同步

完成以上配置後,回到Cloudera Manager主頁,部署客戶端配置並重啓相關服務。

4.Sentry測試

4.1建立hive超級用戶


1.使用beeline鏈接HiveServer2,並登陸hive用戶

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.0 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: hive
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.0)
Driver: Hive JDBC (version 1.1.0-cdh5.12.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000>

注意:標紅部分爲輸入的hive用戶,輸入的hive用戶並未真正的校驗

2.建立一個admin角色

0: jdbc:hive2://localhost:10000> create role admin;
...
INFO  : OK
No rows affected (2.52 seconds)
0: jdbc:hive2://localhost:10000>

3.爲admin角色賦予超級權限

0: jdbc:hive2://localhost:10000> grant all on server server1 to role admin;

...

INFO : OK

No rows affected (0.221 seconds)

0: jdbc:hive2://localhost:10000>

4.將admin角色受權給hive用戶組

0: jdbc:hive2://localhost:10000> grant role admin to group hive;

...

INFO : OK

No rows affected (0.162 seconds)

0: jdbc:hive2://localhost:10000>

4.2建立test表


使用beeline登陸hive用戶,建立一個test表,並插入測試數據

0: jdbc:hive2://localhost:10000> create table test (s1 string, s2 string) row format delimitedfields terminated by ',';

...

INFO : OK

No rows affected (0.592 seconds)

0: jdbc:hive2://localhost:10000> insert into test values('a','b'),('1','2');

...

INFO : OK

No rows affected (20.123 seconds)

0: jdbc:hive2://localhost:10000>

4.3建立測試角色並受權給用戶組


建立兩個角色:

read:只能讀default庫test表,並受權給fayson用戶組

write:只能寫default庫test表,並受權給user_w用戶組

注意:集羣全部節點必須存在fayson和user_w用戶,用戶默認用戶組與用戶名一致,賦權是針對用戶組而不是針對用戶。

[root@ip-172-31-6-148 cdh-shell-master]# id fayson
uid=501(fayson) gid=501(fayson) groups=501(fayson)
[root@ip-172-31-6-148 cdh-shell-master]# useradd user_w
[root@ip-172-31-6-148 cdh-shell-master]# id user_w
uid=502(user_w) gid=502(user_w) groups=502(user_w)
[root@ip-172-31-6-148 cdh-shell-master]#

1.使用hive用戶建立建立read和write角色,並受權read角色對test表select權限,write角色對test表insert權限

0: jdbc:hive2://localhost:10000> create role read;

...

INFO : OK

No rows affected (0.094 seconds)

0: jdbc:hive2://localhost:10000> grant select on table test to role read;

...

INFO : OK

No rows affected (0.1 seconds)

0: jdbc:hive2://localhost:10000> create role write;

...

INFO : OK

No rows affected (0.105 seconds)

0: jdbc:hive2://localhost:10000> grant insert on table test to role write;

...

INFO : OK

No rows affected (0.112 seconds)

0: jdbc:hive2://localhost:10000>

2.爲fayson用戶組受權read角色,爲user_w用戶組受權write角色

0: jdbc:hive2://localhost:10000> grant role read to group fayson;
...
INFO  : OK
No rows affected (0.187 seconds)
0: jdbc:hive2://localhost:10000> grant role write to group user_w;
...
INFO  : OK
No rows affected (0.101 seconds)
0: jdbc:hive2://localhost:10000>

4.4beeline驗證


1.使用fayson用戶登陸beeline進行驗證

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.0 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
scan complete in 2ms
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: fayson
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.0)
Driver: Hive JDBC (version 1.1.0-cdh5.12.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.351 seconds)
0: jdbc:hive2://localhost:10000> select * from test;
...
INFO  : OK
+----------+----------+--+
| test.s1  | test.s2  |
+----------+----------+--+
| a        | b        |
| 1        | 2        |
+----------+----------+--+
2 rows selected (0.24 seconds)
0: jdbc:hive2://localhost:10000>
0: jdbc:hive2://localhost:10000> insert into test values("2", "222");
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->action=insert; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000>

2.使用user_w用戶登陸beeline驗證

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.0 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
scan complete in 2ms
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: user_w
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.0)
Driver: Hive JDBC (version 1.1.0-cdh5.12.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.365 seconds)
0: jdbc:hive2://localhost:10000> select  * from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User user_w does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->Column=s1->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> insert into test values("2", "333");
...
INFO  : Completed executing command(queryId=hive_20170902183535_56bcd189-544a-453f-9752-e40a9fed60c5); Time taken: 17.762 seconds
INFO  : OK
No rows affected (18.035 seconds)
0: jdbc:hive2://localhost:10000>

驗證總結:

fayson用戶所屬組爲fayson擁有test表讀權限,因此只能對test表進行selecth和count操做不能進行insert操做;

user_w用戶所屬組爲user_w擁有test表寫權限,因此只能對test表進行insert操做不能進行select和count操做;

4.5HDFS驗證


1.切換至fayson用戶下,瀏覽和查看/user/hive/warehouse/test數據目錄及文件

[root@ip-172-31-6-148 ~]# su fayson
[fayson@ip-172-31-6-148 root]$ cd /home/fayson/
[fayson@ip-172-31-6-148 ~]$ ll
total 4
-rw-rw-r-- 1 fayson fayson 19 Sep  5 12:55 test.txt
[fayson@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=fayson, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[fayson@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse/test
Found 1 items
-rwxrwx--x+  3 hive hive          8 2017-09-05 12:52 /user/hive/warehouse/test/000000_0
[fayson@ip-172-31-6-148 ~]$ hadoop fs -cat /user/hive/warehouse/test/000000_0
a,b
1,2
[fayson@ip-172-31-6-148 ~]$ hadoop fs -put test.txt /user/hive/warehouse/test
put: Permission denied: user=fayson, access=WRITE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[fayson@ip-172-31-6-148 ~]$

2.切換user_w用戶下,瀏覽和查看/user/hive/warehouse/test數據目錄及文件

[root@ip-172-31-6-148 ~]# su user_w
[user_w@ip-172-31-6-148 root]$ cd /home/user_w/
[user_w@ip-172-31-6-148 ~]$ cat test.txt 
333,5555
eeee,dddd
[user_w@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=user_w, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[user_w@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse/test
ls: Permission denied: user=user_w, access=READ_EXECUTE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[user_w@ip-172-31-6-148 ~]$ hadoop fs -put test.txt /user/hive/warehouse/test
[user_w@ip-172-31-6-148 ~]$

測試總結:

fayson用戶所屬用戶組爲fayson,該組只擁有對test表的讀權限,所以fayson用戶不能對/user/hive/warehouse目錄下除test之外的其它目錄進行查看(包含父目錄),而且不能向test目錄put文件,只能瀏覽和查看test目錄下的文件。

user_w用戶所屬用戶組爲user_w,該組只擁有對test表的寫權限,所以user_w用戶不能對/user/hive/warehouse目錄下的任何目錄進行查看(包含父目錄),而且只擁有向test目錄put文件的權限。說明Sentry實現了HDFS的ACL同步。

4.6Hue驗證


1.使用admin用戶登陸Hue,分別建立fayson和user_w用戶

2.使用fayson用戶登陸Hue

能夠查看test表

能夠對test表進行count操做

只有SELECT權限不能向test表插入數據

FileBrower驗證

不能夠瀏覽/user/hive/warehouse目錄

能夠瀏覽有SELECT權限的/user/hive/warehouse/test數據目錄

能夠查看/user/hive/warehouse/test目錄下的全部數據文件,但不能修改

3.使用user_w用戶登陸Hue

由於無SELECT權限,因此不能查詢表信息

不能對test表進行count操做

能夠向test表中插入數據

FileBrowser

不能夠瀏覽test表數據目錄的父目錄/user/hive/warehouse

無SELECT權限也不能瀏test表的數據目錄/user/hive/warehouse/test

測試總結:

fayson和user_w用戶均能經過hue界面看到test表,擁有read角色的fayson用戶組能對test表進行select和count操做,而且能經過File Browser瀏覽和查看test表的數據目錄/user/hive/warehouse/test。擁有write角色的user_w用戶組只能對test表進行insert操做,但不能經過File Browser瀏覽和查看test表的數據目錄/user/hive/warehouse/test。說明Sentry在命令行的操做和受權在Hue中依舊有效。

4.7Impala驗證


1.使用fayson用戶測試

登陸集羣任意節點命令行下切換到fayson用戶下

[root@ip-172-31-6-148 ~]# su fayson
[fayson@ip-172-31-6-148 root]$

在命令行執行impala-shell命令

[Not connected] > connect ip-172-31-10-118.fayson.com:21000;
...
[ip-172-31-10-118.fayson.com:21000] > show tables;
Query: show tables
+------+
| name |
+------+
| test |
+------+
Fetched 1 row(s) in 0.05s
[ip-172-31-10-118.fayson.com:21000] > select * from test;
...
+----+----------+
| s1 | s2       |
+----+----------+
| 1  | tttttttt |
+----+----------+
Fetched 1 row(s) in 5.32s
[ip-172-31-10-118.fayson.com:21000] > select count(*) from test;
...
+----------+
| count(*) |
+----------+
| 1        |
+----------+
Fetched 1 row(s) in 0.14s
[ip-172-31-10-118.fayson.com:21000] > insert into test values('2', 'test2');
Query: insert into test values('2', 'test2')
Query submitted at: 2017-09-11 01:37:56 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
ERROR: AuthorizationException: User 'fayson' does not have privileges to execute 'INSERT' on: default.test
[ip-172-31-10-118.fayson.com:21000] >

2.使用user_w用戶測試

登陸集羣任意節點命令行下切換到user_w用戶下

[root@ip-172-31-6-148 ~]# su user_w
[user_w@ip-172-31-6-148 root]$ impala-shell

在命令行執行命令impala-shell,進行以下操做

[user_w@ip-172-31-6-148 root]$ impala-shell
...
[Not connected] > connect ip-172-31-10-118.fayson.com:21000;
...
Query: show tables
+------+
| name |
+------+
| test |
+------+
Fetched 1 row(s) in 0.06s
[ip-172-31-10-118.fayson.com:21000] > select * from test;
Query: select * from test
Query submitted at: 2017-09-11 01:41:17 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
ERROR: AuthorizationException: User 'user_w' does not have privileges to execute 'SELECT' on: default.test

[ip-172-31-10-118.fayson.com:21000] > select count(*) from test;
Query: select count(*) from test
Query submitted at: 2017-09-11 01:41:23 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
ERROR: AuthorizationException: User 'user_w' does not have privileges to execute 'SELECT' on: default.test

[ip-172-31-10-118.fayson.com:21000] > insert into test values('2', 'impala insert');
Query: insert into test values('2', 'impala insert')
Query submitted at: 2017-09-11 01:41:48 (Coordinator: http://ip-172-31-10-118.fayson.com:25000)
Query progress can be monitored at: http://ip-172-31-10-118.fayson.com:25000/query_plan?query_id=bd4a433465037682:77a7c3c400000000
Modified 1 row(s) in 0.71s
[ip-172-31-10-118.fayson.com:21000] >

驗證總結:

Impala與Sentry集成後可使用Sentry來進行權限管理,擁有read角色的fayson用戶組只能對test表進行select和count操做不能插入數據,擁有write角色的user_w

用戶組只能對test表插入數據不能進行select和count操做。說明Sentry實現了Hive權限與Impala的同步。

5.Sentry列權限管理驗證


1.在集羣全部節點新增fayson_r用戶

[root@ip-172-31-6-148 cdh-shell-bak]# useradd fayson_r
[root@ip-172-31-6-148 cdh-shell-bak]# id fayson_r
uid=504(fayson_r) gid=504(fayson_r) groups=504(fayson_r)
[root@ip-172-31-6-148 cdh-shell-bak]#

2.使用beeline登陸hive用戶

使用hive用戶建立columnread角色,併爲角色受權test表s1列的讀權限,將columnread角色受權給fayson_r用戶組。

[root@ip-172-31-6-148 cdh-shell-bak]# beeline 
Beeline version 1.1.0-cdh5.12.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: hive
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.1)
Driver: Hive JDBC (version 1.1.0-cdh5.12.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> create role columnread;
...
INFO  : OK
No rows affected (0.225 seconds)
0: jdbc:hive2://localhost:10000> grant select(s1) on table test to role columnread;
...
INFO  : OK
No rows affected (0.095 seconds)
0: jdbc:hive2://localhost:10000> grant role columnread to group fayson_r;
...
INFO  : OK
No rows affected (0.091 seconds)
0: jdbc:hive2://localhost:10000>

3.使用beeline登陸fayson_r用戶測試

[root@ip-172-31-6-148 cdh-shell-bak]# beeline 
Beeline version 1.1.0-cdh5.12.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000
scan complete in 2ms
Connecting to jdbc:hive2://localhost:10000
Enter username for jdbc:hive2://localhost:10000: fayson_r
Enter password for jdbc:hive2://localhost:10000: 
Connected to: Apache Hive (version 1.1.0-cdh5.12.1)
Driver: Hive JDBC (version 1.1.0-cdh5.12.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> show tables;
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.304 seconds)
0: jdbc:hive2://localhost:10000> select * from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson_r does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->Column=s2->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> select s1 from test;
...
INFO  : OK
+-------+--+
|  s1   |
+-------+--+
| a     |
| 1     |
| 111   |
| 333   |
| eeee  |
+-------+--+
5 rows selected (0.197 seconds)
0: jdbc:hive2://localhost:10000> select s2 from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson_r does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->Column=s2->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> select count(*) from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson_r does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000> select count(s1) from test;
...
INFO  : OK
+------+--+
| _c0  |
+------+--+
| 5    |
+------+--+
1 row selected (23.855 seconds)
0: jdbc:hive2://localhost:10000>

4.瀏覽HDFS目錄

[fayson_r@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=fayson_r, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[fayson_r@ip-172-31-6-148 ~]$ hadoop fs -ls /user/hive/warehouse/test
ls: Permission denied: user=fayson_r, access=READ_EXECUTE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[fayson_r@ip-172-31-6-148 ~]$

5.使用admin用戶登陸Hue,建立fayson_r用戶,而後使用fayson_r用戶登陸

測試總結:

fayson_r用戶所屬用戶組爲fayson_r,該組只擁有對test表s1列的讀權限,所以在select和count的時候只能對s1列進行select和count,fayson_r用戶無權限瀏覽/user/hive/warehouse下的全部目錄;使用hue只能對test表s1列進行select和count操做,無權限瀏覽/user/hive/warehouse目錄及目錄下全部子目錄。

注意:Sentry只支持SELECT的列受權,不能用於INSERT和ALL的列受權。

6.備註

在使用beeline進行受權驗證時,只是輸入了username未作用戶信息校驗。在未啓用認證服務的集羣下,該文檔的Sentry配置方式只適用於測試環境,不能用於生產環境。

在集羣啓用Sentry服務後,因爲Sentry不支持Hive CLI權限管理,因此建議禁用Hive CLI。但在非安全環境下,不能經過hadoop.proxyuser.hive.groups來限制訪問用戶組。

7.Hive受權參考


  1. 角色建立和刪除
create role test;
drop role test;
  1. 角色受權和取消受權

表受權給角色

grant select on table test_table to role role_name;
revoke select on table test_table to role role_name;

列受權給角色

grant select(column1,column2) on table test_table to role role_name;
revoke select(column1,column2) on table test_table to role role_name;
  1. 組受權和取消受權
grant role role_name to group user_group;
revoke role role_name to group user_group;

注意:Sentry適用於用戶組受權,不適用與用戶受權;

8.常見問題


  1. 不能建立角色,異常以下

0: jdbc:hive2://localhost:10000> create role admin;

Error: Error whilecompiling statement: FAILED:InvalidConfigurationException hive.server2.authentication can't be none innon-testing mode (state=42000,code=40000)

0: jdbc:hive2://localhost:10000>

緣由:因爲集羣未啓用Kerberos,須要配置sentry.hive.testing.mode爲true

解決方法:參考3.1的第3步配置。

醉酒鞭名馬,少年多浮誇! 嶺南浣溪沙,嘔吐酒肆下!摯友不願放,數據玩的花!

推薦關注Hadoop實操,第一時間,分享更多Hadoop乾貨,歡迎轉發和分享。

相關文章
相關標籤/搜索