0031-如何在CDH啓用Kerberos的狀況下安裝及使用Sentry(一)

時間 2019-11-10

標籤如何 cdh 啓用 kerberos 狀況安裝使用 sentry 简体版

原文原文鏈接

1.文檔編寫目的node

本文檔主要講述如何在啓用Kerberos的CDH集羣中安裝配置及使用Sentry。mysql

內容概述

1.如何安裝Sentry服務sql

2.Hive/Impala/Hue/HDFS服務如何與Sentry集成shell

3.Sentry測試數據庫

測試環境

1.操做系統爲CentOS6.5oop

2.CM和CDH版本爲5.11.1測試

3.採用root用戶操做ui

前置條件

1.CDH集羣運行正常操作系統

2.集羣已啓用Kerberos且正常使用命令行

2.Sentry安裝

1.在MySQL中建立sentry數據庫

建表語句：

create database sentry default character set utf8;

CREATE USER 'sentry'@'%' IDENTIFIED BY 'password';

GRANT ALL PRIVILEGES ON sentry. * TO 'sentry'@'%';

FLUSH PRIVILEGES;

命令行操做：

[root@ip-172-31-6-148 527-hive-HIVEMETASTORE]# mysql -uroot -p
Enter password: 
...
mysql> create database sentry default character set utf8;
Query OK, 1 row affected (0.00 sec)
mysql> CREATE USER 'sentry'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)
mysql> GRANT ALL PRIVILEGES ON sentry. * TO 'sentry'@'%';
Query OK, 0 rows affected (0.00 sec)
mysql> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)
mysql>

2.進入Cloudera Manager控制檯點擊「添加服務」

3.進入服務添加界面,選擇Sentry服務，點擊「繼續」

4.選擇Sentry Server及Gateway的安裝節點，點擊「繼續」

5.輸入Sentry服務的數據庫信息，點擊測試，測試經過，點擊「繼續」

6.等待服務安裝成功，點擊「繼續」

7.點擊「完成」，Sentry服務至此安裝完成。

3.Sentry配置

3.1Hive配置

1.配置Hive使用Sentry服務

2.關閉Hive的用戶模擬功能

3.2Impala配置

配置Impala使用Sentry

3.3Hue配置

配置Hue使用Sentry

3.4HDFS配置

配置HDFS開啓ACLs與Sentry權限同步

完成以上配置後，回到Cloudera Manager主頁，部署客戶端配置並重啓相關服務。

4.Sentry測試

4.1建立hive超級用戶

使用hive用戶登陸Kerberos，操做以下

[root@ip-172-31-6-148 196-hive-HIVEMETASTORE]# kinit -kt hive.keytab hive/ip-172-31-6-148.fayson.com@FAYSON.COM   
[root@ip-172-31-6-148 196-hive-HIVEMETASTORE]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: hive/ip-172-31-6-148.fayson.com@FAYSON.COM

Valid starting     Expires            Service principal
09/07/17 02:26:04  09/08/17 02:26:04  krbtgt/FAYSON.COM@FAYSON.COM
        renew until 09/12/17 02:26:04
[root@ip-172-31-6-148 196-hive-HIVEMETASTORE]#

1.使用beeline鏈接HiveServer2

[root@ip-172-31-6-148 196-hive-HIVEMETASTORE]# beeline 
Beeline version 1.1.0-cdh5.12.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-6-148@FAYSON.COM
scan complete in 3ms
Connecting to jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-6-148@FAYSON.COM
Connected to: Apache Hive (version 1.1.0-cdh5.12.1)
Driver: Hive JDBC (version 1.1.0-cdh5.12.1)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000/>

2.建立admin角色

0: jdbc:hive2://localhost:10000/> create role admin;
...
INFO  : OK
No rows affected (0.37 seconds)
0: jdbc:hive2://localhost:10000/>

3.爲admin角色賦予管理員權限

0: jdbc:hive2://localhost:10000_> grant_ all on server server1 to role admin;

...

INFO : OK

No rows affected (0.221 seconds)

0: jdbc:hive2://localhost:10000_>_

4.將admin角色受權給hive用戶組

0: jdbc:hive2://localhost:10000_> grant_ role admin to group hive;

...

INFO : OK

No rows affected (0.162 seconds)

0: jdbc:hive2://localhost:10000_>_

以上操做建立了一個admin角色：

admin : 具備管理員權限，能夠讀寫全部數據庫，並受權給hive組（對應操做系統的組）

4.2建立test表

使用hive用戶登陸Kerberos，經過beeline登陸HiveServer2，建立test表，並插入測試數據

0: jdbc:hive2://localhost:10000_> create_ tabletest (s1 string, s2 string) row format delimited fields terminated by ',';

...

INFO : OK

No rows affected (0.592 seconds)

0: jdbc:hive2://localhost:10000> insert into test values('a','b'),('1','2');

...

INFO : OK

No rows affected (20.123 seconds)

0: jdbc:hive2://localhost:10000_>_

4.3建立測試角色並將角色受權給用戶組

建立兩個角色：

read：只能讀default庫test表，並受權給fayson用戶組

write：只能寫default庫test表，並受權給user_w用戶組

注意：集羣全部節點必須存在fayson和user_w用戶，用戶默認用戶組與用戶名一致，賦權是針對用戶組而不是針對用戶。

[root@ip-172-31-6-148 cdh-shell-master]# id fayson
uid=501(fayson) gid=501(fayson) groups=501(fayson)
[root@ip-172-31-6-148 cdh-shell-master]# useradd user_w
[root@ip-172-31-6-148 cdh-shell-master]# id user_w
uid=502(user_w) gid=502(user_w) groups=502(user_w)
[root@ip-172-31-6-148 cdh-shell-master]#

1.使用hive用戶建立read和write角色，並受權read角色對test表的select權限，write角色對test表的insert權限

0: jdbc:hive2://localhost:10000_> create_ role read;

...

INFO : OK

No rows affected (0.094 seconds)

0: jdbc:hive2://localhost:10000> grant select on table test torole read;

...

INFO : OK

No rows affected (0.1 seconds)

0: jdbc:hive2://localhost:10000> create role write;

...

INFO : OK

No rows affected (0.105 seconds)

0: jdbc:hive2://localhost:10000> grant insert on table test to role write;

...

INFO : OK

No rows affected (0.112 seconds)

0: jdbc:hive2://localhost:10000_>_

2.將read角色受權給fayson用戶組，write角色受權給user_w用戶組

0: jdbc:hive2://localhost:10000> grant role read to group fayson;
...
INFO  : OK
No rows affected (0.187 seconds)
0: jdbc:hive2://localhost:10000> grant role write to group user_w;
...
INFO  : OK
No rows affected (0.101 seconds)
0: jdbc:hive2://localhost:10000>

3.使用kadmin建立fayson和user_w用戶

[root@ip-172-31-6-148 ~]# kadmin.local
Authenticating as principal hive/admin@FAYSON.COM with password.
kadmin.local:  addprinc fayson@FAYSON.COM
WARNING: no policy specified for fayson@FAYSON.COM; defaulting to no policy
Enter password for principal "fayson@FAYSON.COM": 
Re-enter password for principal "fayson@FAYSON.COM": 
Principal "fayson@FAYSON.COM" created.
kadmin.local:  addprinc user_w@FAYSON.COM
WARNING: no policy specified for user_w@FAYSON.COM; defaulting to no policy
Enter password for principal "user_w@FAYSON.COM": 
Re-enter password for principal "user_w@FAYSON.COM": 
Principal "user_w@FAYSON.COM" created.
kadmin.local:

4.4beeline驗證

1.使用fayson用戶登陸Kerberos

[root@ip-172-31-6-148 ~]# kdestroy
[root@ip-172-31-6-148 ~]# kinit fayson
Password for fayson@FAYSON.COM: 
[root@ip-172-31-6-148 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: fayson@FAYSON.COM

Valid starting     Expires            Service principal
09/07/17 02:48:35  09/08/17 02:48:35  krbtgt/FAYSON.COM@FAYSON.COM
        renew until 09/14/17 02:48:35
[root@ip-172-31-6-148 ~]#

經過beeline鏈接HiveServer2進行驗證

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-6-148.fayson.com@FAYSON.COM
...
0: jdbc:hive2://localhost:10000/> show tables;
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.403 seconds)
0: jdbc:hive2://localhost:10000/> select * from test;
...
INFO  : OK
+----------+----------+--+
| test.s1  | test.s2  |
+----------+----------+--+
| a        | b        |
| 1        | 2        |
| 111      | 222      |
| a        | b        |
| 1        | 2        |
| 333      | 5555     |
| eeee     | dddd     |
+----------+----------+--+
7 rows selected (0.282 seconds)
0: jdbc:hive2://localhost:10000/> insert into test values("2", "222");
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User fayson does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->action=insert; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000/>

執行Hive的MapReduce任務

0: jdbc:hive2://localhost:10000/> select count(*) from test;
...
INFO  : OK
+------+--+
| _c0  |
+------+--+
| 7    |
+------+--+
1 row selected (30.688 seconds)
0: jdbc:hive2://localhost:10000/>

2.使用user_w用戶登陸Kerberos

[root@ip-172-31-6-148 ~]# kdestroy
[root@ip-172-31-6-148 ~]# kinit user_w
Password for user_w@FAYSON.COM: 
[root@ip-172-31-6-148 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: user_w@FAYSON.COM

Valid starting     Expires            Service principal
09/07/17 03:01:56  09/08/17 03:01:56  krbtgt/FAYSON.COM@FAYSON.COM
        renew until 09/14/17 03:01:56
[root@ip-172-31-6-148 ~]#

使用beeline登陸HiveServer2驗證

[root@ip-172-31-6-148 ~]# beeline 
Beeline version 1.1.0-cdh5.12.1 by Apache Hive
beeline> !connect jdbc:hive2://localhost:10000/;principal=hive/ip-172-31-6-148.fayson.com@FAYSON.COM
...
0: jdbc:hive2://localhost:10000/> show tables;
...
INFO  : OK
+-----------+--+
| tab_name  |
+-----------+--+
| test      |
+-----------+--+
1 row selected (0.343 seconds)
0: jdbc:hive2://localhost:10000/> select  * from test;
Error: Error while compiling statement: FAILED: SemanticException No valid privileges
 User user_w does not have privileges for QUERY
 The required privileges: Server=server1->Db=default->Table=test->Column=s1->action=select; (state=42000,code=40000)
0: jdbc:hive2://localhost:10000/> insert into test values("2", "333");
...
INFO  : OK
No rows affected (19.379 seconds)
0: jdbc:hive2://localhost:10000/>

驗證總結：

fayson用戶所屬組爲fayson擁有test表讀權限，只能對test表進行select和count操做不能進行insert操做；

user_w用戶所屬組爲user_w擁有test表寫權限，只能對test表進行insert操做不能進行select操做；

4.5HDFS驗證

1.使用fayson用戶登陸Kerberos，進行以下操做

使用HDFS命令對/user/hive/warehouse/test進行cat、ls、put等操做

[root@ip-172-31-6-148 ~]# hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=fayson, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[root@ip-172-31-6-148 ~]# hadoop fs -ls /user/hive/warehouse/test
Found 5 items
-rwxrwx--x+  3 hive hive          8 2017-09-05 12:52 /user/hive/warehouse/test/000000_0
-rwxrwx--x+  3 hive hive          8 2017-09-05 13:44 /user/hive/warehouse/test/000000_0_copy_1
-rwxrwx--x+  3 hive hive          8 2017-09-07 02:36 /user/hive/warehouse/test/000000_0_copy_2
-rwxrwx--x+  3 hive hive          6 2017-09-07 03:04 /user/hive/warehouse/test/000000_0_copy_3
-rwxrwx--x+  3 hive hive         19 2017-09-05 13:01 /user/hive/warehouse/test/test.txt
[root@ip-172-31-6-148 ~]# hadoop fs -cat /user/hive/warehouse/test/test.txt
333,5555
eeee,dddd
[root@ip-172-31-6-148 ~]# hadoop fs -rm /user/hive/warehouse/test/test.txt
rm: Failed to move to trash: hdfs://ip-172-31-6-148.fayson.com:8020/user/hive/warehouse/test/test.txt: Permission denied: user=fayson, access=WRITE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[root@ip-172-31-6-148 ~]# hadoop fs -put a.txt /user/hive/warehouse/test/
put: Permission denied: user=fayson, access=WRITE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[root@ip-172-31-6-148 ~]#

2.使用user_w用戶登陸Kerberos，進行以下操做

[root@ip-172-31-6-148 ~]# kdestroy
[root@ip-172-31-6-148 ~]# kinit user_w
Password for user_w@FAYSON.COM: 
[root@ip-172-31-6-148 ~]# hadoop fs -ls /user/hive/warehouse
ls: Permission denied: user=user_w, access=READ_EXECUTE, inode="/user/hive/warehouse":hive:hive:drwxrwx--x
[root@ip-172-31-6-148 ~]# hadoop fs -ls /user/hive/warehouse/test
ls: Permission denied: user=user_w, access=READ_EXECUTE, inode="/user/hive/warehouse/test":hive:hive:drwxrwx--x
[root@ip-172-31-6-148 ~]# hadoop fs -cat /user/hive/warehouse/test/test.txt
cat: Permission denied: user=user_w, access=READ, inode="/user/hive/warehouse/test/test.txt":hive:hive:-rwxrwx--x
[root@ip-172-31-6-148 ~]# hadoop fs -rm /user/hive/warehouse/test/test.txt
17/09/07 03:21:21 INFO fs.TrashPolicyDefault: Moved: 'hdfs://ip-172-31-6-148.fayson.com:8020/user/hive/warehouse/test/test.txt' to trash at: hdfs://ip-172-31-6-148.fayson.com:8020/user/user_w/.Trash/Current/user/hive/warehouse/test/test.txt
[root@ip-172-31-6-148 ~]# hadoop fs -put a.txt /user/hive/warehouse/test/
[root@ip-172-31-6-148 ~]#

fayson用戶所屬組爲fayson，擁有test表的讀權限，能夠對test表的數據目錄（/user/hive/warehouse/test）瀏覽及查看目錄下文件內容，不能刪除/test/目錄下文件，也不能向test目錄put文件。

user_w用戶所屬組爲user_w，擁有test表的write權限，能夠對test表的數據目錄put文件及刪除數據文件操做，但不能瀏覽及查看目錄下的文件內容。

說明Sentry實現了HDFS的ACL同步。

4.6Hue驗證

1.使用Hue的管理員，添加Hue的測試用戶fayson和user_w

2.使用fayson用戶登陸Hue，驗證read權限

能夠查看test表數據

能夠進行Count操做

不能Insert操做

File Browser瀏覽

不能瀏覽父目錄/user/hive/warehouse

能夠瀏覽test表的數據目錄/user/hive/warehouse/test

能夠查看/user/hive/warehouse/test目錄下文件內容

不能修改/user/hive/warehouse/test目錄下數據文件

3.使用user_w用戶登陸Hue，驗證write權限

不能夠查看test表

不能夠Count操做

能夠向test表插入數據

FileBrowser操做

不能夠瀏覽父目錄/user/hive/warehouse

不能夠瀏覽test表的數據目錄/user/hive/warehouse/test

fayson和user_w用戶均能經過hue界面看到test表，擁有read角色的fayson用戶組能對test進行select和count操做，而且能經過File Browser瀏覽和查看test表的數據目錄/user/hive/warehouse/test。擁有write角色的user_w用戶組只能對test表進行insert操做，但不能經過File Browser瀏覽和查看test表的數據目錄/user/hive/warehouse/test。說明Sentry在命令行的操做和受權在Hue中依舊有效。

4.7Impala驗證

1.使用fayson用戶登陸Kerberos

[root@ip-172-31-6-148 ~]# kinit fayson
Password for fayson@FAYSON.COM: 
[root@ip-172-31-6-148 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: fayson@FAYSON.COM

Valid starting     Expires            Service principal
09/07/17 06:36:05  09/08/17 06:36:05  krbtgt/FAYSON.COM@FAYSON.COM
        renew until 09/14/17 06:36:05
[root@ip-172-31-6-148 ~]# impala-shell 
Starting Impala Shell without Kerberos authentication
...
Connected to ip-172-31-9-33.fayson.com:21000
Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build 5131a031f4aa38c1e50c430373c55ca53e0517b9)
[ip-172-31-9-33.fayson.com:21000] > show tables;
Query: show tables
+------+
| name |
+------+
| test |
+------+
Fetched 1 row(s) in 0.02s
[ip-172-31-9-33.fayson.com:21000] > select * from test;
...
+--------+----------+
| s1     | s2       |
+--------+----------+
| testaa | testbbb  |
| 111    | 222      |
| 222    | 2323     |
| 2      | 333      |
| a      | b        |
| 1      | 2        |
| 1      | test     |
| 2      | fayson   |
| 3      | zhangsan |
| a      | b        |
| 1      | 2        |
+--------+----------+
Fetched 11 row(s) in 0.19s
[ip-172-31-9-33.fayson.com:21000] > select count(*) from test;
...
+----------+
| count(*) |
+----------+
| 11       |
+----------+
Fetched 1 row(s) in 0.14s
[ip-172-31-9-33.fayson.com:21000] > insert into test values('test44','test55');
Query: insert into test values('test44','test55')
Query submitted at: 2017-09-07 06:37:00 (Coordinator: http://ip-172-31-9-33.fayson.com:25000)
ERROR: AuthorizationException: User 'fayson@FAYSON.COM' does not have privileges to execute 'INSERT' on: default.test

[ip-172-31-9-33.fayson.com:21000] >

2.使用user_w用戶登陸Kerberos

登陸Impala-shell進行操做

[root@ip-172-31-6-148 ~]# impala-shell 
...
***********************************************************************************
Welcome to the Impala shell.
(Impala Shell v2.9.0-cdh5.12.1 (5131a03) built on Thu Aug 24 09:27:32 PDT 2017)

***********************************************************************************
[Not connected] > connect ip-172-31-9-33.fayson.com:21000;
Connected to ip-172-31-9-33.fayson.com:21000
Server version: impalad version 2.9.0-cdh5.12.1 RELEASE (build 5131a031f4aa38c1e50c430373c55ca53e0517b9)
[ip-172-31-9-33.fayson.com:21000] > show tables;
Query: show tables
+------+
| name |
+------+
| test |
+------+
Fetched 1 row(s) in 0.29s
[ip-172-31-9-33.fayson.com:21000] > select * from test;
Query: select * from test
Query submitted at: 2017-09-07 06:31:23 (Coordinator: http://ip-172-31-9-33.fayson.com:25000)
ERROR: AuthorizationException: User 'user_w@FAYSON.COM' does not have privileges to execute 'SELECT' on: default.test

[ip-172-31-9-33.fayson.com:21000] > insert into test values('222','2323');
Query: insert into test values('222','2323')
Query submitted at: 2017-09-07 06:32:07 (Coordinator: http://ip-172-31-9-33.fayson.com:25000)
Query progress can be monitored at: http://ip-172-31-9-33.fayson.com:25000/query_plan?query_id=ec406e621c7534c7:6bcbbd5300000000
Modified 1 row(s) in 0.63s
[ip-172-31-9-33.fayson.com:21000] >

驗證總結：

Impala與Sentry集成後能夠使用Sentry來進行權限管理，擁有read角色的fayson用戶組只能對test表進行select和count操做不能插入數據，擁有write角色的user_w

用戶組只能對test表插入數據不能進行select和count操做。說明Sentry實現了Hive權限與Impala的同步。