Apache Sentry實戰之旅(一)—— Impala+Sentry整合

Impala默認是以impala這個超級用戶運行服務,執行DMLDDL操做的,要實現不一樣用戶之間細粒度的權限控制,須要與Sentry整合。SentryApache下的一個開源項目,它基於RBAC的受權模型實現了權限控制,Impala與它整合之後,就能實現不一樣用戶之間在應用層的權限認證,從而控制用戶的DMLDDLDCL操做權限。Sentry爲確保數據安全,提供了一個統一平臺,可使用現有的Hadoop Kerberos實現安全認證,同時,經過HiveImpala訪問數據時可使用一樣的Sentry協議。本文會對Sentry進行簡單的介紹並演示Impala+Sentry整合後的實際效果。html

Sentry介紹

Apache SentryCloudera公司發佈的一個用於權限控制的Hadoop開源組件,已於2016年3月順利從孵化器畢業,成爲Apache頂級項目。它基於RBAC的受權模型實現了細粒度的權限控制,Sentry目前能夠與Apache HiveHive Metastore/HCatalogApache SolrImpalaHDFS(僅限於Hive表數據)整合實現權限控制。如下是Sentry以及它與Hadoop其餘組件整合的一張概覽圖:java

這張概覽圖的成員能夠按身份分爲兩部分:mysql

一、Sentry服務組件:git

  • Sentry Server: 服務提供層。它基於RPC協議實現,主要負責管理權限數據,提供了安全的查詢和保存元數據的RPC接口
  • Data Engine:數據引擎層。它有兩個職責:一是負責加載Sentry插件,二是攔截全部訪問資源的客戶端(如HiveImpala)請求,並轉發到Sentry Plugin中進行權限驗證
  • Sentry Plugin:權限認證層。這是Sentry受權的核心組件,負責斷定從數據處理層獲取的權限信息與服務提供層已保存的權限信息是否匹配
  • Policy Metadata:數據存儲層。負責權限數據的存儲,Sentry支持使用ini文件和關係型DB來存儲權限數據。當使用ini文件時,這個文件能夠存在於本地路徑或者HDFS中,基於文件的方式在使用程序修改過程當中會存在資源競爭,不利於維護;當使用關係型DB時,Sentry將權限信息持久化到DB中,併爲應用層提供API接口方便建立、查詢、更新和刪除這些數據。Sentry可使用不少後端的數據庫,例如MySQLPostgres等等,它使用ORMDataNucleus來完成持久化操做。

二、Sentry使用者組件:github

Impala、Hive、Solr爲表明的各個組件組成了Sentry使用者組件,在Sentry中,這些組件都是以客戶端的身份調用Sentry服務的。web

簡單地講,Sentry是用一種相似C/S架構的方式來向外提供服務,全部使用Sentry的組件均可以被視爲一個Sentry客戶端,使用RPC協議來與Sentry Server端交互。使用了Sentry以後,這些客戶端grant/revoke管理的權限徹底被Sentry接管,grant/revoke的執行也徹底在Sentry中實現。對於全部引擎的受權信息也存儲在由Sentry設定的統一的數據庫中,這樣全部引擎的權限就實現了集中管理。sql

Sentry受權包括如下幾種角色:shell

  • 資源。能夠是Server、Database、Table或者URL(例如:HDFS或者本地路徑)。Sentry1.5中支持對列進行受權
  • 權限。受權訪問某一個資源的規則
  • 角色。角色是一系列權限的集合
  • 用戶和組。一個組是一系列用戶的集合。Sentry 的組映射是能夠擴展的。默認狀況下,Sentry使用Hadoop的組映射(能夠是操做系統組或者LDAP中的組)。Sentry容許你將用戶和組進行關聯,你能夠將一系列的用戶放入到一個組中。Sentry不能直接給一個用戶或組受權,須要將權限受權給角色,角色能夠受權給一個組而不是一個用戶

安裝Sentry Server

環境

Sentry版本:1.5.1-cdh5.16.1數據庫

JDK版本:jdk1.8.0_212express

Maven版本:apache-maven-3.6.1

Impala版本:2.12.0-cdh5.16.1

Hadoop版本:hadoop-2.6.0-cdh5.16.1

編譯安裝Sentry Server

接下來使用maven編譯生成Sentry安裝包文件。步驟以下:

一、下載源碼:

git clone https://github.com/cloudera/sentry.git

切換到1.5.1-cdh5.16.1這個tag

git checkout -b cdh5.16.1-release cdh5.16.2-release

源碼結構:

二、編譯打包:

mvn -Dmaven.test.skip=true clean package

編譯打包完成後,生成的Sentry安裝包在下圖所標識的目錄下:

三、設置環境變量:

解壓Sentry壓縮包到指定目錄下,同時下載hadoop-2.6.0-cdh5.16.1.tar.gz並解壓,編輯/etc/profile,設置HadoopSentry環境變量:

HADOOP_HOME=/data/sentry/hadoop-2.6.0-cdh5.16.1
HADOOP_LIBEXEC_DIR=${HADOOP_HOME}/libexec
SENTRY_HOME=/data/sentry/apache-sentry-1.5.1-cdh5.16.1-bin
export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HADOOP_LIBEXEC_DIR:$SENTRY_HOME/bin:$PATH

四、配置sentry-site.xml

轉到Sentry解壓目錄的conf文件夾下,修改sentry-site.xml配置文件:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

<configuration>
   <property>
      <name>sentry.service.server.rpc-address</name>
      <value>hadoop21-test1-rgtj5-tj1</value>
  </property>

  <property>
      <name>sentry.service.server.rpc-port</name>
      <value>8038</value>
  </property>

  <property>
      <name>sentry.service.admin.group</name>
      <value>hadoop</value>
  </property>

  <property>
      <name>sentry.service.allow.connect</name>
      <value>hadoop</value>
  </property>

  <property>
      <name>sentry.store.group.mapping</name>
      <value>org.apache.sentry.provider.common.HadoopGroupMappingService</value>
  </property>
      
  <property>
      <name>sentry.service.reporting</name>
      <value>JMX</value>
  </property>

  <property>
      <name>sentry.service.web.enable</name>
      <value>true</value>
  </property>

  <property> 
      <name>sentry.service.web.port</name>  
      <value>51000</value> 
  </property>  

  <property> 
      <name>sentry.service.web.authentication.type</name>  
      <value>NONE</value> 
  </property> 
    
  <property>
      <name>sentry.verify.schema.version</name>
      <value>true</value>  
  </property>

  <property>
    <name>sentry.service.security.mode</name>
    <value>none</value>
  </property>

  <property>
    <name>sentry.store.jdbc.url</name>
    <value>jdbc:mysql://localhost:3306/sentry_test?useSSL=false</value>
  </property>

  <property>
      <name>sentry.store.jdbc.driver</name>
      <value>com.mysql.jdbc.Driver</value>
  </property>

  <property>
      <name>sentry.store.jdbc.user</name>
      <value>root</value>
  </property>

  <property>
      <name>sentry.store.jdbc.password</name>
      <value>123456</value>
  </property>
</configuration>

五、建立MySQL數據庫表:

CREATE DATABASE `sentry_test` /*!40100 DEFAULT CHARACTER SET utf8 */;

六、初始化Sentry數據庫表:

mysql-connector-java-5.1.47.jar放到Sentry解壓目錄的lib文件夾下,而後執行如下命令建立Sentry數據庫表:

sentry --command schema-tool --conffile  ${SENTRY_HOME}/conf/sentry-site.xml --dbType mysql --initSchema

顯示如下信息表示鏈接到數據庫並初始化數據庫表成功:

Sentry store connection URL:     jdbc:mysql://localhost:3306/sentry_test?useSSL=false
Sentry store Connection Driver :         com.mysql.jdbc.Driver
Sentry store connection User:    root
Starting sentry store schema initialization to 1.5.0-cdh5-2
Initialization script sentry-mysql-1.5.0-cdh5-2.sql
Connecting to jdbc:mysql://localhost:3306/sentry_test?useSSL=false
Connected to: MySQL (version 5.6.24-72.2-log)
Driver: MySQL Connector Java (version mysql-connector-java-5.1.47 ( Revision: fe1903b1ecb4a96a917f7ed3190d80c049b1de29 ))
Transaction isolation: TRANSACTION_REPEATABLE_READ
Autocommit status: true
No rows affected (0.006 seconds)
No rows affected (0.001 seconds)
No rows affected (0.002 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.004 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.001 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.003 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.016 seconds)
No rows affected (0.007 seconds)
No rows affected (0.006 seconds)
No rows affected (0.007 seconds)
No rows affected (0.006 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.006 seconds)
No rows affected (0.006 seconds)
No rows affected (0.003 seconds)
No rows affected (0.005 seconds)
No rows affected (0.002 seconds)
No rows affected (0.004 seconds)
1 row affected (0.002 seconds)
No rows affected (0.003 seconds)
No rows affected (0.007 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
No rows affected (0.006 seconds)
No rows affected (0.005 seconds)
No rows affected (0.002 seconds)
No rows affected (0.006 seconds)
No rows affected (0.002 seconds)
No rows affected (0.004 seconds)
No rows affected (0.003 seconds)
No rows affected (0.004 seconds)
No rows affected (0.005 seconds)
No rows affected (0.003 seconds)
No rows affected (0.006 seconds)
No rows affected (0.006 seconds)
No rows affected (0.006 seconds)
No rows affected (0.003 seconds)
No rows affected (0.003 seconds)
No rows affected (0.006 seconds)
No rows affected (0.004 seconds)
No rows affected (0.004 seconds)
No rows affected (0.002 seconds)
No rows affected (0.006 seconds)
No rows affected (0.005 seconds)
No rows affected (0.004 seconds)
No rows affected (0.006 seconds)
No rows affected (0.002 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
No rows affected (0.005 seconds)
Closing: 0: jdbc:mysql://localhost:3306/sentry_test?useSSL=false
Initialization script completed
Sentry schemaTool completed

七、運行Sentry命令,啓動Sentry服務端:

nohup sentry --command service --conffile ${SENTRY_HOME}/conf/sentry-site.xml>sentry.out 2>&1 &

在瀏覽器輸入如下地址訪問Sentry Web UI,驗證是否安裝成功:

http://localhost:51000/

Web UI以下圖所示:

Impala+Sentry整合

一、引入Sentry依賴

apache-sentry-1.5.1-cdh5.16.1-bin/lib目錄下相關jar拷貝到/usr/lib/impala/lib目錄下,或者使用以下命令創建Sentry jar包的軟連接也行:

#!/bin/bash
SENTRY_HOME=/data/impala/apache-sentry-1.5.1-cdh5.16.1-bin

sudo rm -rf /usr/lib/impala/lib/sentry-*.jar

sudo ln -s $SENTRY_HOME/lib/sentry-binding-hive-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-binding-hive.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-common.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-model-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-db.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-model-kafka-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-kafka.jar
sudo ln -s $SENTRY_HOME/lib/sentry-core-model-search-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-search.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-common.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-db.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-kafka-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-kafka.jar
sudo ln -s $SENTRY_HOME/lib/sentry-policy-search-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-search.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-cache-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-cache.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-common.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-db-sh.jar
sudo ln -s $SENTRY_HOME/lib/sentry-provider-file-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-file.jar

最終的Sentry jar包依賴以下:

lrwxrwxrwx 1 root root       90 Jul  6 11:00 sentry-binding-hive.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-binding-hive-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       89 Jul  6 11:00 sentry-core-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-common-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       91 Jul  6 11:00 sentry-core-model-db.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-db-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       94 Jul  6 11:00 sentry-core-model-kafka.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-kafka-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       95 Jul  6 11:00 sentry-core-model-search.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-search-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       91 Jul  6 11:00 sentry-policy-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-common-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       87 Jul  6 11:00 sentry-policy-db.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-db-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       90 Jul  6 11:00 sentry-policy-kafka.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-kafka-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       91 Jul  6 11:00 sentry-policy-search.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-search-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       92 Jul  6 11:00 sentry-provider-cache.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-cache-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       93 Jul  6 11:00 sentry-provider-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-common-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       89 Jul  6 11:00 sentry-provider-db-sh.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-db-1.5.1-cdh5.16.1.jar
lrwxrwxrwx 1 root root       91 Jul  6 11:00 sentry-provider-file.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-file-1.5.1-cdh5.16.1.jar

注意:使用sentryImpalaCDH版本必定要對應上,好比我這裏使用的是Impala版本是CDH5.16.1,那麼sentry的也須要是,不然會由於jar版本問題致使Impala啓動過程當中拋出一些異常,好比:

java.lang.NoClassDefFoundError: org/apache/sentry/provider/cache/SentryPrivilegeCache

若是不知道Impala依賴的一些外部組件的版本,能夠在Impala源碼的Impala/bin/impala-config.sh裏找到,如Impala cdh5-2.12.0_5.16.1版本的依賴信息在該配置文件裏定義以下:

# Versions of Hadoop ecosystem dependencies.
# ------------------------------------------
export CDH_MAJOR_VERSION=5
export IMPALA_HADOOP_VERSION=2.6.0-cdh5.16.1
unset IMPALA_HADOOP_URL
export IMPALA_HBASE_VERSION=1.2.0-cdh5.16.1
unset IMPALA_HBASE_URL
export IMPALA_HIVE_VERSION=1.1.0-cdh5.16.1
unset IMPALA_HIVE_URL
export IMPALA_SENTRY_VERSION=1.5.1-cdh5.16.1
unset IMPALA_SENTRY_URL
export IMPALA_PARQUET_VERSION=1.5.0-cdh5.16.1
export IMPALA_LLAMA_MINIKDC_VERSION=1.0.0
unset IMPALA_LLAMA_MINIKDC_URL
export IMPALA_KITE_VERSION=1.0.0-cdh5.16.1

二、建立sentry-site.xml

apache-sentry-1.5.1-cdh5.16.1-bin/conf目錄下的sentry-site.xml.service.template文件拷貝到/etc/impala/conf目錄下:

# 拷貝
cp apache-sentry-1.5.1-cdh5.16.1-bin/conf/sentry-site.xml.service.template /etc/impala/conf/
# 重命名
cd /etc/impala/conf/
mv sentry-site.xml.service.template sentry-site.xml

編輯sentry-site.xml爲如下內容:

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
-->

<!-- WARNING!!! This file is provided for documentation purposes ONLY!              -->
<!-- WARNING!!! You should copy to sentry-site.xml and make modification instead.   -->

<configuration>

  <!--Sentry Server端口-->
  <property>
     <name>sentry.service.client.server.rpc-port</name>
     <value>8038</value>
  </property>

  <!--Sentry Server服務器地址-->
  <property>
     <name>sentry.service.client.server.rpc-addresses</name>
     <value>hadoop21-test1-rgtj5-tj1</value>
  </property>

  <!--客戶端鏈接Sentry Server超時時間,以毫秒爲單位,默認爲200000毫秒-->
  <property>
     <name>sentry.service.client.server.rpc-connection-timeout</name>
     <value>200000</value>
  </property>

  <!--權限存儲方式:數據庫或者ini配置文件-->
  <property>
    <name>sentry.hive.provider.backend</name>
    <value>org.apache.sentry.provider.db.SimpleDBProviderBackend</value>
  </property>

  <!--權限認證方式,支持Kerberos認證,設置爲none表示不啓用認證  -->
  <property>
     <name>sentry.service.security.mode</name>
     <value>none</value>
  </property>

</configuration>

三、啓用權限認證

編輯/etc/default/impala配置文件,修改以下兩個配置啓用Sentry權限認證:

  • 修改IMPALA_CATALOG_ARGS選項,增長-sentry_config=/etc/impala/conf/sentry-site.xml配置
  • 修改IMPALA_SERVER_ARGS選項,增長-sentry_config=/etc/impala/conf/sentry-site.xml-server_name=sentryserver配置

配置文件最終內容以下:

IMPALA_CATALOG_SERVICE_HOST=hadoop21-test1-rgtj5-tj1
IMPALA_STATE_STORE_HOST=hadoop21-test1-rgtj5-tj1
IMPALA_STATE_STORE_PORT=24000
IMPALA_BACKEND_PORT=22000
IMPALA_LOG_DIR=/data/log/impala

IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} -sentry_config=/etc/impala/conf/sentry-site.xml"
IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}"
IMPALA_SERVER_ARGS=" \
    -log_dir=${IMPALA_LOG_DIR} \
    -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \
    -state_store_port=${IMPALA_STATE_STORE_PORT} \
    -use_statestore \
    -state_store_host=${IMPALA_STATE_STORE_HOST} \
    -be_port=${IMPALA_BACKEND_PORT} \
    -kudu_master_hosts=hadoop21-test1-rgtj5-tj1:7051,hadoop20-test1-rgtj5-tj1:7051,hadoop22-test1-rgtj5-tj1:7051,hadoop-bi06-test1-rgtj5-tj1:7051,hadoop-bi07-test1-rgtj5-tj1:7051 \
    -sentry_config=/etc/impala/conf/sentry-site.xml \
    -server_name=sentryserver"

ENABLE_CORE_DUMPS=true

# LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib
# MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar
# IMPALA_BIN=/usr/lib/impala/sbin
# IMPALA_HOME=/usr/lib/impala
# HIVE_HOME=/usr/lib/hive
# HBASE_HOME=/usr/lib/hbase
# IMPALA_CONF_DIR=/etc/impala/conf
# HADOOP_CONF_DIR=/etc/impala/conf
# HIVE_CONF_DIR=/etc/impala/conf
# HBASE_CONF_DIR=/etc/impala/conf

四、重啓Impala服務,驗證權限

重啓Impala服務:

sudo service impala-state-store restart
sudo service impala-catalog restart
sudo service impala-server restart

打開impala-shell,驗證權限配置是否成功,具體操做以下:

(1)切換到hadoop用戶,打開impala-shell,建立一個admin角色:

[hadoop21-test1-rgtj5-tj1:21000] > create role admin_role;
Query: create role admin_role
Fetched 0 row(s) in 0.35s

(2)爲admin角色賦予超級權限:

[hadoop21-test1-rgtj5-tj1:21000] > GRANT ALL ON SERVER sentryserver TO ROLE admin_role;
Query: GRANT ALL ON SERVER sentryserver TO ROLE admin_role
Query submitted at: 2019-07-06 10:40:11 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=15475b39691bd167:66c1403300000000
Fetched 0 row(s) in 0.13s

(3)將admin角色受權給hadoop用戶組:

[hadoop21-test1-rgtj5-tj1:21000] > GRANT ROLE admin_role TO GROUP hadoop;
Query: GRANT ROLE admin_role TO GROUP hadoop
Query submitted at: 2019-07-06 10:41:53 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=434bb908587eaf31:65887a5a00000000
Fetched 0 row(s) in 0.11s

(4)建立一個test庫和test表,並插入測試數據:

[hadoop21-test1-rgtj5-tj1:21000] > create database test;    
Query: create database test
Fetched 0 row(s) in 0.29s
[hadoop21-test1-rgtj5-tj1:21000] > use test;
Query: use test
[hadoop21-test1-rgtj5-tj1:21000] > CREATE TABLE test(x INT, y STRING) STORED AS PARQUET; 
Query: CREATE TABLE test(x INT, y STRING) STORED AS PARQUET
Fetched 0 row(s) in 0.16s
[hadoop21-test1-rgtj5-tj1:21000] > INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three'); 
Query: INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three')
Query submitted at: 2019-07-06 11:18:33 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=ce4e7f66f1209531:641f39a900000000
Modified 3 row(s) in 5.47s

由於hadoop用戶是超級管理員並擁有ALL的權限,所以執行如下SELECT語句便能很快看到咱們剛插入的數據:

[hadoop21-test1-rgtj5-tj1:21000] > select * from test;
Query: select * from test
Query submitted at: 2019-07-06 11:19:50 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000)
Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=34e4b5594e3d0c6:8cfb1acb00000000
+---+-------+
| x | y     |
+---+-------+
| 1 | one   |
| 2 | two   |
| 3 | three |
+---+-------+
Fetched 3 row(s) in 1.87s

接着咱們切換到root用戶,運行impala-shell,對咱們剛剛建立的test庫進行操做:

[hadoop21-test1-rgtj5-tj1:21000] > use test;
Query: use test
ERROR: AuthorizationException: User 'root' does not have privileges to access: test.*.*

提示root用戶沒有操做test庫的權限,至此,說明Sentry權限認證已經生效。

各類受權操做語法以下:

建立角色:CREATE ROLE <role name>
組分配角色:GRANT ROLE <role name> TO GROUP <group name>
服務級賦權:GRANT <ALL|SELECT|UPDATE> ON SERVER <server name> TO ROLE <role name>
數據庫賦權:GRANT <ALL|SELECT|UPDATE> ON DATABASE <database name> TO ROLE <role name>
表賦權:GRANT <ALL|SELECT|UPDATE> ON TABLE <database name>.<table name> TO ROLE <role name>
字段權限:GRANT SELECT(column name)ON TABLE <table name> TO ROLE <role name>;
回收組權限:REVOKE ROLE <role name> FROM GROUP <group name>
回收字段權限:REVOKE SELECT <column name> ON TABLE <table name> FROM ROLE <role name>;
回收數據庫權限:REVOKE <ALL|SELECT|UPDATE> ON DATABASE <database name> FROM ROLE <role name>
查看某個角色的權限:SHOW GRANT ROLE <role name>
各類查看命令:
SHOW ROLES;
SHOW CURRENT ROLES;
SHOW ROLE GRANT GROUP <group name>;
SHOW GRANT ROLE <role name>;
SHOW GRANT ROLE <role name> on OBJECT <object name>;

總結

一、Impala服務的權限安全,認證(Kerberos/LDAP)是第一步,受權(Sentry)是第二步。若是要啓用受權,必須先啓用認證。本文在測試過程當中不啓用認證而只啓用Sentry受權,強烈不建議在生產系統中這樣使用,由於若是沒有用戶認證,受權沒有任何意義形同虛設,用戶能夠隨意使用任何超級用戶登陸Impala,並不會作密碼校驗。

二、Impala是不區分底層存儲用戶的,Sentry控制的只是Impala應用層的操做權限,底層操做HDFS的仍是impala用戶,也就是啓動impalad的用戶。不區分底層存儲用戶主要是由於C++libhdfsHadoop2時還不支持doAs

三、Impala中的受權處理過程相似於Hive中的受權處理過程,主要的區別在於權限信息的緩存。ImpalaCatalog服務管理並緩存數據庫schema元數據和Sentry權限元數據,並將其傳播到全部Impala Server節點。所以,Impala中的受權驗證在本地進行,並且速度更快。能夠用下圖進行歸納:

參考資料

官方資料:

博客文章:

Hadoop實操公衆號:

相關文章
相關標籤/搜索