Impala
默認是以impala
這個超級用戶運行服務,執行DML
和DDL
操做的,要實現不一樣用戶之間細粒度的權限控制,須要與Sentry
整合。Sentry
是Apache
下的一個開源項目,它基於RBAC
的受權模型實現了權限控制,Impala
與它整合之後,就能實現不一樣用戶之間在應用層的權限認證,從而控制用戶的DML
、DDL
、DCL
操做權限。Sentry
爲確保數據安全,提供了一個統一平臺,可使用現有的Hadoop Kerberos
實現安全認證,同時,經過Hive
或Impala
訪問數據時可使用一樣的Sentry
協議。本文會對Sentry
進行簡單的介紹並演示Impala+Sentry
整合後的實際效果。html
Apache Sentry
是Cloudera
公司發佈的一個用於權限控制的Hadoop
開源組件,已於2016年3月順利從孵化器畢業,成爲Apache
頂級項目。它基於RBAC
的受權模型實現了細粒度的權限控制,Sentry
目前能夠與Apache Hive
、Hive Metastore
/HCatalog
、Apache Solr
、Impala
和HDFS
(僅限於Hive
表數據)整合實現權限控制。如下是Sentry
以及它與Hadoop
其餘組件整合的一張概覽圖:java
這張概覽圖的成員能夠按身份分爲兩部分:mysql
一、Sentry服務組件:git
Sentry Server
: 服務提供層。它基於RPC
協議實現,主要負責管理權限數據,提供了安全的查詢和保存元數據的RPC
接口Data Engine
:數據引擎層。它有兩個職責:一是負責加載Sentry
插件,二是攔截全部訪問資源的客戶端(如Hive
或Impala
)請求,並轉發到Sentry Plugin
中進行權限驗證Sentry Plugin
:權限認證層。這是Sentry
受權的核心組件,負責斷定從數據處理層獲取的權限信息與服務提供層已保存的權限信息是否匹配Policy Metadata
:數據存儲層。負責權限數據的存儲,Sentry
支持使用ini
文件和關係型DB
來存儲權限數據。當使用ini
文件時,這個文件能夠存在於本地路徑或者HDFS
中,基於文件的方式在使用程序修改過程當中會存在資源競爭,不利於維護;當使用關係型DB
時,Sentry
將權限信息持久化到DB
中,併爲應用層提供API
接口方便建立、查詢、更新和刪除這些數據。Sentry
可使用不少後端的數據庫,例如MySQL
、Postgres
等等,它使用ORM
庫DataNucleus
來完成持久化操做。二、Sentry使用者組件:github
以Impala、Hive、Solr
爲表明的各個組件組成了Sentry
使用者組件,在Sentry
中,這些組件都是以客戶端的身份調用Sentry
服務的。web
簡單地講,Sentry
是用一種相似C/S
架構的方式來向外提供服務,全部使用Sentry
的組件均可以被視爲一個Sentry
客戶端,使用RPC
協議來與Sentry Server
端交互。使用了Sentry
以後,這些客戶端grant/revoke
管理的權限徹底被Sentry
接管,grant/revoke
的執行也徹底在Sentry
中實現。對於全部引擎的受權信息也存儲在由Sentry
設定的統一的數據庫中,這樣全部引擎的權限就實現了集中管理。sql
Sentry
受權包括如下幾種角色:shell
Server、Database、Table
或者URL
(例如:HDFS
或者本地路徑)。Sentry1.5
中支持對列進行受權Sentry
的組映射是能夠擴展的。默認狀況下,Sentry
使用Hadoop
的組映射(能夠是操做系統組或者LDAP
中的組)。Sentry
容許你將用戶和組進行關聯,你能夠將一系列的用戶放入到一個組中。Sentry
不能直接給一個用戶或組受權,須要將權限受權給角色,角色能夠受權給一個組而不是一個用戶Sentry
版本:1.5.1-cdh5.16.1
數據庫
JDK
版本:jdk1.8.0_212
express
Maven
版本:apache-maven-3.6.1
Impala
版本:2.12.0-cdh5.16.1
Hadoop
版本:hadoop-2.6.0-cdh5.16.1
接下來使用maven
編譯生成Sentry
安裝包文件。步驟以下:
一、下載源碼:
git clone https://github.com/cloudera/sentry.git
切換到1.5.1-cdh5.16.1
這個tag
:
git checkout -b cdh5.16.1-release cdh5.16.2-release
源碼結構:
二、編譯打包:
mvn -Dmaven.test.skip=true clean package
編譯打包完成後,生成的Sentry
安裝包在下圖所標識的目錄下:
三、設置環境變量:
解壓Sentry
壓縮包到指定目錄下,同時下載hadoop-2.6.0-cdh5.16.1.tar.gz並解壓,編輯/etc/profile
,設置Hadoop
和Sentry
環境變量:
HADOOP_HOME=/data/sentry/hadoop-2.6.0-cdh5.16.1 HADOOP_LIBEXEC_DIR=${HADOOP_HOME}/libexec SENTRY_HOME=/data/sentry/apache-sentry-1.5.1-cdh5.16.1-bin export PATH=$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HADOOP_LIBEXEC_DIR:$SENTRY_HOME/bin:$PATH
四、配置sentry-site.xml
:
轉到Sentry
解壓目錄的conf
文件夾下,修改sentry-site.xml
配置文件:
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <configuration> <property> <name>sentry.service.server.rpc-address</name> <value>hadoop21-test1-rgtj5-tj1</value> </property> <property> <name>sentry.service.server.rpc-port</name> <value>8038</value> </property> <property> <name>sentry.service.admin.group</name> <value>hadoop</value> </property> <property> <name>sentry.service.allow.connect</name> <value>hadoop</value> </property> <property> <name>sentry.store.group.mapping</name> <value>org.apache.sentry.provider.common.HadoopGroupMappingService</value> </property> <property> <name>sentry.service.reporting</name> <value>JMX</value> </property> <property> <name>sentry.service.web.enable</name> <value>true</value> </property> <property> <name>sentry.service.web.port</name> <value>51000</value> </property> <property> <name>sentry.service.web.authentication.type</name> <value>NONE</value> </property> <property> <name>sentry.verify.schema.version</name> <value>true</value> </property> <property> <name>sentry.service.security.mode</name> <value>none</value> </property> <property> <name>sentry.store.jdbc.url</name> <value>jdbc:mysql://localhost:3306/sentry_test?useSSL=false</value> </property> <property> <name>sentry.store.jdbc.driver</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>sentry.store.jdbc.user</name> <value>root</value> </property> <property> <name>sentry.store.jdbc.password</name> <value>123456</value> </property> </configuration>
五、建立MySQL
數據庫表:
CREATE DATABASE `sentry_test` /*!40100 DEFAULT CHARACTER SET utf8 */;
六、初始化Sentry
數據庫表:
將mysql-connector-java-5.1.47.jar
放到Sentry
解壓目錄的lib
文件夾下,而後執行如下命令建立Sentry
數據庫表:
sentry --command schema-tool --conffile ${SENTRY_HOME}/conf/sentry-site.xml --dbType mysql --initSchema
顯示如下信息表示鏈接到數據庫並初始化數據庫表成功:
Sentry store connection URL: jdbc:mysql://localhost:3306/sentry_test?useSSL=false Sentry store Connection Driver : com.mysql.jdbc.Driver Sentry store connection User: root Starting sentry store schema initialization to 1.5.0-cdh5-2 Initialization script sentry-mysql-1.5.0-cdh5-2.sql Connecting to jdbc:mysql://localhost:3306/sentry_test?useSSL=false Connected to: MySQL (version 5.6.24-72.2-log) Driver: MySQL Connector Java (version mysql-connector-java-5.1.47 ( Revision: fe1903b1ecb4a96a917f7ed3190d80c049b1de29 )) Transaction isolation: TRANSACTION_REPEATABLE_READ Autocommit status: true No rows affected (0.006 seconds) No rows affected (0.001 seconds) No rows affected (0.002 seconds) No rows affected (0.001 seconds) No rows affected (0.001 seconds) No rows affected (0.001 seconds) No rows affected (0.004 seconds) No rows affected (0.001 seconds) No rows affected (0.001 seconds) No rows affected (0.001 seconds) No rows affected (0.004 seconds) No rows affected (0.004 seconds) No rows affected (0.004 seconds) No rows affected (0.003 seconds) No rows affected (0.004 seconds) No rows affected (0.004 seconds) No rows affected (0.016 seconds) No rows affected (0.007 seconds) No rows affected (0.006 seconds) No rows affected (0.007 seconds) No rows affected (0.006 seconds) No rows affected (0.005 seconds) No rows affected (0.004 seconds) No rows affected (0.004 seconds) No rows affected (0.004 seconds) No rows affected (0.005 seconds) No rows affected (0.004 seconds) No rows affected (0.004 seconds) No rows affected (0.006 seconds) No rows affected (0.006 seconds) No rows affected (0.003 seconds) No rows affected (0.005 seconds) No rows affected (0.002 seconds) No rows affected (0.004 seconds) 1 row affected (0.002 seconds) No rows affected (0.003 seconds) No rows affected (0.007 seconds) No rows affected (0.005 seconds) No rows affected (0.004 seconds) No rows affected (0.005 seconds) No rows affected (0.005 seconds) No rows affected (0.005 seconds) No rows affected (0.006 seconds) No rows affected (0.005 seconds) No rows affected (0.002 seconds) No rows affected (0.006 seconds) No rows affected (0.002 seconds) No rows affected (0.004 seconds) No rows affected (0.003 seconds) No rows affected (0.004 seconds) No rows affected (0.005 seconds) No rows affected (0.003 seconds) No rows affected (0.006 seconds) No rows affected (0.006 seconds) No rows affected (0.006 seconds) No rows affected (0.003 seconds) No rows affected (0.003 seconds) No rows affected (0.006 seconds) No rows affected (0.004 seconds) No rows affected (0.004 seconds) No rows affected (0.002 seconds) No rows affected (0.006 seconds) No rows affected (0.005 seconds) No rows affected (0.004 seconds) No rows affected (0.006 seconds) No rows affected (0.002 seconds) No rows affected (0.005 seconds) No rows affected (0.005 seconds) No rows affected (0.005 seconds) Closing: 0: jdbc:mysql://localhost:3306/sentry_test?useSSL=false Initialization script completed Sentry schemaTool completed
七、運行Sentry
命令,啓動Sentry
服務端:
nohup sentry --command service --conffile ${SENTRY_HOME}/conf/sentry-site.xml>sentry.out 2>&1 &
在瀏覽器輸入如下地址訪問Sentry Web UI
,驗證是否安裝成功:
http://localhost:51000/
Web UI
以下圖所示:
將apache-sentry-1.5.1-cdh5.16.1-bin/lib
目錄下相關jar
拷貝到/usr/lib/impala/lib
目錄下,或者使用以下命令創建Sentry jar
包的軟連接也行:
#!/bin/bash SENTRY_HOME=/data/impala/apache-sentry-1.5.1-cdh5.16.1-bin sudo rm -rf /usr/lib/impala/lib/sentry-*.jar sudo ln -s $SENTRY_HOME/lib/sentry-binding-hive-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-binding-hive.jar sudo ln -s $SENTRY_HOME/lib/sentry-core-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-common.jar sudo ln -s $SENTRY_HOME/lib/sentry-core-model-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-db.jar sudo ln -s $SENTRY_HOME/lib/sentry-core-model-kafka-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-kafka.jar sudo ln -s $SENTRY_HOME/lib/sentry-core-model-search-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-core-model-search.jar sudo ln -s $SENTRY_HOME/lib/sentry-policy-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-common.jar sudo ln -s $SENTRY_HOME/lib/sentry-policy-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-db.jar sudo ln -s $SENTRY_HOME/lib/sentry-policy-kafka-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-kafka.jar sudo ln -s $SENTRY_HOME/lib/sentry-policy-search-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-policy-search.jar sudo ln -s $SENTRY_HOME/lib/sentry-provider-cache-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-cache.jar sudo ln -s $SENTRY_HOME/lib/sentry-provider-common-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-common.jar sudo ln -s $SENTRY_HOME/lib/sentry-provider-db-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-db-sh.jar sudo ln -s $SENTRY_HOME/lib/sentry-provider-file-1.5.1-cdh5.16.1.jar /usr/lib/impala/lib/sentry-provider-file.jar
最終的Sentry jar
包依賴以下:
lrwxrwxrwx 1 root root 90 Jul 6 11:00 sentry-binding-hive.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-binding-hive-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 89 Jul 6 11:00 sentry-core-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-common-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-core-model-db.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-db-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 94 Jul 6 11:00 sentry-core-model-kafka.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-kafka-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 95 Jul 6 11:00 sentry-core-model-search.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-core-model-search-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-policy-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-common-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 87 Jul 6 11:00 sentry-policy-db.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-db-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 90 Jul 6 11:00 sentry-policy-kafka.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-kafka-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-policy-search.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-policy-search-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 92 Jul 6 11:00 sentry-provider-cache.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-cache-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 93 Jul 6 11:00 sentry-provider-common.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-common-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 89 Jul 6 11:00 sentry-provider-db-sh.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-db-1.5.1-cdh5.16.1.jar lrwxrwxrwx 1 root root 91 Jul 6 11:00 sentry-provider-file.jar -> /data/impala/apache-sentry-1.5.1-cdh5.16.1-bin/lib/sentry-provider-file-1.5.1-cdh5.16.1.jar
注意:使用sentry
和Impala
的CDH
版本必定要對應上,好比我這裏使用的是Impala
版本是CDH5.16.1
,那麼sentry
的也須要是,不然會由於jar
版本問題致使Impala
啓動過程當中拋出一些異常,好比:
java.lang.NoClassDefFoundError: org/apache/sentry/provider/cache/SentryPrivilegeCache
若是不知道Impala
依賴的一些外部組件的版本,能夠在Impala
源碼的Impala/bin/impala-config.sh
裏找到,如Impala cdh5-2.12.0_5.16.1版本的依賴信息在該配置文件裏定義以下:
# Versions of Hadoop ecosystem dependencies. # ------------------------------------------ export CDH_MAJOR_VERSION=5 export IMPALA_HADOOP_VERSION=2.6.0-cdh5.16.1 unset IMPALA_HADOOP_URL export IMPALA_HBASE_VERSION=1.2.0-cdh5.16.1 unset IMPALA_HBASE_URL export IMPALA_HIVE_VERSION=1.1.0-cdh5.16.1 unset IMPALA_HIVE_URL export IMPALA_SENTRY_VERSION=1.5.1-cdh5.16.1 unset IMPALA_SENTRY_URL export IMPALA_PARQUET_VERSION=1.5.0-cdh5.16.1 export IMPALA_LLAMA_MINIKDC_VERSION=1.0.0 unset IMPALA_LLAMA_MINIKDC_URL export IMPALA_KITE_VERSION=1.0.0-cdh5.16.1
將apache-sentry-1.5.1-cdh5.16.1-bin/conf
目錄下的sentry-site.xml.service.template
文件拷貝到/etc/impala/conf
目錄下:
# 拷貝 cp apache-sentry-1.5.1-cdh5.16.1-bin/conf/sentry-site.xml.service.template /etc/impala/conf/ # 重命名 cd /etc/impala/conf/ mv sentry-site.xml.service.template sentry-site.xml
編輯sentry-site.xml
爲如下內容:
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. --> <!-- WARNING!!! This file is provided for documentation purposes ONLY! --> <!-- WARNING!!! You should copy to sentry-site.xml and make modification instead. --> <configuration> <!--Sentry Server端口--> <property> <name>sentry.service.client.server.rpc-port</name> <value>8038</value> </property> <!--Sentry Server服務器地址--> <property> <name>sentry.service.client.server.rpc-addresses</name> <value>hadoop21-test1-rgtj5-tj1</value> </property> <!--客戶端鏈接Sentry Server超時時間,以毫秒爲單位,默認爲200000毫秒--> <property> <name>sentry.service.client.server.rpc-connection-timeout</name> <value>200000</value> </property> <!--權限存儲方式:數據庫或者ini配置文件--> <property> <name>sentry.hive.provider.backend</name> <value>org.apache.sentry.provider.db.SimpleDBProviderBackend</value> </property> <!--權限認證方式,支持Kerberos認證,設置爲none表示不啓用認證 --> <property> <name>sentry.service.security.mode</name> <value>none</value> </property> </configuration>
編輯/etc/default/impala
配置文件,修改以下兩個配置啓用Sentry
權限認證:
IMPALA_CATALOG_ARGS
選項,增長-sentry_config=/etc/impala/conf/sentry-site.xml
配置IMPALA_SERVER_ARGS
選項,增長-sentry_config=/etc/impala/conf/sentry-site.xml
和-server_name=sentryserver
配置配置文件最終內容以下:
IMPALA_CATALOG_SERVICE_HOST=hadoop21-test1-rgtj5-tj1 IMPALA_STATE_STORE_HOST=hadoop21-test1-rgtj5-tj1 IMPALA_STATE_STORE_PORT=24000 IMPALA_BACKEND_PORT=22000 IMPALA_LOG_DIR=/data/log/impala IMPALA_CATALOG_ARGS=" -log_dir=${IMPALA_LOG_DIR} -sentry_config=/etc/impala/conf/sentry-site.xml" IMPALA_STATE_STORE_ARGS=" -log_dir=${IMPALA_LOG_DIR} -state_store_port=${IMPALA_STATE_STORE_PORT}" IMPALA_SERVER_ARGS=" \ -log_dir=${IMPALA_LOG_DIR} \ -catalog_service_host=${IMPALA_CATALOG_SERVICE_HOST} \ -state_store_port=${IMPALA_STATE_STORE_PORT} \ -use_statestore \ -state_store_host=${IMPALA_STATE_STORE_HOST} \ -be_port=${IMPALA_BACKEND_PORT} \ -kudu_master_hosts=hadoop21-test1-rgtj5-tj1:7051,hadoop20-test1-rgtj5-tj1:7051,hadoop22-test1-rgtj5-tj1:7051,hadoop-bi06-test1-rgtj5-tj1:7051,hadoop-bi07-test1-rgtj5-tj1:7051 \ -sentry_config=/etc/impala/conf/sentry-site.xml \ -server_name=sentryserver" ENABLE_CORE_DUMPS=true # LIBHDFS_OPTS=-Djava.library.path=/usr/lib/impala/lib # MYSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java.jar # IMPALA_BIN=/usr/lib/impala/sbin # IMPALA_HOME=/usr/lib/impala # HIVE_HOME=/usr/lib/hive # HBASE_HOME=/usr/lib/hbase # IMPALA_CONF_DIR=/etc/impala/conf # HADOOP_CONF_DIR=/etc/impala/conf # HIVE_CONF_DIR=/etc/impala/conf # HBASE_CONF_DIR=/etc/impala/conf
重啓Impala
服務:
sudo service impala-state-store restart sudo service impala-catalog restart sudo service impala-server restart
打開impala-shell
,驗證權限配置是否成功,具體操做以下:
(1)切換到hadoop
用戶,打開impala-shell
,建立一個admin
角色:
[hadoop21-test1-rgtj5-tj1:21000] > create role admin_role; Query: create role admin_role Fetched 0 row(s) in 0.35s
(2)爲admin
角色賦予超級權限:
[hadoop21-test1-rgtj5-tj1:21000] > GRANT ALL ON SERVER sentryserver TO ROLE admin_role; Query: GRANT ALL ON SERVER sentryserver TO ROLE admin_role Query submitted at: 2019-07-06 10:40:11 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000) Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=15475b39691bd167:66c1403300000000 Fetched 0 row(s) in 0.13s
(3)將admin
角色受權給hadoop
用戶組:
[hadoop21-test1-rgtj5-tj1:21000] > GRANT ROLE admin_role TO GROUP hadoop; Query: GRANT ROLE admin_role TO GROUP hadoop Query submitted at: 2019-07-06 10:41:53 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000) Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=434bb908587eaf31:65887a5a00000000 Fetched 0 row(s) in 0.11s
(4)建立一個test
庫和test
表,並插入測試數據:
[hadoop21-test1-rgtj5-tj1:21000] > create database test; Query: create database test Fetched 0 row(s) in 0.29s [hadoop21-test1-rgtj5-tj1:21000] > use test; Query: use test [hadoop21-test1-rgtj5-tj1:21000] > CREATE TABLE test(x INT, y STRING) STORED AS PARQUET; Query: CREATE TABLE test(x INT, y STRING) STORED AS PARQUET Fetched 0 row(s) in 0.16s [hadoop21-test1-rgtj5-tj1:21000] > INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three'); Query: INSERT INTO test VALUES (1, 'one'), (2, 'two'), (3, 'three') Query submitted at: 2019-07-06 11:18:33 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000) Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=ce4e7f66f1209531:641f39a900000000 Modified 3 row(s) in 5.47s
由於hadoop
用戶是超級管理員並擁有ALL
的權限,所以執行如下SELECT
語句便能很快看到咱們剛插入的數據:
[hadoop21-test1-rgtj5-tj1:21000] > select * from test; Query: select * from test Query submitted at: 2019-07-06 11:19:50 (Coordinator: http://hadoop21-test1-rgtj5-tj1:25000) Query progress can be monitored at: http://hadoop21-test1-rgtj5-tj1:25000/query_plan?query_id=34e4b5594e3d0c6:8cfb1acb00000000 +---+-------+ | x | y | +---+-------+ | 1 | one | | 2 | two | | 3 | three | +---+-------+ Fetched 3 row(s) in 1.87s
接着咱們切換到root
用戶,運行impala-shell
,對咱們剛剛建立的test
庫進行操做:
[hadoop21-test1-rgtj5-tj1:21000] > use test; Query: use test ERROR: AuthorizationException: User 'root' does not have privileges to access: test.*.*
提示root
用戶沒有操做test
庫的權限,至此,說明Sentry
權限認證已經生效。
各類受權操做語法以下:
建立角色:CREATE ROLE <role name> 組分配角色:GRANT ROLE <role name> TO GROUP <group name> 服務級賦權:GRANT <ALL|SELECT|UPDATE> ON SERVER <server name> TO ROLE <role name> 數據庫賦權:GRANT <ALL|SELECT|UPDATE> ON DATABASE <database name> TO ROLE <role name> 表賦權:GRANT <ALL|SELECT|UPDATE> ON TABLE <database name>.<table name> TO ROLE <role name> 字段權限:GRANT SELECT(column name)ON TABLE <table name> TO ROLE <role name>; 回收組權限:REVOKE ROLE <role name> FROM GROUP <group name> 回收字段權限:REVOKE SELECT <column name> ON TABLE <table name> FROM ROLE <role name>; 回收數據庫權限:REVOKE <ALL|SELECT|UPDATE> ON DATABASE <database name> FROM ROLE <role name> 查看某個角色的權限:SHOW GRANT ROLE <role name> 各類查看命令: SHOW ROLES; SHOW CURRENT ROLES; SHOW ROLE GRANT GROUP <group name>; SHOW GRANT ROLE <role name>; SHOW GRANT ROLE <role name> on OBJECT <object name>;
一、Impala
服務的權限安全,認證(Kerberos/LDAP
)是第一步,受權(Sentry
)是第二步。若是要啓用受權,必須先啓用認證。本文在測試過程當中不啓用認證而只啓用Sentry
受權,強烈不建議在生產系統中這樣使用,由於若是沒有用戶認證,受權沒有任何意義形同虛設,用戶能夠隨意使用任何超級用戶登陸Impala
,並不會作密碼校驗。
二、Impala
是不區分底層存儲用戶的,Sentry
控制的只是Impala
應用層的操做權限,底層操做HDFS
的仍是impala
用戶,也就是啓動impalad
的用戶。不區分底層存儲用戶主要是由於C++
的libhdfs
在Hadoop2
時還不支持doAs
。
三、Impala
中的受權處理過程相似於Hive
中的受權處理過程,主要的區別在於權限信息的緩存。Impala
的Catalog
服務管理並緩存數據庫schema
元數據和Sentry
權限元數據,並將其傳播到全部Impala Server
節點。所以,Impala
中的受權驗證在本地進行,並且速度更快。能夠用下圖進行歸納:
官方資料:
博客文章:
Hadoop實操公衆號: