基於Feign初探Ranger Api

Ranger Api之User管理

在大數據平臺之權限管理組件 - Aapche Ranger一文中咱們瞭解了Ranger以及安裝部署過程以及Admin可視化界面的使用。

除了能夠在可視化的Ranger Admin界面上進行權限、用戶等管理外，Ranger還支持經過REST API來完成這些操做。由於咱們若是要開發本身的大數據平臺，可能並不會使用Ranger Admin的可視化界面，而是但願在本身的大數據平臺界面去操做Ranger。有了Ranger Api的支持，咱們就能夠輕易實現這一點。

關於Ranger Api的官方文檔以下：

• https://ranger.apache.org/apidocs/index.html

本小節簡單演示下User Api的使用，User Api用於管理用戶，對用戶進行增刪改查。首先建立一個空Maven項目。因爲要經過http請求Api，因此須要用到http請求工具。這裏用到的是feign，完整的pom文件內容以下：

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>ranger-client</artifactId>
    <version>1.0-SNAPSHOT</version>

    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
        </dependency>
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <version>1.18.4</version>
        </dependency>
        <dependency>
            <groupId>com.netflix.feign</groupId>
            <artifactId>feign-core</artifactId>
            <version>8.18.0</version>
        </dependency>
        <dependency>
            <groupId>com.netflix.feign</groupId>
            <artifactId>feign-jackson</artifactId>
            <version>8.18.0</version>
        </dependency>
        <dependency>
            <groupId>com.netflix.feign</groupId>
            <artifactId>feign-okhttp</artifactId>
            <version>8.18.0</version>
        </dependency>
        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
            <version>2.9.8</version>
        </dependency>
        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
            <version>21.0</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.25</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <version>3.8.0</version>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

首先定義一個配置類，配置用於訪問ranger api的用戶名密碼：

package com.example.ranger.config;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class RangerAuthConfig {
    private String username = "admin";
    private String password = "admin";
}

再定義一個配置類，用於提供http客戶端配置：

package com.example.ranger.config;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import feign.Logger;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class RangerClientConfig {
    private int connectionTimeoutMills = 5 * 1000;
    private int readTimeoutMills = 30 * 1000;
    private Logger.Level level = Logger.Level.BASIC;
    private String url = "http://192.168.243.161:6080";
    private RangerAuthConfig authConfig = new RangerAuthConfig();
}

聲明一個請求攔截器，用於在發起請求以前添加一些請求頭：

package com.example.ranger.interceptor;

import feign.RequestInterceptor;
import feign.RequestTemplate;

public class RangerHeadersInterceptor implements RequestInterceptor {

    @Override
    public void apply(RequestTemplate template) {
        template.header("Accept", "application/json");
        template.header("X-XSRF_HEADER", "\"\"");
        template.header("Content-Type", "application/json");
    }
}

一般在實際的開發中，咱們會定義一個業務異常，用於對異常信息進行自定義封裝：

package com.example.ranger.exception;

import java.io.Serializable;

public class RangerClientException extends RuntimeException implements Serializable {
    private static final long serialVersionUID = -4441189815976639860L;
    private Throwable cause;
    private int status;
    private String message;

    public RangerClientException(int status, String message) {
        this.status = status;
        this.message = message;
    }

    @Override
    public String getMessage() {
        return String.format("%s http status = %s", message, status);
    }

    @Override
    public String toString() {
        return String.format("%s http status = %s", message, status);
    }
}

自定義一個異常信息解析器，當請求發生異常時，feign會調用該方法返回咱們自定義的異常類：

package com.example.ranger.decoder;

import com.example.ranger.exception.RangerClientException;
import feign.Response;
import feign.Util;
import feign.codec.ErrorDecoder;

import java.io.IOException;

public class RangerErrorDecoder implements ErrorDecoder {

    @Override
    public Exception decode(String methodKey, Response response) {
        return new RangerClientException(
                response.status(), errorMessage(methodKey, response)
        );
    }

    private String errorMessage(String methodKey, Response response) {
        String msg = String.format("status %s reading %s", response.status(), methodKey);
        if (response.body() != null) {
            try {
                msg += "content:\n" + Util.toString(response.body().asReader());
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

        return msg;
    }
}

完成上面feign相關的前置準備後，咱們就能夠開始編寫請求ranger api的代碼了。首先，定義用戶接口的請求和響應實體：

package com.example.ranger.model;

import com.fasterxml.jackson.annotation.JsonIgnoreProperties;
import lombok.AllArgsConstructor;
import lombok.Builder;
import lombok.Data;
import lombok.NoArgsConstructor;

import java.util.List;

/**
 * 用戶信息實體類
 * https://ranger.apache.org/apidocs/json_VXUser.html
 *
 * @author 01
 * @date 2020-11-12
 **/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class User {
    private int id;
    private String name;
    private String createDate;
    private String updateDate;
    private String owner;
    private String updateBy;
    private String firstName;
    private String lastName;
    private String emailAddress;
    private String password;
    private String description;
    private int status;
    private int isVisible;
    private int userSource;
    private List<String> userRoleList;
}

定義用戶api相關的接口，這是Feign這種聲明式http客戶端的作法：

package com.example.ranger.api;

import com.example.ranger.model.User;
import feign.Param;
import feign.RequestLine;

/**
 * 用戶相關api
 * https://ranger.apache.org/apidocs/resource_XUserREST.html
 *
 * @author 01
 * @date 2020-11-12
 **/
public interface UserFeignClient {

    /**
     * 建立用戶接口
     * https://ranger.apache.org/apidocs/resource_XUserREST.html#resource_XUserREST_secureCreateXUser_POST
     *
     * @param user user
     * @return 用戶信息
     */
    @RequestLine("POST /service/xusers/secure/users")
    User createUser(User user);

    /**
     * 刪除用戶
     * https://ranger.apache.org/apidocs/resource_XUserREST.html#resource_XUserREST_deleteSingleUserByUserId_DELETE
     *
     * @param id          用戶id
     * @param forceDelete 是否強制刪除
     */
    @RequestLine("DELETE /service/xusers/secure/users/id/{id}?forceDelete={forceDelete}")
    void deleteUser(@Param("id") Integer id,
                    @Param("forceDelete") boolean forceDelete);

    /**
     * 獲取用戶信息
     * https://ranger.apache.org/apidocs/resource_XUserREST.html#resource_XUserREST_getXUserByUserName_GET
     *
     * @param name 用戶名稱
     * @return 用戶信息
     */
    @RequestLine("GET /service/xusers/users/userName/{name} ")
    User getUserByName(@Param("name") String name);
}

而後咱們在此以外再包一層，一般咱們會在這一層作一些額外的處理，例如參數校驗、結果校驗之類的：

package com.example.ranger.api;

import com.example.ranger.exception.RangerClientException;
import com.example.ranger.model.User;
import lombok.AllArgsConstructor;

@AllArgsConstructor
public class UserApi {

    private final UserFeignClient userClient;

    public User createUser(User user) throws RangerClientException {
        return userClient.createUser(user);
    }

    public void deleteUser(Integer id, boolean forceDelete) {
        userClient.deleteUser(id, forceDelete);
    }

    public User getUserByName(String name) throws RangerClientException {
        return userClient.getUserByName(name);
    }
}

最後定義一個客戶端工具類，提供一個統一的操做入口，以便於外部使用：

package com.example.ranger;

import com.example.ranger.api.PolicyApi;
import com.example.ranger.api.PolicyFeignClient;
import com.example.ranger.api.UserApi;
import com.example.ranger.api.UserFeignClient;
import com.example.ranger.config.RangerClientConfig;
import com.example.ranger.decoder.RangerErrorDecoder;
import com.example.ranger.interceptor.RangerHeadersInterceptor;
import com.fasterxml.jackson.annotation.JsonInclude;
import com.fasterxml.jackson.databind.DeserializationFeature;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import feign.Feign;
import feign.Logger;
import feign.Request;
import feign.auth.BasicAuthRequestInterceptor;
import feign.jackson.JacksonDecoder;
import feign.jackson.JacksonEncoder;
import feign.okhttp.OkHttpClient;
import lombok.Getter;
import lombok.extern.slf4j.Slf4j;

import java.util.concurrent.atomic.AtomicBoolean;

@Slf4j
public class RangerClient {

    @Getter
    private UserApi userApi;

    @Getter
    private PolicyApi policyApi;

    private final RangerClientConfig rangerClientConfig;

    public RangerClient(RangerClientConfig rangerClientConfig) {
        this.rangerClientConfig = rangerClientConfig;
    }

    private static final ObjectMapper MAPPER = new ObjectMapper()
            .setSerializationInclusion(JsonInclude.Include.NON_NULL)
            .configure(SerializationFeature.INDENT_OUTPUT, true)
            .configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
    private static final JacksonEncoder ENCODER = new JacksonEncoder(MAPPER);
    private static final JacksonDecoder DECODER = new JacksonDecoder(MAPPER);

    /**
     * 標識client是否已啓動
     */
    private final AtomicBoolean started = new AtomicBoolean(false);

    /**
     * 配置client的構建信息
     *
     * @return {@link Feign.Builder}
     */
    private Feign.Builder feignBuilder() {
        return Feign.builder()
                .logger(new Logger.JavaLogger())
                .logLevel(rangerClientConfig.getLevel())
                .options(new Request.Options(
                        rangerClientConfig.getConnectionTimeoutMills(),
                        rangerClientConfig.getReadTimeoutMills()
                )).encoder(ENCODER).decoder(DECODER)
                .client(new OkHttpClient())
                .errorDecoder(new RangerErrorDecoder())
                .requestInterceptor(new RangerHeadersInterceptor())
                .requestInterceptor(new BasicAuthRequestInterceptor(
                        rangerClientConfig.getAuthConfig().getUsername(),
                        rangerClientConfig.getAuthConfig().getPassword()
                ));
    }

    /**
     * 啓動client
     */
    public void start() {
        if (started.get()) {
            log.info("ranger client is already started");
            return;
        }

        userApi = new UserApi(feignBuilder().target(
                UserFeignClient.class, rangerClientConfig.getUrl()
        ));
        policyApi = new PolicyApi(feignBuilder().target(
                PolicyFeignClient.class, rangerClientConfig.getUrl()
        ));
        started.set(true);
    }

    /**
     * 中止client
     */
    public void stop() {
        if (started.get()) {
            started.set(false);
        } else {
            log.info("ranger client is not started");
        }
    }
}

完成以上的功能代碼編寫後，咱們來寫一些單元測試，去驗證一下功能是否都正確實現了：

package com.example.ranger.api;

import com.example.ranger.RangerClient;
import com.example.ranger.config.RangerClientConfig;
import com.example.ranger.model.User;
import org.junit.Before;
import org.junit.Test;

import java.util.Collections;

import static org.junit.Assert.assertNotNull;

public class UserApiTest {

    private static RangerClient rangerClient;

    @Before
    public void initRangerClient() {
        rangerClient = new RangerClient(new RangerClientConfig());
        rangerClient.start();
    }

    @Test
    public void testCreateUser() {
        User user = User.builder().name("test")
                .firstName("first").lastName("last").password("user@123")
                .isVisible(1).status(1).userSource(0)
                .userRoleList(Collections.singletonList("ROLE_USER"))
                .build();

        User result = rangerClient.getUserApi().createUser(user);
        assertNotNull(result);
        System.out.println(result);
    }

    @Test
    public void testDeleteUser() {
        User result = rangerClient.getUserApi().getUserByName("test");
        assertNotNull(result);
        rangerClient.getUserApi().deleteUser(result.getId(), true);
    }

    @Test
    public void testGetUserByName() {
        User result = rangerClient.getUserApi().getUserByName("test");
        assertNotNull(result);
        System.out.println(result);
    }
}

運行testCreateUser這個單元測試，而後到ranger admin上查看是否有新增相應的用戶：

而後再運行testDeleteUser這個單元測試，看看該用戶是否能被正常刪除：

---

Ranger Api之Policy管理

本小節將介紹使用Policy Api對Ranger上的權限策略進行管理。首先定義接口的請求/響應實體類，因爲Policy稍微複雜點，須要定義的類也比較多：

/**
 * 策略所做用的資源，即hdfs目錄、hive的庫/表/列等
 * https://ranger.apache.org/apidocs/json_RangerPolicyResource.html
 *
 * @author 01
 * @date 2020-11-12
 **/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyResource {
    private List<String> values = Lists.newArrayList();
    private Boolean isExcludes;
    private Boolean isRecursive;
}

/**
 * https://ranger.apache.org/apidocs/json_RangerPolicyItemCondition.html
 *
 * @author 01
 * @date 2020-11-12
 **/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyItemCondition {
    private String type;
    private List<String> value = Lists.newArrayList();
}

/**
 * 策略條件項中的權限信息，即在該項中擁有哪些權限，對應「Permissions」
 * https://ranger.apache.org/apidocs/json_RangerPolicyItemAccess.html
 *
 * @author 01
 * @date 2020-11-12
 **/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyItemAccess {
    private String type;
    private Boolean isAllowed;
}

/**
 * 策略中的條件項，對應「Allow Conditions」或「Deny Conditions」中的每一欄信息
 * https://ranger.apache.org/apidocs/json_RangerPolicyItem.html
 *
 * @author 01
 * @date 2020-11-12
 **/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class PolicyItem {
    private List<PolicyItemAccess> accesses = Lists.newArrayList();
    private Set<String> users = Sets.newHashSet();
    private List<String> groups = Lists.newArrayList();
    private List<PolicyItemCondition> conditions = Lists.newArrayList();
    private Boolean delegateAdmin;
}

/**
 * 策略實體
 * https://ranger.apache.org/apidocs/json_RangerPolicy.html
 *
 * @author 01
 * @date 2020-11-12
 **/
@Data
@Builder
@NoArgsConstructor
@AllArgsConstructor
@JsonIgnoreProperties(ignoreUnknown = true)
public class Policy {
    private Map<String, PolicyResource> resources;
    private List<PolicyItem> policyItems = Lists.newArrayList();
    private List<PolicyItem> denyPolicyItems = Lists.newArrayList();
    private List<PolicyItem> allowExceptions = Lists.newArrayList();
    private List<PolicyItem> denyExceptions = Lists.newArrayList();
    private List<Object> dataMaskPolicyItems = Lists.newArrayList();
    private List<Object> rowFilterPolicyItems = Lists.newArrayList();

    private int id;
    private String guid;
    private boolean isEnabled;
    private int version;
    private String service;
    private String name;
    private int policyType;
    private String description;
    private boolean isAuditEnabled;
}

定義權限策略相關api的接口：

package com.example.ranger.api;

import com.example.ranger.model.Policy;
import feign.Param;
import feign.RequestLine;

import java.util.List;

/**
 * 權限策略相關api
 * https://ranger.apache.org/apidocs/resource_PublicAPIsv2.html
 * https://ranger.apache.org/apidocs/resource_ServiceREST.html
 *
 * @author 01
 * @date 2020-11-12
 **/
public interface PolicyFeignClient {

    /**
     * 建立策略
     * https://ranger.apache.org/apidocs/resource_PublicAPIsv2.html#resource_PublicAPIsv2_createPolicy_POST
     *
     * @param policy 策略信息
     * @return 策略信息
     */
    @RequestLine("POST /service/public/v2/api/policy")
    Policy createPolicy(Policy policy);

    /**
     * 刪除策略
     * https://ranger.apache.org/apidocs/resource_ServiceREST.html#resource_ServiceREST_deletePolicy_DELETE
     *
     * @param id 策略id
     */
    @RequestLine("DELETE /service/plugins/policies/{id}")
    void deletePolicy(@Param("id") Integer id);

    /**
     * 經過服務和策略名稱獲取策略信息
     * https://ranger.apache.org/apidocs/resource_PublicAPIsv2.html#resource_PublicAPIsv2_getPolicyByName_GET
     *
     * @param serviceName 服務名稱
     * @param policyName  策略名稱
     * @return 策略信息
     */
    @RequestLine("GET /service/public/v2/api/service/{serviceName}/policy/{policyName}")
    Policy getPolicyByName(@Param("serviceName") String serviceName,
                           @Param("policyName") String policyName);

    /**
     * 獲取指定服務下的策略信息列表
     *
     * @param serviceName 服務名稱
     * @return 該服務下的策略信息列表
     */
    @RequestLine("GET /service/public/v2/api/service/{serviceName}/policy")
    List<Policy> getAllPoliciesByService(@Param("serviceName") String serviceName);
}

一樣，在接口之上再包一層：

package com.example.ranger.api;

import com.example.ranger.model.Policy;
import lombok.AllArgsConstructor;

import java.util.List;

@AllArgsConstructor
public class PolicyApi {

    private final PolicyFeignClient policyFeignClient;

    public Policy getPolicyByName(String serviceName, String policyName)  {
        return policyFeignClient.getPolicyByName(serviceName, policyName);
    }

    public List<Policy> getAllPoliciesByService(String serviceName)  {
        return policyFeignClient.getAllPoliciesByService(serviceName);
    }

    public Policy createPolicy(Policy policy)  {
        return policyFeignClient.createPolicy(policy);
    }

    public void deletePolicy(Integer id) {
        policyFeignClient.deletePolicy(id);
    }
}

修改RangerClient，增長PolicyApi相關代碼：

@Slf4j
public class RangerClient {

    @Getter
    private PolicyApi policyApi;

    ...

    /**
     * 啓動client
     */
    public void start() {
        if (started.get()) {
            log.info("ranger client is already started");
            return;
        }

        userApi = new UserApi(feignBuilder().target(
                UserFeignClient.class, rangerClientConfig.getUrl()
        ));
        policyApi = new PolicyApi(feignBuilder().target(
                PolicyFeignClient.class, rangerClientConfig.getUrl()
        ));
        started.set(true);
    }

    ...
}

編寫單元測試：

package com.example.ranger.api;

import com.example.ranger.RangerClient;
import com.example.ranger.config.RangerClientConfig;
import com.example.ranger.model.Policy;
import com.example.ranger.model.PolicyItem;
import com.example.ranger.model.PolicyItemAccess;
import com.example.ranger.model.PolicyResource;
import org.junit.Before;
import org.junit.Test;

import java.util.*;

import static org.junit.Assert.assertNotNull;

public class PolicyApiTest {

    private static RangerClient rangerClient;

    @Before
    public void initRangerClient() {
        rangerClient = new RangerClient(new RangerClientConfig());
        rangerClient.start();
    }

    @Test
    public void testCreatePolicy() {
        PolicyResource policyResource = PolicyResource.builder()
                .values(Collections.singletonList("/testdir2"))
                .isRecursive(true)
                .build();

        Map<String, PolicyResource> policyResourceMap = new HashMap<>();
        policyResourceMap.put("path", policyResource);

        Set<String> users = new HashSet<>();
        users.add("hive");

        List<PolicyItemAccess> policyItemAccessList = new ArrayList<>();
        policyItemAccessList.add(PolicyItemAccess.builder().type("read").build());
        policyItemAccessList.add(PolicyItemAccess.builder().type("write").build());
        policyItemAccessList.add(PolicyItemAccess.builder().type("execute").build());

        PolicyItem policyItem = PolicyItem.builder()
                .delegateAdmin(true).users(users)
                .accesses(policyItemAccessList)
                .build();

        Policy policy = Policy.builder()
                .service("dev_hdfs")
                .name("test_ranger_api")
                .isEnabled(true).policyType(0)
                .resources(policyResourceMap)
                .policyItems(Collections.singletonList(policyItem))
                .build();

        Policy result = rangerClient.getPolicyApi().createPolicy(policy);
        assertNotNull(result);
        System.out.println(result.getName());
    }

    @Test
    public void testGetPolicyByName() {
        Policy result = rangerClient.getPolicyApi()
                .getPolicyByName("dev_hdfs", "test_ranger_api");
        assertNotNull(result);
        System.out.println(result.getName());
    }

    @Test
    public void testGetAllPoliciesByService() {
        List<Policy> result = rangerClient.getPolicyApi()
                .getAllPoliciesByService("dev_hdfs");
        assertNotNull(result);
        System.out.println(result.size());
    }

    @Test
    public void testDeletePolicy() {
        Policy result = rangerClient.getPolicyApi()
                .getPolicyByName("dev_hdfs", "test_ranger_api");
        assertNotNull(result);
        rangerClient.getPolicyApi().deletePolicy(result.getId());
        System.out.println(result.getName());
    }
}

執行testCreatePolicy單元測試，到ranger admin上驗證是否建立了相應的策略：

查看策略內容是否與代碼中定義的一致：

---

本文的代碼倉庫：

• https://gitee.com/Zero-One/ranger-client-demo