多是全網首個支持阿里雲Elasticsearch Xapck鑑權的Skywalking

多是全網首個支持阿里雲Elasticsearch Xapck鑑權的Skywalking

對Skywalking有興趣的同窗參見:年輕人的第一個APM-Skywalking

以前在搭建Skywalking的時候發現,官方Skywalking 5.X並支持有鑑權的Elasticsearch.

而我司有其餘需求已經購買了阿里雲的Elasticsearch,諮詢過阿里雲技術支持後他們表示並不能去掉鑑權,因此只好本身想辦法了.

又在Skywalking技術羣問了一圈,有其餘人也遇到過相似的問題,可是最後仍是選擇自建ES了.

實在不想本身再浪費精力去搭建ES了,仍是以爲能夠嘗試一下別的方案.

而後諮詢了一下wusheng大大以後,他說能夠本身嘗試換一個支持XPack鑑權的Client,應該沒什麼太大的問題.

因而就開始了"填坑"之旅.

首先是引入x-pack-transport支持

apm-collector/apm-collector-component/client-component/pom.xml

<dependency>
            <groupId>org.elasticsearch.client</groupId>
            <artifactId>x-pack-transport</artifactId>
            <version>${elasticsearch.client.version}</version>
        </dependency>

        <repositories>
        <repository>
            <id>elasticsearch-releases</id>
            <url>https://artifacts.elastic.co/maven</url>
            <releases>
                <enabled>true</enabled>
            </releases>
            <snapshots>
                <enabled>false</enabled>
            </snapshots>
        </repository>
    </repositories>

接着在 ...in/java/org/apache/skywalking/apm/collector/client/elasticsearch/ElasticSearchClient.java

加入PreBuiltXPackTransportClient的初始化

private final String securityUser;


 private PreBuiltXPackTransportClient initXPackClient() {
        Settings settings = Settings.builder()
                .put("cluster.name", clusterName)
                .put("xpack.security.transport.ssl.enabled", false)
                .put("xpack.security.user", securityUser)
                .put("client.transport.sniff", false).build();
        return new PreBuiltXPackTransportClient(settings);
     }
     private PreBuiltTransportClient initClient() {
        Settings settings = Settings.builder()            Settings settings = Settings.builder()
            .put("cluster.name", clusterName)                    .put("cluster.name", clusterName)
            .put("client.transport.sniff", clusterTransportSniffer)                    .put("client.transport.sniff", clusterTransportSniffer)
            .build();                    .build();
        return new PreBuiltTransportClient(settings);
    }

     // 新增 private final String securityUser;
    // 判斷這個變量是否是null或者空字符串,若是是就默認初始化,不是則使用initXPackClient初始化
    // 改一下initialize 方法

    private final String securityUser;


     @Override
    public void initialize() throws ClientException {
        if (securityUser == null || "".equals(securityUser)) {
            client = initClient();
        } else {
            client = initXPackClient();
        }

而後還要把apm-collector/pom.xml的elasticsearch.client.version 版本改爲5.3.3.

改完以後由於5.3.3和原來5.5.0有點不同,須要修改一下很幾個地方的代碼.

這時候建議直接使用IDEA build 一下,哪裏報錯改哪裏就好.

主要都是 searchResponse.getHits().totalHits 改爲searchResponse.getHits().totalHits()

神奇發現5.5.0版本的ES Client把5.3.3的searchResponse.getHits().totalHits() 方法改爲了屬性.

不經感慨都是人才啊...

別的一些基本都是引入 import org.elasticsearch.action.bulk.byscroll.BulkByScrollResponse;

所有代碼在這裏:liguobao/incubator-skywalking

完整改好的代碼在liguobao/incubator-skywalking

同時配置的時候添加一下 securityUser參數,若是ES有鑑權就傳入,沒有的話就不傳,這樣就達到鑑權和不鑑權兩種需求的兼容了.

支持xpack的docker部署方案

完整原文連接:Skywalking-Dcoker for ES xpack 鏡像

Dockerfile說明

apache-skywalking-apm-incubating.tar.gz爲支持ES X-Pack修改後打包出來的壓縮包,此倉庫沒有這個文件的.

能夠去QQ羣:Apache SkyWalking交流羣(392443393)羣文件中下載apache-skywalking-apm-incubating-xpack.tar.gz

或者自行編譯liguobao/incubator-skywalking/tree/5.x 此版本的源碼.

編譯步驟:

# Prepare git, JDK8 and maven3
git clone https://github.com/liguobao/incubator-skywalking.git
cd incubator-skywalking/
git checkout 5.x
#Switch to the tag by using git checkout [tagname] (Optional, switch if want to build a release from source codes)
git submodule init
git submodule update
Run ./mvnw clean package -DskipTests
#All packages are in /dist.(.tar.gz for Linux and .zip for Windows).

Docker 鏡像名稱:liguobao/skywalking-docker

拉取鏡像（Pull Image）:

docker pull liguobao/skywalking-docker:5.0.RC2.xpack

運行鏡像（Run）for ES xpack:

• docker run -p 8080:8080 -p 10800:10800 -p 11800:11800 -p 12800:12800 -e ES_CLUSTER_NAME=elasticsearch -e ES_ADDRESSES=192.168.2.96:9300 -e SECURITY_USER='elastic:password' -d liguobao/skywalking-docker:5.0.RC2.xpack
• 使用瀏覽器訪問http://localhost:8080便可.
• 日誌掛載 -v /your/log/path:/apache-skywalking-apm-incubating/logs

環境變量（Environment Variables）

• ES_CLUSTER_NAME,ES_ADDRESSES:elasticsearch 地址和集羣名稱。注意：此處Elasticsearch地址中的端口務必是Elasticsearch TCP端口。
• SECURITY_USER,SECURITY_USER:elasticsearch 的帳號密碼,使用X-Pack實現的,常見阿里雲ES,格式爲:'user:password'.此參數不傳入或者傳入'' ,默認使用沒有受權的client.
• NAMING_BIND_HOST,NAMING_BIND_PORT:OS real network IP(binding required),for agent to find collector cluster.
• BIND_HOST,REMOTE_BIND_PORT:OS real network IP(binding required),for collector nodes communicate with each other in cluster. collectorN --(gRPC) --> collectorM
• AGENT_GRPC_BIND_PORT:OS real network IP(binding required),for agent to uplink data(trace/metrics) to collector. agent--(gRPC)--> collector
• AGENT_JETTY_BIND_HOST,AGENT_JETTY_BIND_PORT:OS real network IP(binding required), for agent to uplink data(trace/metrics) to collector through HTTP. agent--(HTTP)--> collector

-UI_JETTY_BIND_HOST,UI_JETTY_BIND_PORT:Stay in 0.0.0.0 if UI starts up in default mode.Change it to OS real network IP(binding required), if deploy collector in different machine.

與elasticsearch-shanghai-zone鏡像配合使用請參考

• wutang/elasticsearch-shanghai-zone
• quick start

後記

原本還打算把代碼提給主倉庫的,可是wusheng 大大說xpack客戶端和Apache要求的受權有衝突,遂...

那就留着本身玩了.

拜...