Paeony：開源Etcdv3-cpp-api 庫入門介紹

時間  2020-12-07
標籤 c++ git github vim api 網絡 多線程 負載均衡 線程 欄目 C&C++ 简体版
原文   原文鏈接
日期 做者 版本 備註
2020-12-7 dingbin v1.0

Paeony：Etcdv3-cpp-api 庫簡介

Paeony是做者開源的用C++語言實現成熟的etcd v3版本客戶端API庫。Github地址是：https://github.com/apollo008/paeony 。它達到穩定可靠的企業級應用效果。它在原始單純etcdv3-cpp-api基礎上，封裝了不少面向負載均衡和鏈接重試的特性。它具備如下優點：c++

  1. 重試機制: 當etcd服務端不可鏈接或網絡出現問題後，Paeony客戶端會自動作出合理時間間隔的屢次鏈接重試，而且兼顧從新鏈接的負載均衡，直到再次鏈接成功爲止，保證系統不由於一個etcd server鏈接不可用而宕機；鏈接重試過程同步輸出詳細日誌，幫助用戶迅速跟蹤定位Etcd服務鏈接情況異常緣由，以快速解決etcd服務鏈接問題。整個重連過程用戶無感知。
  2. 鏈接狀態監控: Paeony初始化以後會一直對etcd鏈接進行監聽，一旦發現鏈接狀態發生變化將會做出相應的處理。
  3. Etcd客戶端鏈接實例管理:Paeony會對etcdv3客戶端到server集羣的鏈接進行管理，並在須要的時候重建etcd鏈接實例，保證與etcd集羣鏈接的可靠性。
  4. 採用做者開源成熟穩定的日誌系統 tulip-log , Github地址：https://github.com/apollo008/tulip-log ,同步輸出豐富的etcdv3客戶端操做狀態和鏈接狀態日誌信息，便於排查線上問題。
  5. 對etcdv3 客戶端的watch和leaseKeepAlive 過程作了尤爲加強支持，使得該etcdv3 客戶端不會由於某個etcd server不可用或網絡抖動致使watch失效或關鍵心跳節點丟失等嚴重問題。頑強穩定地支持適用各類網絡環境和etcd servers 鏈接異常情況的watch 和leaseKeepAlive表現。
  6. 客戶端所有接口都支持多線程同步訪問，保證多線程操做訪問下結果也能正確可靠。
  7. 支持Etcdv3版本事務Transaction操做。
  8. 結合demo示例：src/paeony/paeony/core/examples/hello_paeony.cpp, 接口設計便捷易用。

基於Paeony的Etcd v3 c++ api基本用法

項目的src/paeony/paeony/core/examples目錄給出了經常使用的etcdv3 client使用示例。git

具體來講，demo使用入口見 src/paeony/paeony/core/examples/hello_paeony.cpp代碼示例：github

#include "paeony/core/examples/paeony_demo.h"

PAEONY_USE_NAMESPACE;
STD_USE_NAMESPACE;

int main(int argc, char** argv) {
    TLOG_CONFIG("logger.conf");
    TLOG_DECLARE_AND_SETUP_LOGGER(HELLOPAEONY, MAIN);

    EtcdDemoConnInfo info;
    PaeonyDemo demo(info);

    demo.DemoEtcdv3ClientGetPut();
    demo.DemoEtcdv3Lease();
    demo.DemoEtcdv3Watch();
    demo.DemoEtcdv3Transaction();

    TLOG_LOG_FLUSH();
    TLOG_LOG_SHUTDOWN();

    return 0;
}

具體的4種主要的Etcdv3 client 接口Demo參考文件：src/paeony/paeony/core/examples/paeony_demo.cppvim

  • Etcdv3Client/Get/Put

/**
 *@brief     Method to demo etcdv3 client and Get/Put method
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:14
 */
void PaeonyDemo::DemoEtcdv3ClientGetPut() {
    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);

    try {
        while (true) {
            paeony::EtcdRangeResponse getResponse = client->Get(m_info.m_etctTestKey, true, false,
                                                               false, 0, 3,
                                                               2, 50);
            if (getResponse.IsOk()) {
                TLOG_LOG(INFO,"Successfully get, value=[%s].",
                (getResponse.m_rpcResp.kvs_size()>0?getResponse.m_rpcResp.kvs(0).value().c_str(): "") );
            }
            else {
                TLOG_LOG(ERROR,"Failed to get,error_code is:[%d],operation failed,detail message:[%s]",
                                      getResponse.GetErrorCode(), getResponse.GetErrorMsg().c_str() );

            }
            EtcdPutResponse putResponse = client->Put(m_info.m_etctTestKey + "/123","hello,paeony!!!",
                                                                                  0, false, 5, 2, 50);
            if (putResponse.IsOk()) {
                TLOG_LOG(INFO,"Successfully put key:[%s]", m_info.m_etctTestKey.c_str());
            }
            else {
                TLOG_LOG(ERROR,"Failed to put, error_code is:[%d],operation failed,detail message:[%s]",
                                       putResponse.GetErrorCode(),putResponse.GetErrorMsg().c_str() );
            }
            sleep(3);
        }
    }
    catch (std::exception const & ex)
    {
        TLOG_LOG(ERROR,"Communication problem,details:[%s]", ex.what());
    }
}

  • Watch

/**
 *@brief     Method used to demo Etcdv3 Watch interface
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:15
 */
void PaeonyDemo::DemoEtcdv3Watch() {

    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);

    string key = "/paeony/TestWatch";
    client ->Put(key,"0");

    function<void(EtcdWatchResponse)> watchCallback = [](EtcdWatchResponse response) {
        PaeonyDemo::_logger->Log(tulip::TLOG_LEVELNO_ERROR,__FILE__,__LINE__,__FUNCTION__,
        "got watch response, with response's events size:[%d] and if for the firstResponseCallback:[%s]",
        response.m_rpcResp.events_size(), (response.m_bFirstResponse ? "true":"false"));
    };

    vector<WatchCreateRequest::FilterType>  filtersVec;
    EtcdWatchWorkerPtr watchWorker = make_shared<EtcdWatchWorker>(client,key,true,true,true,filtersVec,
                                                                 watchCallback,100,30,4 * 1024 * 1024);

    watchWorker->Start();
    while(!watchWorker->IsStarted()){
        TLOG_LOG(INFO, "wait EtcdWatchWorker start...");
        usleep(10);
    }

    TLOG_LOG(INFO,"EtcdWatchWorker started......");
    sleep(180);
    TLOG_LOG(INFO, "=======================now terminate EtcdWatchWorker...");
    watchWorker->Terminate();
    TLOG_LOG(INFO,"EtcdWatchWorker terminate, wait done");
    watchWorker->Join();
    TLOG_LOG(INFO,"EtcdWatchWorker done.");
    int remainTimeSecond = 10;
    int t = 0;
    while (remainTimeSecond - t * 2 > 0) {
        TLOG_LOG(INFO,"Wait 10 seconds to exit Function:[%s], remain [%d] second..",
        __FUNCTION__ , remainTimeSecond - t * 2);
        ++t;
        sleep(2);
    }
    TLOG_LOG(INFO,"now exit function:[%s]", __FUNCTION__ );
}

  • Lease

/**
 *@brief     Method used to demo Etcdv3 Lease interface
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:15
 */
void PaeonyDemo::DemoEtcdv3Lease() {
    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);
    int64_t ttl = 60;
    EtcdLeaseGrantResponse lgr;
    do {
        lgr = client ->LeaseGrant(ttl,0,3,2,50);
        if (lgr.IsOk()) {
            break;
        }
        else {
        TLOG_LOG(ERROR,"Lease grant errCode:[%d],errMsg:[%s]",
                       lgr.GetErrorCode(),lgr.GetErrorMsg().c_str());
        }
        sleep(1);
    } while(true);

    client ->Put("/paeony/TestLeaseKeepAlive","0",lgr.m_rpcResp.id());

    function<void(LeaseGrantResponse)> reLeaseGrantCallback = [client](LeaseGrantResponse response) {
        time_t tm = time(NULL);
        ostringstream oss;
        oss <<tm;
        client ->Put("/paeony/TestLeaseKeepAlive",oss.str(),response.id());
        PaeonyDemo::_logger->Log(tulip::TLOG_LEVELNO_INFO,__FILE__,__LINE__,__FUNCTION__,
                    "got reLeaseGrant response, with id:[%ld] and ttl:[%ld]", response.id(),response.ttl());
    };

    EtcdLeaseKeepAliveWorkerPtr leaseKeepAliveWorker = make_shared<EtcdLeaseKeepAliveWorker> (client,
               lgr.m_rpcResp.id(),lgr.m_rpcResp.ttl(),reLeaseGrantCallback,100,ttl + 2,2 * 1024 * 1024);

    leaseKeepAliveWorker->Start();
    while(!leaseKeepAliveWorker->IsStarted()){
        TLOG_LOG(INFO,"wait LeaseKeepAliveWorker start");
        usleep(10);
    }
    TLOG_LOG(INFO,"LeaseKeepAliveWorker started");
    sleep(60 * 60);

    TLOG_LOG(INFO,"=======================now terminate LeaseKeepAliveWorker...");
    leaseKeepAliveWorker->Terminate();
    TLOG_LOG(INFO,"LeaseKeepAliveWorker terminate, wait done");
    leaseKeepAliveWorker->Join();
    TLOG_LOG(INFO,"LeaseKeepAliveWorker done.");

    int remainTimeSecond = 3 * 60;
    int t = 0;
    while (remainTimeSecond - t * 5 > 0) {
        TLOG_LOG(INFO,"Wait 3 minute to exit Function:[%s], remain [%d] second...",
                                             __FUNCTION__,remainTimeSecond - t * 5);
        ++t;
        sleep(5);
    }
    TLOG_LOG(INFO,"now exit function:[%s]",__FUNCTION__);
}

  • Transaction

/**
 *@brief     Method used to demo Etcdv3 transaction interface
 *@author    dingbinthu@163.com
 *@date      2020/12/7, 上午1:15
 */
void PaeonyDemo::DemoEtcdv3Transaction() {
    string key = "/paeony/TestTxn";

    Compare compare;
    compare.set_result(etcdserverpb::Compare::CompareResult::Compare_CompareResult_EQUAL);
    compare.set_target(etcdserverpb::Compare::CompareTarget::Compare_CompareTarget_VERSION);
    compare.set_key(key);
    compare.set_version(0);

    TransactionRequestPtrVec successTxnRequestsVec;
    TransactionRequestPtr put1 = 
                    make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key,"1"));
    successTxnRequestsVec.push_back(put1);
    TransactionRequestPtr range1 = 
                    make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key));
    successTxnRequestsVec.push_back(range1);
    TransactionRequestPtr put2 = 
                   make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key + "1","2"));
    successTxnRequestsVec.push_back(put2);
    TransactionRequestPtr range2 = 
                   make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key + "1"));
    successTxnRequestsVec.push_back(range2);


    TransactionRequestPtrVec  failureTxnRequestsVec;
    TransactionRequestPtr put1_ = 
                       make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key,"1_"));
    failureTxnRequestsVec.push_back(put1_);
    TransactionRequestPtr range1_ = 
                       make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key));
    failureTxnRequestsVec.push_back(range1_);
    TransactionRequestPtr put2_ = 
                      make_shared<TransactionPutRequest>(Etcdv3Client::s_ConstructPutRequest(key+"2","2_"));
    failureTxnRequestsVec.push_back(put2_);
    TransactionRequestPtr range2_ = 
                      make_shared<TransactionRangeRequest>(Etcdv3Client::s_ConstructRangeRequest(key +"2"));
    failureTxnRequestsVec.push_back(range2_);

    Etcdv3ClientPtr client = std::make_shared<Etcdv3Client>(m_info.m_certificateContents,
                                                            m_info.m_user,
                                                            m_info.m_passwd,
                                                            m_info.m_allEndpoints,
                                                            time(NULL),
                                                            m_info.m_etctTestKey);
    EtcdTxnResponse txnResponse = 
                         client->Transaction(compare,successTxnRequestsVec,failureTxnRequestsVec,8,3,50);

    if (txnResponse.IsOk()) {
        TLOG_LOG(INFO, "Succeed in executing transaction operation whose txnResponse's succeeded is:[%s]",
                                                    (txnResponse.m_rpcResp.succeeded() ? "true":"false"));
    }
    else {
        TLOG_LOG(ERROR,"Failed to execute transaction operation with errorCode:[%d] and errorMsg:[%s]",
                                         txnResponse.GetErrorCode(), txnResponse.GetErrorMsg().c_str() );
    }
}

Tulip-log的配置文件示例參考：src/paeony/paeony/core/examples/logger.confapi

編譯和安裝

目前支持類Unix環境下編譯安裝，Paeony項目依賴的第三方庫有：網絡

  • protobuf
  • grpc
  • openssl
  • tulip-log

編譯安裝paeony以前先要安裝以上4個依賴庫。多線程

具體編譯和安裝方法以下：負載均衡

git clone https://github.com/apollo008/paeony.git paeony.git
cd paeony.git
vim CMakeLists.txt 修改第8行：
原來是：set(PAEONY_DEPEND_PREFIX_DIR /path/to/install/share)
將/path/to/install/share 替換爲 安裝上面4個目標依賴庫的路徑，好比${HOME}/local,注意確保該路徑有寫權限。
修改以後後續不要再改變該路徑。

mkdir build-dir
cd build-dir
#安裝依賴
cmake -DENABLE_BUILD_SHARE=ON ../src
執行完以上這一步便可完成依賴庫。注意不須要再執行make 和make install了。

接下來安裝paeony
#安裝paeony
首先保持根目錄下CMakeLists.txt第8行修改的內容再也不改變；
cd build-dir
rm -rf *
cmake -DCMAKE_INSTALL_PREFIX=/path/to/install  ../src
make -j10
make install

執行完以上，便可在/path/to/install目錄下生成bin、lib、include3個目錄。其中lib目錄是libpaeony.so; bin目錄下是hello_paeony可執行程序； include目錄是paeony庫的頭文件。ui

注意：運行demo程序hello_paeony時，須要在當前目錄放置tulip-log的日誌配置文件logger.conf ，同時該目錄下要已經建立好logs 目錄供輸出文件名滾動的日誌文件。線程

其它

相關細節或其它未盡事宜，可聯繫 dingbinthu@163.com 探討諮詢。

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程