RPC服務框架探索之Thrift

前言
架構服務化後，須要實現一套方便調用各服務的框架，如今開源如日中天，優先會尋找開源實現，若是沒有合適自家公司業務的，纔會考慮從零開發，尤爲是一切以KPI爲準繩的公司，誰會跟錢過不去？N個月以前，公司大神就開始調研了，最後選中了Thrift這個RPC服務框架。使用不熟悉的技術，我會感到很恐懼，它就至關於一個黑盒，我對它一無所知，它是如何運轉的？出了問題該如何解決？帶着一絲不安，查閱了相關技術文檔。

 

RPC
很早以前據說過soap,restful api,rpc之類的服務協議，一直都沒有機會深刻實踐，對它們理解的不夠深。它們的目的都是提供本地調用遠程服務的能力，只是實現方式不一樣而已。RPC(remote procedure call)意思是遠程過程調用，編碼時能夠把它看成本地方法同樣調用，無需關心內部的實現細節，對於調用方很友好很簡單。我查閱資料，發現RPC之類的東西很早很早之前就出現了，存在便是合理的，確定有它的理由。跟本地調用相比有什麼優勢缺點呢？根據查閱的資料以及本身的理解總結以下：
優點
1 提升系統吞吐能力
2 業務服務解耦
3 更易構建服務分佈式集羣
4 基礎服務重用更方便
劣勢
1 因網絡開銷方法執行時間更長
2 系統更復雜對運維挑戰很大
3 排錯成本增長
4 數據序列化消耗CPU資源
如今是移動互聯時代，數據無時無刻不在產生着，隨着時間的推移，隨着用戶數的增長，隨着更多業務的開展，隨着功能迭代的頻繁，只有業務服務化只有更易構建分佈式集羣的架構才能應對這些挑戰，畢竟單臺服務器的能力是有限的，即便是IOE這種高端設備。尤爲是如今地下黑產如此猖獗的今天，時不時的遭受到類DDOS攻擊，能夠方便擴展節點的架構是多麼重要，不然會死的很慘。
所以經過RPC協議實現微服務架構是利大於弊的。

 

Thrift
Thrift是RPC服務協議的一種實現，它是由Facebook開發，2007年開源而且2008年成爲Apache開源項目。它的實現目標是支持多語言跨平臺簡單易使用高性能對業務開發透明屏蔽實現細節專一業務開發。大部分語言提供了類庫，可讓開發人員只專一在業務接口實現部分。

實現原理
它由傳輸層，協議層，處理器，服務層所組成，每一個部分不互相依賴，職責單一。
傳輸層：流協議傳輸數據，支持socket，http，文件等媒介
協議層：數據封包解包，支持binary，compact，json等
處理器：接口代理，請求轉發給相應接口處理
服務層：組裝傳輸層，協議層，處理器，提供RPC服務
客戶端調用接口-》客戶端數據封包-》傳輸層-》服端解包-》服端處理器-》服端接口處理。處理完成遵循一樣的鏈路響應。

服端網絡模型
簡單說明，不作深刻了解。
1 單進程
2 單進程多線程
3 單進程事件驅動

IDL
IDL(interface description language)接口描述語言，它包含簡單數據類型定義，複雜數據類型結構體定義，複雜數據類型列表定義集合定義，異常類型定義，命名空間聲明，接口定義規範等，經過這些規範的組合能夠定義好任意複雜的接口，定義好以後，使用IDL編譯器能夠生成指定語言源碼，傳輸層協議層處理器這些代碼會自動生成，只須要專一業務接口具體實現便可。具體類型定義參考官方文檔吧。

安裝
安裝很簡單，不一樣的系統官方文檔都有說明，不須要徹底死板按照步驟執行，相關依賴若是存在能夠跳過相應步驟。安裝成功以後就可使用thrift命令編譯IDL文件了。

命令
定義好的IDL文件名以.thrift結尾，經過thrift編譯。命令參數說明以下：

  1 Usage: thrift [options] file
  2 Options:
  3   -version    Print the compiler version
  4   -o dir      Set the output directory for gen-* packages
  5                (default: current directory)
  6   -out dir    Set the ouput location for generated files.
  7                (no gen-* folder will be created)
  8   -I dir      Add a directory to the list of directories
  9                 searched for include directives
 10   -nowarn     Suppress all compiler warnings (BAD!)
 11   -strict     Strict compiler warnings on
 12   -v[erbose]  Verbose mode
 13   -r[ecurse]  Also generate included files
 14   -debug      Parse debug trace to stdout
 15   --allow-neg-keys  Allow negative field keys (Used to preserve protocol
 16                 compatibility with older .thrift files)
 17   --allow-64bit-consts  Do not print warnings about using 64-bit constants
 18   --gen STR   Generate code with a dynamically-registered generator.
 19                 STR has the form language[:key1=val1[,key2[,key3=val3]]].
 20                 Keys and values are options passed to the generator.
 21                 Many options will not require values.
 22 
 23 Options related to audit operation
 24    --audit OldFile   Old Thrift file to be audited with 'file'
 25   -Iold dir    Add a directory to the list of directories
 26                 searched for include directives for old thrift file
 27   -Inew dir    Add a directory to the list of directories
 28                 searched for include directives for new thrift file
 29 
 30 Available generators (and options):
 31   as3 (AS3):
 32     bindable:        Add [bindable] metadata to all the struct classes.
 33   c_glib (C, using GLib):
 34   cocoa (Cocoa):
 35     log_unexpected:  Log every time an unexpected field ID or type is encountered.
 36     debug_descriptions:
 37                      Allow use of debugDescription so the app can add description via a cateogory/extension
 38     validate_required:
 39                      Throws exception if any required field is not set.
 40     async_clients:   Generate clients which invoke asynchronously via block syntax.
 41     pods:            Generate imports in Cocopods framework format.
 42     promise_kit:     Generate clients which invoke asynchronously via promises.
 43   cpp (C++):
 44     cob_style:       Generate "Continuation OBject"-style classes.
 45     no_client_completion:
 46                      Omit calls to completion__() in CobClient class.
 47     no_default_operators:
 48                      Omits generation of default operators ==, != and <
 49     templates:       Generate templatized reader/writer methods.
 50     pure_enums:      Generate pure enums instead of wrapper classes.
 51     include_prefix:  Use full include paths in generated files.
 52     moveable_types:  Generate move constructors and assignment operators.
 53   csharp (C#):
 54     async:           Adds Async support using Task.Run.
 55     wcf:             Adds bindings for WCF to generated classes.
 56     serial:          Add serialization support to generated classes.
 57     nullable:        Use nullable types for properties.
 58     hashcode:        Generate a hashcode and equals implementation for classes.
 59     union:           Use new union typing, which includes a static read function for union types.
 60   d (D):
 61   dart (Dart):
 62     library_name:    Optional override for library name.
 63     library_prefix:  Generate code that can be used within an existing library.
 64                      Use a dot-separated string, e.g. "my_parent_lib.src.gen"
 65     pubspec_lib:     Optional override for thrift lib dependency in pubspec.yaml,
 66                      e.g. "thrift: 0.x.x".  Use a pipe delimiter to separate lines,
 67                      e.g. "thrift:|  git:|    url: git@foo.com"
 68   delphi (delphi):
 69     ansistr_binary:  Use AnsiString for binary datatype (default is TBytes).
 70     register_types:  Enable TypeRegistry, allows for creation of struct, union
 71                      and container instances by interface or TypeInfo()
 72     constprefix:     Name TConstants classes after IDL to reduce ambiguities
 73     events:          Enable and use processing events in the generated code.
 74     xmldoc:          Enable XMLDoc comments for Help Insight etc.
 75   erl (Erlang):
 76     legacynames: Output files retain naming conventions of Thrift 0.9.1 and earlier.
 77     maps:        Generate maps instead of dicts.
 78     otp16:       Generate non-namespaced dict and set instead of dict:dict and sets:set.
 79   go (Go):
 80     package_prefix=  Package prefix for generated files.
 81     thrift_import=   Override thrift package import path (default:git.apache.org/thrift.git/lib/go/thrift)
 82     package=         Package name (default: inferred from thrift file name)
 83     ignore_initialisms
 84                      Disable automatic spelling correction of initialisms (e.g. "URL")
 85     read_write_private
 86                      Make read/write methods private, default is public Read/Write
 87   gv (Graphviz):
 88     exceptions:      Whether to draw arrows from functions to exception.
 89   haxe (Haxe):
 90     callbacks        Use onError()/onSuccess() callbacks for service methods (like AS3)
 91     rtti             Enable @:rtti for generated classes and interfaces
 92     buildmacro=my.macros.Class.method(args)
 93                      Add @:build macro calls to generated classes and interfaces
 94   hs (Haskell):
 95   html (HTML):
 96     standalone:      Self-contained mode, includes all CSS in the HTML files.
 97                      Generates no style.css file, but HTML files will be larger.
 98     noescape:        Do not escape html in doc text.
 99   java (Java):
100     beans:           Members will be private, and setter methods will return void.
101     private-members: Members will be private, but setter methods will return 'this' like usual.
102     nocamel:         Do not use CamelCase field accessors with beans.
103     fullcamel:       Convert underscored_accessor_or_service_names to camelCase.
104     android:         Generated structures are Parcelable.
105     android_legacy:  Do not use java.io.IOException(throwable) (available for Android 2.3 and above).
106     option_type:     Wrap optional fields in an Option type.
107     java5:           Generate Java 1.5 compliant code (includes android_legacy flag).
108     reuse-objects:   Data objects will not be allocated, but existing instances will be used (read and write).
109     sorted_containers:
110                      Use TreeSet/TreeMap instead of HashSet/HashMap as a implementation of set/map.
111     generated_annotations=[undated|suppress]:
112                      undated: suppress the date at @Generated annotations
113                      suppress: suppress @Generated annotations entirely
114   javame (Java ME):
115   js (Javascript):
116     jquery:          Generate jQuery compatible code.
117     node:            Generate node.js compatible code.
118     ts:              Generate TypeScript definition files.
119   json (JSON):
120     merge:           Generate output with included files merged
121   lua (Lua):
122     omit_requires:   Suppress generation of require 'somefile'.
123   ocaml (OCaml):
124   perl (Perl):
125   php (PHP):
126     inlined:         Generate PHP inlined files
127     server:          Generate PHP server stubs
128     oop:             Generate PHP with object oriented subclasses
129     rest:            Generate PHP REST processors
130     nsglobal=NAME:   Set global namespace
131     validate:        Generate PHP validator methods
132     json:            Generate JsonSerializable classes (requires PHP >= 5.4)
133   py (Python):
134     twisted:         Generate Twisted-friendly RPC services.
135     tornado:         Generate code for use with Tornado.
136     no_utf8strings:  Do not Encode/decode strings using utf8 in the generated code. Basically no effect for Python 3.
137     coding=CODING:   Add file encoding declare in generated file.
138     slots:           Generate code using slots for instance members.
139     dynamic:         Generate dynamic code, less code generated but slower.
140     dynbase=CLS      Derive generated classes from class CLS instead of TBase.
141     dynfrozen=CLS    Derive generated immutable classes from class CLS instead of TFrozenBase.
142     dynexc=CLS       Derive generated exceptions from CLS instead of TExceptionBase.
143     dynimport='from foo.bar import CLS'
144                      Add an import line to generated code to find the dynbase class.
145     package_prefix='top.package.'
146                      Package prefix for generated files.
147     old_style:       Deprecated. Generate old-style classes.
148   rb (Ruby):
149     rubygems:        Add a "require 'rubygems'" line to the top of each generated file.
150     namespaced:      Generate files in idiomatic namespaced directories.
151   st (Smalltalk):
152   swift (Swift):
153     log_unexpected:  Log every time an unexpected field ID or type is encountered.
154     debug_descriptions:
155                      Allow use of debugDescription so the app can add description via a cateogory/extension
156     async_clients:   Generate clients which invoke asynchronously via block syntax.
157     promise_kit:     Generate clients which invoke asynchronously via promises.
158   xml (XML):
159     merge:           Generate output with included files merged
160     no_default_ns:   Omit default xmlns and add idl: prefix to all elements
161     no_namespaces:   Do not add namespace definitions to the XML model
162   xsd (XSD):

View Code

 

實踐
日常使用PHP開發，因此使用PHP實踐下，對它有更深刻的瞭解，通常服端開發使用編譯型的語言，執行效率更高，以實現簡單的NoSQL功能爲例子(源碼裏有各類語言的實現例子)。
1 接口定義(nosql.thrift)

 1/*
 3  * thrift簡單示例 模仿thrift源碼教程
 4  * nosql數據庫簡單實現
 5  */
 6  
 7 #由wadeyu建立 wadeyu.cnblogs.com
 8 
 9 /**
10  * 命名空間
11  */
12 namespace cpp NoSql
13 namespace java NoSql
14 namespace php NoSql
15 
16 /**
17  * 異常:無效參數
18  */
19 exception InvalidParametorException{
20 }
21 
22 /**
23  * 定義服務
24  */
25 service NoSqlService{
26     /**
27      * 獲取key的值
28      */
29     string get(1:string key) throws (1:InvalidParametorException ex),
30     
31     /**
32      * 設置值
33      */
34     bool set_(1:string key, 2:string value) throws (1:InvalidParametorException ex),
35     
36     /**
37      * 自增
38      */
39     i32 incr(1:string key) throws (1:InvalidParametorException ex),
40 }

2 編譯(生成服端代碼)

 1 [wadeyu@localhost thriftdemo]$ thrift --gen php:server -out ./server ./meta/nosql.thrift 

3 編寫服端啓動代碼

 1 <?php
 2 /*
 3  * php簡單服端腳本
 4  */
 5  
 6 #author by wadeyu: wadeyu.cnblogs.com
 7 
 8 define('BASE_DIR',dirname(__FILE__) . '/');
 9 define('VENDOR_DIR',BASE_DIR.'vendor/');
10 
11 use Server\NoSqlHandler;
12 use NoSql\NoSqlServiceProcessor;
13 use Thrift\Factory\TBinaryProtocolFactory;
14 use Thrift\Factory\TTransportFactory;
15 use Thrift\Server\TServerSocket;
16 use Thrift\Server\TSimpleServer;
17 
18 $loader = include_once VENDOR_DIR.'autoload.php';
19 $loader->addPsr4('Server\\',BASE_DIR.'server/');
20 
21 include_once BASE_DIR.'server/NoSql/NoSqlService.php';
22 include_once BASE_DIR.'server/NoSql/Types.php';
23 
24 $serverTransport = new TServerSocket('localhost',9090);
25 $clientTransport = new TTransportFactory;
26 $binaryProtocol  = new TBinaryProtocolFactory;
27 $nosqlProcessor  = new NoSqlServiceProcessor( new NoSqlHandler );
28 $simpleServer    = new TSimpleServer(
29     $nosqlProcessor,
30     $serverTransport,
31     $clientTransport,
32     $clientTransport,
33     $binaryProtocol,
34     $binaryProtocol
35 );
36 echo "start listening:localhost:9090 \n";
37 $simpleServer->serve();

4 啓動服務

[wadeyu@localhost thriftdemo]$ php phpserver.php 
start listening:localhost:9090

5 編譯(生成客戶端代碼)

[wadeyu@localhost thriftdemo]$ thrift --gen php -out ./client ./meta/nosql.thrift

6 客戶端接口調用

 1 <?php
 2 /*
 3  * php簡單客戶端腳本
 4  */
 5  
 6 #author by wadeyu: wadeyu.cnblogs.com
 7 
 8 define('BASE_DIR',dirname(__FILE__) . '/');
 9 define('VENDOR_DIR',BASE_DIR.'vendor/');
10 
11 use Thrift\Protocol\TBinaryProtocol;
12 use Thrift\Transport\TSocket;
13 use NoSql\NoSqlServiceClient;
14 
15 $loader = include_once VENDOR_DIR.'autoload.php';
16 
17 include_once BASE_DIR.'client/NoSql/NoSqlService.php';
18 include_once BASE_DIR.'client/NoSql/Types.php';
19 
20 $transport = new TSocket('localhost',9090);
21 $protocol  = new TBinaryProtocol($transport);
22 $service   = new NoSqlServiceClient($protocol);
23 $startTime = microtime(true);
24 try{
25     $transport->open();
26     $key1 = 'test1';
27     $ret = $service->get($key1);
28     var_dump($ret);
29     $ret = $service->set_($key1,'test1');
30     var_dump($ret);
31     $key2 = 'test2';
32     $ret = $service->incr($key2);
33     var_dump($ret);
34     $transport->close();
35 }catch(Exception $ex){
36     throw $ex;
37 }
38 var_dump('cost:'.(microtime(true)-$startTime));

1 [wadeyu@localhost thriftdemo]$ php phpclient.php 
2 string(4) "NULL"
3 bool(true)
4 int(1)
5 string(21) "cost:0.22478580474854"

 

示例源碼下載
https://github.com/wadeyu/thriftdemo/archive/master.zip

 

後記
紙上得來終覺淺，得知此事要躬行。其實很早就開始寫這篇文章了，本身很懶，拖了幾個星期才完成，之後要改掉這個毛病了。

 

參考資料
【1】thrift官方文檔

http://thrift.apache.org/docs/
【2】thrift安裝

http://thrift.apache.org/docs/install/
【3】thrift IDL規範

http://thrift.apache.org/docs/idl

http://thrift.apache.org/docs/types
【4】thrift實現論文

http://thrift.apache.org/static/files/thrift-20070401.pdf