Google.ProtocolBuffers.dll 之.Net應用（一）

原創文章，轉載必需註明出處：http://www.cnblogs.com/wu-jian/

http://www.cnblogs.com/wu-jian/archive/2011/02/22/1961104.html

核心提示：Google Protocol Buffers是google出品的一個協議生成工具，特色就是跨平臺，Google Protocol Buffers 快速入門（帶生成C#源碼的方法），效率高，速度快，只有 AddressBookProtos.cs有用，將這個文件連同Google.ProtocolBuffers.dll一塊兒.

前言

最近接到一個跨平臺的測試項目，服務端Linux，是Java開發的一系列Socket接口，客戶端Windows，因此準備用.Net。本想這種跨主流平臺的Socket通訊應該不成問題，但隨着代碼進程，隨着一次次反覆調試，我發現我錯了。花了一週時間至今二者仍呈現北方網通和南方電信的姿態。

不過總有意外驚喜，過程當中認識了Protocol Buffer，比XML、比JSON更爲強悍，語言無關、平臺無關、更小的存儲、更少的歧義、更高的性能，其實Google一直在貢獻，不管是Copy Left的仍是Copy Right的，回頭看看咱們的百度，抄IM抄商城抄遊戲抄視頻抄房地產，還有搜索永遠排第一卻打不開的百度文庫，印象中JQuery盛行N久以後百度開源了一個JS庫，記憶裏這也是百度爲中國互聯網技術作的惟一貢獻，大公司的責任吶，好了，再說就偏離主題了。

Protocal Buffer官方站點：http://code.google.com/p/protobuf/ ，遺憾的是不支持.Net，但社區的力量不容忽視，MySQL最近還推出社區版呢，從這個連接能夠看到Protobuf的社區陣營：http://code.google.com/p/protobuf/wiki/ThirdPartyAddOns

OK，本文主要描述本身在.Net中基於應用層面使用Protobuf的一些體會，做爲學習筆記與你們分享，我的能力有限，不足之處還請及時指正。

 

需求

Java爲服務端，.Net爲客戶端，Socket通訊，使用Protobuf進行數據封裝和傳輸，以下圖：

DEMO中構造了3個簡單的.proto文件供各客戶端使用：

message MyRequest {
// 版本號
required int32 version  = 1 ;
// 姓名
required string name  = 2 ;
// 我的網站
optional string website  = 3 [ default = " http://www.paotiao.com/ " ];
// 附加數據
optional bytes data  = 4 ;
}

message MyResponse {
// 版本號
required int32 version  = 1 ;
// 響應結果
required int32 result  = 2 ;
}

message MyData {
// 我的簡介
optional string resume  = 1 [ default = " I'm goodman " ];
}

其中MyRequest爲客戶端的請求，MyResponse爲服務端的響應，MyData做爲一個屬性附加在MyRequest的data字段中，提醒注意這個byte類型的data字段，爲此花費了最多時間並最終導至放棄Protobuf-net來作跨平臺的應用。

 

Protobuf-net

官方站點：http://code.google.com/p/protobuf-net/

Protobuf-net是第三方中最強大應用最普遍的一個，支持.Net、C#、WCF、VB，而且DEMO豐富，網上可查到的資料也最多。

生成.CS類文件

安裝後經過 protogen.exe 就可將.proto文件生成.cs文件(Demo中我將命令封裝在/tools/getCS.bat中)：

 echo on
protogen -i:ProtoMyRequest.proto -o:ProtoMyRequest.cs
protogen -i:ProtoMyResponse.proto -o:ProtoMyResponse.cs
protogen -i:ProtoMyData.proto -o:ProtoMyData.cs

接着將生成的3個.cs文件包含在項目中，同時在項目中引用protobuf-net.dll

代碼示例(服務端與客戶端)

?

using  System; using  System.IO; using  System.Text; using  System.Threading; using  System.Net; using  System.Net.Sockets; using  System.Runtime.Serialization; using  System.Runtime.Serialization.Formatters.Binary; using  ProtoBuf; //自定義 using  ProtoMyData; using  ProtoMyRequest; using  ProtoMyResponse;   namespace  protobuf_net {      class  Program      {          private  static  ManualResetEvent allDone = new  ManualResetEvent( false );            static  void  Main( string [] args)          {              beginDemo();          }            private  static  void  beginDemo()          {              //啓動服務端              TcpListener server = new  TcpListener(IPAddress.Parse( "127.0.0.1" ), 9527);              server.Start();              server.BeginAcceptTcpClient(clientConnected, server);              Console.WriteLine( "SERVER : 等待數據 ---" );                //啓動客戶端              ThreadPool.QueueUserWorkItem(runClient);              allDone.WaitOne();                Console.WriteLine( "SERVER : 退出 ---" );              server.Stop();          }            //服務端處理          private  static  void  clientConnected(IAsyncResult result)          {              try              {                  TcpListener server = (TcpListener)result.AsyncState;                  using  (TcpClient client = server.EndAcceptTcpClient(result))                  using  (NetworkStream stream = client.GetStream())                  {                      //獲取                      Console.WriteLine( "SERVER : 客戶端已鏈接，讀取數據 ---" );                      //proto-buf 使用 Base128 Varints 編碼                      MyRequest myRequest = Serializer.DeserializeWithLengthPrefix<MyRequest>(stream, PrefixStyle.Base128);                        //使用C# BinaryFormatter                      IFormatter formatter = new  BinaryFormatter();                      MyData myData = (MyData)formatter.Deserialize( new  MemoryStream(myRequest.data));                      //MyData.MyData mydata = Serializer.DeserializeWithLengthPrefix<MyData.MyData>(new MemoryStream(request.data), PrefixStyle.Base128);                        Console.WriteLine( "SERVER : 獲取成功, myRequest.version={0}, myRequest.name={1}, myRequest.website={2}, myData.resume={3}" , myRequest.version, myRequest.name, myRequest.website, myData.resume);                        //響應(MyResponse)                      MyResponse myResponse = new  MyResponse();                      myResponse.version = myRequest.version;                      myResponse.result = 99;                      Serializer.SerializeWithLengthPrefix(stream, myResponse, PrefixStyle.Base128);                      Console.WriteLine( "SERVER : 響應成功 ---" );                        //DEBUG                      //int final = stream.ReadByte();                      //if (final == 123)                      //{                      //    Console.WriteLine("SERVER: Got client-happy marker");                      //}                      //else                      //{                      //    Console.WriteLine("SERVER: OOPS! Something went wrong");                      //}                      Console.WriteLine( "SERVER: 關閉鏈接 ---" );                      stream.Close();                      client.Close();                  }              }              finally              {                  allDone.Set();              }          }            //客戶端請求          private  static  void  runClient( object  state)          {              try              {                  //構造MyData                  MyData myData = new  MyData();                  myData.resume = "個人我的簡介" ;                    //構造MyRequest                  MyRequest myRequest = new  MyRequest();                  myRequest.version = 1;                  myRequest.name = "吳劍" ;                  myRequest.website = "www.paotiao.com" ;                    //使用C# BinaryFormatter                  using  (MemoryStream ms = new  MemoryStream())                  {                      IFormatter formatter = new  BinaryFormatter();                      formatter.Serialize(ms, myData);                      //Serializer.Serialize(ms, mydata);                      myRequest.data = ms.GetBuffer();                      ms.Close();                  }                  Console.WriteLine( "CLIENT : 對象構造完畢 ..." );                    using  (TcpClient client = new  TcpClient())                  {                      client.Connect( new  IPEndPoint(IPAddress.Parse( "127.0.0.1" ), 9527));                      Console.WriteLine( "CLIENT : socket 鏈接成功 ..." );                        using  (NetworkStream stream = client.GetStream())                      {                          //發送                          Console.WriteLine( "CLIENT : 發送數據 ..." );                          ProtoBuf.Serializer.SerializeWithLengthPrefix(stream, myRequest, PrefixStyle.Base128);                            //接收                          Console.WriteLine( "CLIENT : 等待響應 ..." );                          MyResponse myResponse = ProtoBuf.Serializer.DeserializeWithLengthPrefix<MyResponse>(stream, PrefixStyle.Base128);                            Console.WriteLine( "CLIENT : 成功獲取結果, version={0}, result={1}" , myResponse.version, myResponse.result);                            //DEBUG client-happy marker                          //stream.WriteByte(123);                            //關閉                          stream.Close();                      }                      client.Close();                      Console.WriteLine( "CLIENT : 關閉 ..." );                  }              }              catch  (Exception error)              {                  Console.WriteLine( "CLIENT ERROR : {0}" , error.ToString());              }          }        } //end class }

從代碼中能夠發現protobuf-net已考慮的很是周到，不管是客戶端發送對象仍是服務端接收對象，均只需一行代碼就可實現：

// 客戶端發送對象
ProtoBuf.Serializer.SerializeWithLengthPrefix(stream, myRequest, PrefixStyle.Base128);
// 服務端接收對象
MyRequest myRequest  =  Serializer.DeserializeWithLengthPrefix < MyRequest > (stream, PrefixStyle.Base128);

因此若是Server與Client均使用.Net，Protobuf-net會是理想選擇。

但個人項目須要跨平臺，同時項目中又偏偏使用了byte類型字段，通過反覆調試比較，發現一個關鍵問題：假使proto腳本和對象屬性值徹底同樣，但只要包含byte類型的字段，那麼經過Java序列化的二進制與C#序列化的二進制結果必定不一樣。而Protobuf中Google原生支持Java，那麼幾乎能夠肯定Protobuf-net對Protobuf的支持存在瑕疵。

後來在使用Protobuf-csharp-sport後驗證了這一點，Protobuf-net使用C#的byte[]來實現bytes，而Java以及Protobuf-csharp-port均使用ByteString，前者是無符號的，後者是有符號的，語言的基本差別致使二者沒法兼容，因此最終我只能放棄Protobuf-net。

 

Protobuf-csharp-port

官方站點：http://code.google.com/p/protobuf-csharp-port/

Protobuf-csharp-port的文檔資料、DEMO、應用範圍都不如Protobuf-net，但Protobuf-csharp-port更遵循Google的Protobuf，甚至應用和代碼都幾乎同樣，因此跨平臺，Protobuf-csharp-port是不二之選。

生成.CS類文件

先直接使用Google的 protoc.exe 生成二進制文件。

而後經過 protogen.exe 將二進制文件生成C#類文件(Demo中我將命令封裝在/tools/getCS.bat中)：

echo on
protoc --descriptor_set_out=ProtoMyRequest.protobin --include_imports ProtoMyRequest.proto
protoc --descriptor_set_out=ProtoMyResponse.protobin --include_imports ProtoMyResponse.proto
protoc --descriptor_set_out=ProtoMyData.protobin --include_imports ProtoMyData.proto

protogen ProtoMyRequest.protobin
protogen ProtoMyResponse.protobin
protogen ProtoMyData.protobin

接着將生成的3個.cs文件包含在項目中，同時在項目中引用Google.ProtocolBuffers.dll

代碼示例（服務端與客戶端）

?

using  System; using  System.IO; using  System.Net; using  System.Net.Sockets; using  System.Threading; using  Google.ProtocolBuffers;   namespace  protobuf_csharp_sport {      class  Program      {          private  static  ManualResetEvent allDone = new  ManualResetEvent( false );            static  void  Main( string [] args)          {              beginDemo();          }            private  static  void  beginDemo()          {              //啓動服務端              TcpListener server = new  TcpListener(IPAddress.Parse( "127.0.0.1" ), 9528);              server.Start();              server.BeginAcceptTcpClient(clientConnected, server);              Console.WriteLine( "SERVER : 等待數據 ---" );                //啓動客戶端              ThreadPool.QueueUserWorkItem(runClient);              allDone.WaitOne();                Console.WriteLine( "SERVER : 退出 ---" );              server.Stop();          }            //服務端處理          private  static  void  clientConnected(IAsyncResult result)          {              try              {                  TcpListener server = (TcpListener)result.AsyncState;                  using  (TcpClient client = server.EndAcceptTcpClient(result))                  {                      using  (NetworkStream stream = client.GetStream())                      {                          //獲取                          Console.WriteLine( "SERVER : 客戶端已鏈接，數據讀取中 --- " );                          byte [] myRequestBuffer = new  byte [49];                          int  myRequestLength = 0;                          do                          {                              myRequestLength = stream.Read(myRequestBuffer, 0, myRequestBuffer.Length);                          }                          while  (stream.DataAvailable);                          MyRequest myRequest = MyRequest.ParseFrom(myRequestBuffer);                          MyData myData = MyData.ParseFrom(myRequest.Data);                          Console.WriteLine( "SERVER : 獲取成功, myRequest.Version={0}, myRequest.Name={1}, myRequest.Website={2}, myData.Resume={3}" , myRequest.Version, myRequest.Name, myRequest.Website, myData.Resume);                            //響應(MyResponse)                          MyResponse.Builder myResponseBuilder = MyResponse.CreateBuilder();                          myResponseBuilder.Version = myRequest.Version;                          myResponseBuilder.Result = 99;                          MyResponse myResponse = myResponseBuilder.Build();                          myResponse.WriteTo(stream);                          Console.WriteLine( "SERVER : 響應成功 ---" );                            Console.WriteLine( "SERVER: 關閉鏈接 ---" );                          stream.Close();                                             }                      client.Close();                  }              }              finally              {                  allDone.Set();              }          }            //客戶端請求          private  static  void  runClient( object  state)          {              try              {                  //構造MyData                  MyData.Builder myDataBuilder = MyData.CreateBuilder();                  myDataBuilder.Resume = "個人我的簡介" ;                  MyData myData = myDataBuilder.Build();                                    //構造MyRequest                  MyRequest.Builder myRequestBuilder = MyRequest.CreateBuilder();                  myRequestBuilder.Version = 1;                  myRequestBuilder.Name = "吳劍" ;                  myRequestBuilder.Website = "www.paotiao.com" ;                  //注：直接支持ByteString類型                  myRequestBuilder.Data = myData.ToByteString();                  MyRequest myRequest = myRequestBuilder.Build();                                                    Console.WriteLine( "CLIENT : 對象構造完畢 ..." );                    using  (TcpClient client = new  TcpClient())                  {                      client.Connect( new  IPEndPoint(IPAddress.Parse( "127.0.0.1" ), 9528));                      Console.WriteLine( "CLIENT : socket 鏈接成功 ..." );                        using  (NetworkStream stream = client.GetStream())                      {                          //發送                          Console.WriteLine( "CLIENT : 發送數據 ..." );                          myRequest.WriteTo(stream);                            //接收                          Console.WriteLine( "CLIENT : 等待響應 ..." );                          byte [] myResponseBuffer = new  byte [4];                          int  myResponseLength = 0;                          do                          {                              myResponseLength = stream.Read(myResponseBuffer, 0, myResponseBuffer.Length);                          }                          while  (stream.DataAvailable);                                                 MyResponse myResponse = MyResponse.ParseFrom(myResponseBuffer);                          Console.WriteLine( "CLIENT : 成功獲取結果, myResponse.Version={0}, myResponse.Result={1}" , myResponse.Version, myResponse.Result);                            //關閉                          stream.Close();                      }                      client.Close();                      Console.WriteLine( "CLIENT : 關閉 ..." );                  }              }              catch  (Exception error)              {                  Console.WriteLine( "CLIENT ERROR : {0}" , error.ToString());              }          }        } //end class }

 

Protobuf#

官方站點：http://code.google.com/p/protosharp/

暫未測試Protobuf#

 

結束語

基本花了一週時間瞭解和學習了Google Protobuf在.NET下的應用，也找到了Protobuf跨平臺的方案，但好事多魔，C# Socket發送的protobuf數據包在Java Netty 中怎麼也獲取不到，我想也許是平臺差別，但對Java知之甚少，若有知情人士還請指點迷津。

 

DEMO

DEMO下載：http://files.cnblogs.com/wu-jian/ProtobufDemo.rar

DEMO運行環境：.Net Framework 4.0， VS2010

 

<全文完>

 

做者：吳劍
出處：http://www.cnblogs.com/wu-jian/本文版權歸做者和博客園共有，歡迎轉載，但必需註明出處，而且在文章頁面明顯位置給出原文鏈接，不然保留追究法律責任的權利。