Protocol Buffers學習教程

時間 2019-12-12

原文原文鏈接

最近看公司代碼的過程當中，看到了不少proto後綴的文件，這是個啥玩意？問了大佬，原來這是Protocol Buffers！html

這玩意是幹啥的？查完資料才知道，又是谷歌大佬推的開源組件，這玩意徹底能夠取代XML和JSON的數據交換格式，並且更加快！git

Protocol Buffer 即 PB 是大 Google 公司推行的一套混合語言數據標準, 標準介紹以下:github

是 Google 開源的一種輕便高效的結構化數據存儲格式，能夠用於結構化數據的串行化，也稱做序列化，主要用於數據存儲或是 RPC 數據交換，支持多語言，可拓展. 它很適合作數據存儲或 RPC 數據交換格式. 可用於通信協議、數據存儲等領域的語言無關、平臺無關、可擴展的序列化結構數據格式。golang

感受是否是比較抽象?編程

簡單而言, 它的出現就是想戰勝 xml & json, pb 其實和他倆差很少。json

既然是 Google 推行的, 確定是採用開源策略, 附上 github 地址 (目前 star 超過 2W)編程語言

PB 首頁地址性能

優勢this

◇性能好 / 效率高
◇代碼生成機制 – 這個後面說
◇支持「向後兼容」和「向前兼容」
◇支持多種編程語言google

缺點

◇二進制格式致使可讀性差
爲了提升性能，protobuf 採用了二進制格式進行編碼。這直接致使了可讀性差的問題（嚴格地說，是沒有可讀性）
◇缺少自描述
通常來講，XML 是自描述的，而 protobuf 格式則不是。給你一段二進制格式的協議內容，若是不配合相應的 proto 文件，那簡直就像天書通常。

參考下面這篇文章安裝：

mac 安裝 protocol buffer（2.6.1 版）教程

先看一下簡介：

https://developers.google.cn/protocol-buffers/docs/overview

直接上手：

https://developers.google.cn/protocol-buffers/docs/tutorials

我最近在學Go，這裏就用Go了，你只須要選擇本身感興趣的語言便可。

首先看一下個人項目目錄：

在$GOPATH/src/github.com/chenchi/procotolbuffers/這個目錄。

開始我寫一個addressbook.proto的文件，具體裏面的字段啥含義，直接看上面的文檔，雖然是英文，很容易理解。

syntax = "proto3";
package procotolbuffers;

import "google/protobuf/timestamp.proto";

message Person {
    string name = 1;
    int32 id = 2;  // Unique ID number for this person.
    string email = 3;

    enum PhoneType {
        MOBILE = 0;
        HOME = 1;
        WORK = 2;
    }

    message PhoneNumber {
        string number = 1;
        PhoneType type = 2;
    }

    repeated PhoneNumber phones = 4;

    google.protobuf.Timestamp last_updated = 5;
}

// Our address book file is just one of these.
message AddressBook {
    repeated Person people = 1;
}

接着執行：

go get -u github.com/golang/protobuf/protoc-gen-go

protoc -I=$SRC_DIR --go_out=$DST_DIR $SRC_DIR/addressbook.proto

我打算跟addressbook.proto生成在同一目錄下，因此我就寫當前目錄了，即

protoc -I=. --go_out=. ./addressbook.proto

生成完了就是那個addressbook.pb.go文件，這裏面不少東西：

// Code generated by protoc-gen-go. DO NOT EDIT.
// source: addressbook.proto

package procotolbuffers

import (
    fmt "fmt"
    proto "github.com/golang/protobuf/proto"
    timestamp "github.com/golang/protobuf/ptypes/timestamp"
    math "math"
)

// Reference imports to suppress errors if they are not otherwise used.
var _ = proto.Marshal
var _ = fmt.Errorf
var _ = math.Inf

// This is a compile-time assertion to ensure that this generated file
// is compatible with the proto package it is being compiled against.
// A compilation error at this line likely means your copy of the
// proto package needs to be updated.
const _ = proto.ProtoPackageIsVersion3 // please upgrade the proto package

type Person_PhoneType int32

const (
    Person_MOBILE Person_PhoneType = 0
    Person_HOME   Person_PhoneType = 1
    Person_WORK   Person_PhoneType = 2
)

var Person_PhoneType_name = map[int32]string{
    0: "MOBILE",
    1: "HOME",
    2: "WORK",
}

var Person_PhoneType_value = map[string]int32{
    "MOBILE": 0,
    "HOME":   1,
    "WORK":   2,
}

func (x Person_PhoneType) String() string {
    return proto.EnumName(Person_PhoneType_name, int32(x))
}

func (Person_PhoneType) EnumDescriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{0, 0}
}

type Person struct {
    Name                 string                `protobuf:"bytes,1,opt,name=name,proto3" json:"name,omitempty"`
    Id                   int32                 `protobuf:"varint,2,opt,name=id,proto3" json:"id,omitempty"`
    Email                string                `protobuf:"bytes,3,opt,name=email,proto3" json:"email,omitempty"`
    Phones               []*Person_PhoneNumber `protobuf:"bytes,4,rep,name=phones,proto3" json:"phones,omitempty"`
    LastUpdated          *timestamp.Timestamp  `protobuf:"bytes,5,opt,name=last_updated,json=lastUpdated,proto3" json:"last_updated,omitempty"`
    XXX_NoUnkeyedLiteral struct{}              `json:"-"`
    XXX_unrecognized     []byte                `json:"-"`
    XXX_sizecache        int32                 `json:"-"`
}

func (m *Person) Reset()         { *m = Person{} }
func (m *Person) String() string { return proto.CompactTextString(m) }
func (*Person) ProtoMessage()    {}
func (*Person) Descriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{0}
}

func (m *Person) XXX_Unmarshal(b []byte) error {
    return xxx_messageInfo_Person.Unmarshal(m, b)
}
func (m *Person) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
    return xxx_messageInfo_Person.Marshal(b, m, deterministic)
}
func (m *Person) XXX_Merge(src proto.Message) {
    xxx_messageInfo_Person.Merge(m, src)
}
func (m *Person) XXX_Size() int {
    return xxx_messageInfo_Person.Size(m)
}
func (m *Person) XXX_DiscardUnknown() {
    xxx_messageInfo_Person.DiscardUnknown(m)
}

var xxx_messageInfo_Person proto.InternalMessageInfo

func (m *Person) GetName() string {
    if m != nil {
        return m.Name
    }
    return ""
}

func (m *Person) GetId() int32 {
    if m != nil {
        return m.Id
    }
    return 0
}

func (m *Person) GetEmail() string {
    if m != nil {
        return m.Email
    }
    return ""
}

func (m *Person) GetPhones() []*Person_PhoneNumber {
    if m != nil {
        return m.Phones
    }
    return nil
}

func (m *Person) GetLastUpdated() *timestamp.Timestamp {
    if m != nil {
        return m.LastUpdated
    }
    return nil
}

type Person_PhoneNumber struct {
    Number               string           `protobuf:"bytes,1,opt,name=number,proto3" json:"number,omitempty"`
    Type                 Person_PhoneType `protobuf:"varint,2,opt,name=type,proto3,enum=procotolbuffers.Person_PhoneType" json:"type,omitempty"`
    XXX_NoUnkeyedLiteral struct{}         `json:"-"`
    XXX_unrecognized     []byte           `json:"-"`
    XXX_sizecache        int32            `json:"-"`
}

func (m *Person_PhoneNumber) Reset()         { *m = Person_PhoneNumber{} }
func (m *Person_PhoneNumber) String() string { return proto.CompactTextString(m) }
func (*Person_PhoneNumber) ProtoMessage()    {}
func (*Person_PhoneNumber) Descriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{0, 0}
}

func (m *Person_PhoneNumber) XXX_Unmarshal(b []byte) error {
    return xxx_messageInfo_Person_PhoneNumber.Unmarshal(m, b)
}
func (m *Person_PhoneNumber) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
    return xxx_messageInfo_Person_PhoneNumber.Marshal(b, m, deterministic)
}
func (m *Person_PhoneNumber) XXX_Merge(src proto.Message) {
    xxx_messageInfo_Person_PhoneNumber.Merge(m, src)
}
func (m *Person_PhoneNumber) XXX_Size() int {
    return xxx_messageInfo_Person_PhoneNumber.Size(m)
}
func (m *Person_PhoneNumber) XXX_DiscardUnknown() {
    xxx_messageInfo_Person_PhoneNumber.DiscardUnknown(m)
}

var xxx_messageInfo_Person_PhoneNumber proto.InternalMessageInfo

func (m *Person_PhoneNumber) GetNumber() string {
    if m != nil {
        return m.Number
    }
    return ""
}

func (m *Person_PhoneNumber) GetType() Person_PhoneType {
    if m != nil {
        return m.Type
    }
    return Person_MOBILE
}

// Our address book file is just one of these.
type AddressBook struct {
    People               []*Person `protobuf:"bytes,1,rep,name=people,proto3" json:"people,omitempty"`
    XXX_NoUnkeyedLiteral struct{}  `json:"-"`
    XXX_unrecognized     []byte    `json:"-"`
    XXX_sizecache        int32     `json:"-"`
}

func (m *AddressBook) Reset()         { *m = AddressBook{} }
func (m *AddressBook) String() string { return proto.CompactTextString(m) }
func (*AddressBook) ProtoMessage()    {}
func (*AddressBook) Descriptor() ([]byte, []int) {
    return fileDescriptor_1eb1a68c9dd6d429, []int{1}
}

func (m *AddressBook) XXX_Unmarshal(b []byte) error {
    return xxx_messageInfo_AddressBook.Unmarshal(m, b)
}
func (m *AddressBook) XXX_Marshal(b []byte, deterministic bool) ([]byte, error) {
    return xxx_messageInfo_AddressBook.Marshal(b, m, deterministic)
}
func (m *AddressBook) XXX_Merge(src proto.Message) {
    xxx_messageInfo_AddressBook.Merge(m, src)
}
func (m *AddressBook) XXX_Size() int {
    return xxx_messageInfo_AddressBook.Size(m)
}
func (m *AddressBook) XXX_DiscardUnknown() {
    xxx_messageInfo_AddressBook.DiscardUnknown(m)
}

var xxx_messageInfo_AddressBook proto.InternalMessageInfo

func (m *AddressBook) GetPeople() []*Person {
    if m != nil {
        return m.People
    }
    return nil
}

func init() {
    proto.RegisterEnum("procotolbuffers.Person_PhoneType", Person_PhoneType_name, Person_PhoneType_value)
    proto.RegisterType((*Person)(nil), "procotolbuffers.Person")
    proto.RegisterType((*Person_PhoneNumber)(nil), "procotolbuffers.Person.PhoneNumber")
    proto.RegisterType((*AddressBook)(nil), "procotolbuffers.AddressBook")
}

func init() { proto.RegisterFile("addressbook.proto", fileDescriptor_1eb1a68c9dd6d429) }

var fileDescriptor_1eb1a68c9dd6d429 = []byte{
    // 321 bytes of a gzipped FileDescriptorProto
    0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0xff, 0x7c, 0x91, 0x41, 0x4b, 0xc3, 0x40,
    0x10, 0x85, 0x4d, 0x9a, 0x06, 0x3b, 0x91, 0x5a, 0x17, 0xd1, 0xd0, 0x8b, 0xb1, 0x5e, 0x02, 0x42,
    0x0a, 0x15, 0x4f, 0xa2, 0x60, 0xa1, 0xa0, 0x68, 0x6d, 0x59, 0x2a, 0x5e, 0x04, 0x49, 0xcc, 0xb4,
    0x86, 0x26, 0x99, 0x25, 0xbb, 0x39, 0xf4, 0x27, 0xfa, 0xaf, 0x24, 0x9b, 0x54, 0x44, 0xd0, 0xdb,
    0x9b, 0xdd, 0x6f, 0x76, 0xde, 0x9b, 0x85, 0x83, 0x30, 0x8e, 0x0b, 0x94, 0x32, 0x22, 0x5a, 0x07,
    0xa2, 0x20, 0x45, 0x6c, 0x5f, 0x14, 0xf4, 0x4e, 0x8a, 0xd2, 0xa8, 0x5c, 0x2e, 0xb1, 0x90, 0xfd,
    0x93, 0x15, 0xd1, 0x2a, 0xc5, 0xa1, 0xbe, 0x8e, 0xca, 0xe5, 0x50, 0x25, 0x19, 0x4a, 0x15, 0x66,
    0xa2, 0xee, 0x18, 0x7c, 0x9a, 0x60, 0xcf, 0xb1, 0x90, 0x94, 0x33, 0x06, 0x56, 0x1e, 0x66, 0xe8,
    0x1a, 0x9e, 0xe1, 0x77, 0xb8, 0xd6, 0xac, 0x0b, 0x66, 0x12, 0xbb, 0xa6, 0x67, 0xf8, 0x6d, 0x6e,
    0x26, 0x31, 0x3b, 0x84, 0x36, 0x66, 0x61, 0x92, 0xba, 0x2d, 0x0d, 0xd5, 0x05, 0xbb, 0x02, 0x5b,
    0x7c, 0x50, 0x8e, 0xd2, 0xb5, 0xbc, 0x96, 0xef, 0x8c, 0xce, 0x82, 0x5f, 0x3e, 0x82, 0x7a, 0x44,
    0x30, 0xaf, 0xa8, 0xa7, 0x32, 0x8b, 0xb0, 0xe0, 0x4d, 0x0b, 0xbb, 0x86, 0xbd, 0x34, 0x94, 0xea,
    0xad, 0x14, 0x71, 0xa8, 0x30, 0x76, 0xdb, 0x9e, 0xe1, 0x3b, 0xa3, 0x7e, 0x50, 0x3b, 0x0f, 0xb6,
    0xce, 0x83, 0xc5, 0xd6, 0x39, 0x77, 0x2a, 0xfe, 0xb9, 0xc6, 0xfb, 0xaf, 0xe0, 0xfc, 0x78, 0x95,
    0x1d, 0x81, 0x9d, 0x6b, 0xd5, 0xc4, 0x68, 0x2a, 0x76, 0x09, 0x96, 0xda, 0x08, 0xd4, 0x51, 0xba,
    0xa3, 0xd3, 0x7f, 0x0d, 0x2e, 0x36, 0x02, 0xb9, 0xc6, 0x07, 0xe7, 0xd0, 0xf9, 0x3e, 0x62, 0x00,
    0xf6, 0x74, 0x36, 0xbe, 0x7f, 0x9c, 0xf4, 0x76, 0xd8, 0x2e, 0x58, 0x77, 0xb3, 0xe9, 0xa4, 0x67,
    0x54, 0xea, 0x65, 0xc6, 0x1f, 0x7a, 0xe6, 0xe0, 0x06, 0x9c, 0xdb, 0xfa, 0x4b, 0xc6, 0x44, 0x6b,
    0x36, 0x04, 0x5b, 0x20, 0x89, 0xb4, 0xda, 0x68, 0xb5, 0x95, 0xe3, 0x3f, 0x86, 0xf2, 0x06, 0x8b,
    0x6c, 0x9d, 0xf5, 0xe2, 0x2b, 0x00, 0x00, 0xff, 0xff, 0xba, 0x00, 0x86, 0x05, 0xd9, 0x01, 0x00,
    0x00,
}

裏面的struct實現了proto.Message接口。

驗證一下，我寫了個main.go方法，

驗證寫：

package main

import (
    pb "github.com/chenchi/procotolbuffers"
    "github.com/gogo/protobuf/proto"
    "io/ioutil"
    "log"
)

func main() {
    p := &pb.Person{
        Id:    1234,
        Name:  "chenchi",
        Email: "chenchi@example.com",
        Phones: []*pb.Person_PhoneNumber{
            {Number: "555-4321", Type: pb.Person_HOME},
        },
    }
    
    //寫
    out, err := proto.Marshal(p)
    if err != nil {
        log.Fatalln("Failed to encode address person:", err)
    }
    if err := ioutil.WriteFile("github.com/chenchi/procotolbuffers/main/out", out, 0644); err != nil {
        log.Fatalln("Failed to write address person:", err)
    }
    
}

執行完生成了out文件。成功！

驗證讀：

package main

import (
    "fmt"
    pb "github.com/chenchi/procotolbuffers"
    "github.com/gogo/protobuf/proto"
    "io/ioutil"
    "log"
)

func main() {
    
    //讀
    in, err := ioutil.ReadFile("github.com/chenchi/procotolbuffers/main/out")
    if err != nil {
        log.Fatalln("Error reading file:", err)
    }
    p2 := &pb.Person{}
    if err := proto.Unmarshal(in, p2); err != nil {
        log.Fatalln("Failed to parse address book:", err)
    }
    fmt.Println(p2)

}