原文:medium.com/@amsokol.co…mysql
在此我並不打算重複它的內容。我但願經過一步步的引導讓你們學會如何去開發一個簡單的CRUD 「To Do List」的微服務經過使用gPRC以及HTTP/REST的後端接口。我會演示如何編寫測試用例以及加入中間件(請求ID以及日誌記錄與追蹤)在服務當中。最後甚至還會提供一些例子講述如何構建以及發佈咱們的微服務到Kubernetes上。git
整個教程會分爲4個部分:github
這句話有什麼意義?golang
「To Do List」微服務容許去管理「To Do」列表,ToDo項包括如下字段/屬性:sql
ToDo服務包含典型的增刪改查以及獲取所有項方法。數據庫
Part1完整代碼戳這裏編程
開始以前咱們先構建一個項目的骨架,這裏又一個很是優秀的Go項目的腳手架模板後端
我是用的是Windows 10 x64運行環境,可是我想你將我接下來的cmd指令轉換成MacOs/Linux BASH應該不是什麼大問題api
首先建立文件夾並初始化項目bash
mkdir go-grpc-http-rest-microservice-tutorial
cd go-grpc-http-rest-microservice-tutorial
go mod init github.com/<you>/go-grpc-http-rest-microservice-tutorial
複製代碼
在你的項目裏面建立文件目錄以下
mkdir -p api\proto\v1
複製代碼
這裏的v1就是咱們API的版本號
API版本化:經過將不一樣版本的API代碼放到不一樣文件夾裏面並以此命名是個人最佳實踐
下一步就是在剛建立的v1文件夾裏面建立todo-service.proto文件並加入ToDo服務的定義,咱們先從Create方法寫起:
syntax = "proto3";
package v1;
import "google/protobuf/timestamp.proto";
// Taks we have to do
message ToDo {
// Unique integer identifier of the todo task
int64 id = 1;
// Title of the task
string title = 2;
// Detail description of the todo task
string description = 3;
// Date and time to remind the todo task
google.protobuf.Timestamp reminder = 4;
}
// Request data to create new todo task
message CreateRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity to add
ToDo toDo = 2;
}
// Response that contains data for created todo task
message CreateResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// ID of created task
int64 id = 2;
}
// Service to manage list of todo tasks
service ToDoService {
// Create new todo task
rpc Create(CreateRequest) returns (CreateResponse);
}
複製代碼
戳我查看Proto的編寫語法
如你所見,API定義是跟編程語言,通信協議以及網絡傳輸無關的,這也是protobuf的一個重要標誌
爲了能編譯proto文件咱們須要安裝一些工具和依賴
go get -u github.com/golang/protobuf/protoc-gen-go
複製代碼
# Windows:
.\third_party\protoc-gen.cmd
# MasOS/Linux:
./third_party/protoc-gen.sh
複製代碼
運行之後會建立一個名爲todo-service.pb.go的文件在pkg/model/v1下
接下來讓咱們將剩下的方法駕到ToDo服務當中並編譯
syntax = "proto3";
package v1;
import "google/protobuf/timestamp.proto";
// Taks we have to do
message ToDo {
// Unique integer identifier of the todo task
int64 id = 1;
// Title of the task
string title = 2;
// Detail description of the todo task
string description = 3;
// Date and time to remind the todo task
google.protobuf.Timestamp reminder = 4;
}
// Request data to create new todo task
message CreateRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity to add
ToDo toDo = 2;
}
// Contains data of created todo task
message CreateResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// ID of created task
int64 id = 2;
}
// Request data to read todo task
message ReadRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Unique integer identifier of the todo task
int64 id = 2;
}
// Contains todo task data specified in by ID request
message ReadResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity read by ID
ToDo toDo = 2;
}
// Request data to update todo task
message UpdateRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Task entity to update
ToDo toDo = 2;
}
// Contains status of update operation
message UpdateResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Contains number of entities have beed updated
// Equals 1 in case of succesfull update
int64 updated = 2;
}
// Request data to delete todo task
message DeleteRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Unique integer identifier of the todo task to delete
int64 id = 2;
}
// Contains status of delete operation
message DeleteResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// Contains number of entities have beed deleted
// Equals 1 in case of succesfull delete
int64 deleted = 2;
}
// Request data to read all todo task
message ReadAllRequest{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
}
// Contains list of all todo tasks
message ReadAllResponse{
// API versioning: it is my best practice to specify version explicitly
string api = 1;
// List of all todo tasks
repeated ToDo toDos = 2;
}
// Service to manage list of todo tasks
service ToDoService {
// Create new todo task
rpc Create(CreateRequest) returns (CreateResponse);
// Read todo task
rpc Read(ReadRequest) returns (ReadResponse);
// Update todo task
rpc Update(UpdateRequest) returns (UpdateResponse);
// Delete todo task
rpc Delete(DeleteRequest) returns (DeleteResponse);
// Read all todo tasks
rpc ReadAll(ReadAllRequest) returns (ReadAllResponse);
}
複製代碼
再一次運行如下命令來更新Go的代碼
# Windows:
.\third_party\protoc-gen.cmd
# MasOS/Linux:
./third_party/protoc-gen.sh
複製代碼
到此爲止,API的定義就完成了
我是使用Google Cloud上的MySQL做爲數據庫來持久化作存儲的。你可使用其餘你喜歡的SQL數據庫。
建立ToDo table的MySQL腳本
CREATE TABLE `ToDo` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT,
`Title` varchar(200) DEFAULT NULL,
`Description` varchar(1024) DEFAULT NULL,
`Reminder` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `ID_UNIQUE` (`ID`)
);
複製代碼
我會跳過如何安裝配置SQL數據庫以及建立數據表的步驟
建立文件pkg/service/v1/todo-service.go以及下面的內容
package v1
import (
"context"
"database/sql"
"fmt"
"time"
"github.com/golang/protobuf/ptypes"
"google.golang.org/grpc/codes"
"google.golang.org/grpc/status"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/api/v1"
)
const (
// apiVersion is version of API is provided by server
apiVersion = "v1"
)
// toDoServiceServer is implementation of v1.ToDoServiceServer proto interface
type toDoServiceServer struct {
db *sql.DB
}
// NewToDoServiceServer creates ToDo service
func NewToDoServiceServer(db *sql.DB) v1.ToDoServiceServer {
return &toDoServiceServer{db: db}
}
// checkAPI checks if the API version requested by client is supported by server
func (s *toDoServiceServer) checkAPI(api string) error {
// API version is "" means use current version of the service
if len(api) > 0 {
if apiVersion != api {
return status.Errorf(codes.Unimplemented,
"unsupported API version: service implements API version '%s', but asked for '%s'", apiVersion, api)
}
}
return nil
}
// connect returns SQL database connection from the pool
func (s *toDoServiceServer) connect(ctx context.Context) (*sql.Conn, error) {
c, err := s.db.Conn(ctx)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to connect to database-> "+err.Error())
}
return c, nil
}
// Create new todo task
func (s *toDoServiceServer) Create(ctx context.Context, req *v1.CreateRequest) (*v1.CreateResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
reminder, err := ptypes.Timestamp(req.ToDo.Reminder)
if err != nil {
return nil, status.Error(codes.InvalidArgument, "reminder field has invalid format-> "+err.Error())
}
// insert ToDo entity data
res, err := c.ExecContext(ctx, "INSERT INTO ToDo(`Title`, `Description`, `Reminder`) VALUES(?, ?, ?)",
req.ToDo.Title, req.ToDo.Description, reminder)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to insert into ToDo-> "+err.Error())
}
// get ID of creates ToDo
id, err := res.LastInsertId()
if err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve id for created ToDo-> "+err.Error())
}
return &v1.CreateResponse{
Api: apiVersion,
Id: id,
}, nil
}
// Read todo task
func (s *toDoServiceServer) Read(ctx context.Context, req *v1.ReadRequest) (*v1.ReadResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
// query ToDo by ID
rows, err := c.QueryContext(ctx, "SELECT `ID`, `Title`, `Description`, `Reminder` FROM ToDo WHERE `ID`=?",
req.Id)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to select from ToDo-> "+err.Error())
}
defer rows.Close()
if !rows.Next() {
if err := rows.Err(); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve data from ToDo-> "+err.Error())
}
return nil, status.Error(codes.NotFound, fmt.Sprintf("ToDo with ID='%d' is not found",
req.Id))
}
// get ToDo data
var td v1.ToDo
var reminder time.Time
if err := rows.Scan(&td.Id, &td.Title, &td.Description, &reminder); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve field values from ToDo row-> "+err.Error())
}
td.Reminder, err = ptypes.TimestampProto(reminder)
if err != nil {
return nil, status.Error(codes.Unknown, "reminder field has invalid format-> "+err.Error())
}
if rows.Next() {
return nil, status.Error(codes.Unknown, fmt.Sprintf("found multiple ToDo rows with ID='%d'",
req.Id))
}
return &v1.ReadResponse{
Api: apiVersion,
ToDo: &td,
}, nil
}
// Update todo task
func (s *toDoServiceServer) Update(ctx context.Context, req *v1.UpdateRequest) (*v1.UpdateResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
reminder, err := ptypes.Timestamp(req.ToDo.Reminder)
if err != nil {
return nil, status.Error(codes.InvalidArgument, "reminder field has invalid format-> "+err.Error())
}
// update ToDo
res, err := c.ExecContext(ctx, "UPDATE ToDo SET `Title`=?, `Description`=?, `Reminder`=? WHERE `ID`=?",
req.ToDo.Title, req.ToDo.Description, reminder, req.ToDo.Id)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to update ToDo-> "+err.Error())
}
rows, err := res.RowsAffected()
if err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve rows affected value-> "+err.Error())
}
if rows == 0 {
return nil, status.Error(codes.NotFound, fmt.Sprintf("ToDo with ID='%d' is not found",
req.ToDo.Id))
}
return &v1.UpdateResponse{
Api: apiVersion,
Updated: rows,
}, nil
}
// Delete todo task
func (s *toDoServiceServer) Delete(ctx context.Context, req *v1.DeleteRequest) (*v1.DeleteResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
// delete ToDo
res, err := c.ExecContext(ctx, "DELETE FROM ToDo WHERE `ID`=?", req.Id)
if err != nil {
return nil, status.Error(codes.Unknown, "failed to delete ToDo-> "+err.Error())
}
rows, err := res.RowsAffected()
if err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve rows affected value-> "+err.Error())
}
if rows == 0 {
return nil, status.Error(codes.NotFound, fmt.Sprintf("ToDo with ID='%d' is not found",
req.Id))
}
return &v1.DeleteResponse{
Api: apiVersion,
Deleted: rows,
}, nil
}
// Read all todo tasks
func (s *toDoServiceServer) ReadAll(ctx context.Context, req *v1.ReadAllRequest) (*v1.ReadAllResponse, error) {
// check if the API version requested by client is supported by server
if err := s.checkAPI(req.Api); err != nil {
return nil, err
}
// get SQL connection from pool
c, err := s.connect(ctx)
if err != nil {
return nil, err
}
defer c.Close()
// get ToDo list
rows, err := c.QueryContext(ctx, "SELECT `ID`, `Title`, `Description`, `Reminder` FROM ToDo")
if err != nil {
return nil, status.Error(codes.Unknown, "failed to select from ToDo-> "+err.Error())
}
defer rows.Close()
var reminder time.Time
list := []*v1.ToDo{}
for rows.Next() {
td := new(v1.ToDo)
if err := rows.Scan(&td.Id, &td.Title, &td.Description, &reminder); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve field values from ToDo row-> "+err.Error())
}
td.Reminder, err = ptypes.TimestampProto(reminder)
if err != nil {
return nil, status.Error(codes.Unknown, "reminder field has invalid format-> "+err.Error())
}
list = append(list, td)
}
if err := rows.Err(); err != nil {
return nil, status.Error(codes.Unknown, "failed to retrieve data from ToDo-> "+err.Error())
}
return &v1.ReadAllResponse{
Api: apiVersion,
ToDos: list,
}, nil
}
複製代碼
爲API邏輯實現建立文件pkg/service/v1/todo-service.go 最後文件目錄以下
無論咱們開發什麼都應該編寫測試用例。這是強制遵照的規定。
這裏有一個很棒的模擬庫,用於測試SQL數據庫的交互go-sqlmock.我將會使用它爲咱們的ToDo服務編寫測試用例。
將這個文件放到pkg/service/v1目錄下,當前項目文件結構以下
建立文件pkg/protocol/grpc/server.go並寫入
package grpc
import (
"context"
"log"
"net"
"os"
"os/signal"
"google.golang.org/grpc"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/api/v1"
)
// RunServer runs gRPC service to publish ToDo service
func RunServer(ctx context.Context, v1API v1.ToDoServiceServer, port string) error {
listen, err := net.Listen("tcp", ":"+port)
if err != nil {
return err
}
// register service
server := grpc.NewServer()
v1.RegisterToDoServiceServer(server, v1API)
// graceful shutdown
c := make(chan os.Signal, 1)
signal.Notify(c, os.Interrupt)
go func() {
for range c {
// sig is a ^C, handle it
log.Println("shutting down gRPC server...")
server.GracefulStop()
<-ctx.Done()
}
}()
// start gRPC server
log.Println("starting gRPC server...")
return server.Serve(listen)
}
複製代碼
RunServer函數負責註冊ToDo服務以及啓動gRPC服務
你須要給gPRC服務配置TLS,查看示例學習如何配置
接下來建立pkg/cmd/server/server.go以及對應內容
package cmd
import (
"context"
"database/sql"
"flag"
"fmt"
// mysql driver
_ "github.com/go-sql-driver/mysql"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/protocol/grpc"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/service/v1"
)
// Config is configuration for Server
type Config struct {
// gRPC server start parameters section
// gRPC is TCP port to listen by gRPC server
GRPCPort string
// DB Datastore parameters section
// DatastoreDBHost is host of database
DatastoreDBHost string
// DatastoreDBUser is username to connect to database
DatastoreDBUser string
// DatastoreDBPassword password to connect to database
DatastoreDBPassword string
// DatastoreDBSchema is schema of database
DatastoreDBSchema string
}
// RunServer runs gRPC server and HTTP gateway
func RunServer() error {
ctx := context.Background()
// get configuration
var cfg Config
flag.StringVar(&cfg.GRPCPort, "grpc-port", "", "gRPC port to bind")
flag.StringVar(&cfg.DatastoreDBHost, "db-host", "", "Database host")
flag.StringVar(&cfg.DatastoreDBUser, "db-user", "", "Database user")
flag.StringVar(&cfg.DatastoreDBPassword, "db-password", "", "Database password")
flag.StringVar(&cfg.DatastoreDBSchema, "db-schema", "", "Database schema")
flag.Parse()
if len(cfg.GRPCPort) == 0 {
return fmt.Errorf("invalid TCP port for gRPC server: '%s'", cfg.GRPCPort)
}
// add MySQL driver specific parameter to parse date/time
// Drop it for another database
param := "parseTime=true"
dsn := fmt.Sprintf("%s:%s@tcp(%s)/%s?%s",
cfg.DatastoreDBUser,
cfg.DatastoreDBPassword,
cfg.DatastoreDBHost,
cfg.DatastoreDBSchema,
param)
db, err := sql.Open("mysql", dsn)
if err != nil {
return fmt.Errorf("failed to open database: %v", err)
}
defer db.Close()
v1API := v1.NewToDoServiceServer(db)
return grpc.RunServer(ctx, v1API, cfg.GRPCPort)
}
複製代碼
RunServer函數負責讀取命令行輸入的參數,建立數據庫鏈接,建立ToDo服務實例以及調用以前gPRC服務中的RunServer函數
最後建立如下文件cmd/server/main.go以及內容
package main
import (
"fmt"
"os"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/cmd"
)
func main() {
if err := cmd.RunServer(); err != nil {
fmt.Fprintf(os.Stderr, "%v\n", err)
os.Exit(1)
}
}
複製代碼
以上就是服務端全部的代碼了,當前項目目錄以下
建立文件cmd/client-grpc/main.go以及如下內容
package main
import (
"context"
"flag"
"log"
"time"
"github.com/golang/protobuf/ptypes"
"google.golang.org/grpc"
"github.com/amsokol/go-grpc-http-rest-microservice-tutorial/pkg/api/v1"
)
const (
// apiVersion is version of API is provided by server
apiVersion = "v1"
)
func main() {
// get configuration
address := flag.String("server", "", "gRPC server in format host:port")
flag.Parse()
// Set up a connection to the server.
conn, err := grpc.Dial(*address, grpc.WithInsecure())
if err != nil {
log.Fatalf("did not connect: %v", err)
}
defer conn.Close()
c := v1.NewToDoServiceClient(conn)
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
t := time.Now().In(time.UTC)
reminder, _ := ptypes.TimestampProto(t)
pfx := t.Format(time.RFC3339Nano)
// Call Create
req1 := v1.CreateRequest{
Api: apiVersion,
ToDo: &v1.ToDo{
Title: "title (" + pfx + ")",
Description: "description (" + pfx + ")",
Reminder: reminder,
},
}
res1, err := c.Create(ctx, &req1)
if err != nil {
log.Fatalf("Create failed: %v", err)
}
log.Printf("Create result: <%+v>\n\n", res1)
id := res1.Id
// Read
req2 := v1.ReadRequest{
Api: apiVersion,
Id: id,
}
res2, err := c.Read(ctx, &req2)
if err != nil {
log.Fatalf("Read failed: %v", err)
}
log.Printf("Read result: <%+v>\n\n", res2)
// Update
req3 := v1.UpdateRequest{
Api: apiVersion,
ToDo: &v1.ToDo{
Id: res2.ToDo.Id,
Title: res2.ToDo.Title,
Description: res2.ToDo.Description + " + updated",
Reminder: res2.ToDo.Reminder,
},
}
res3, err := c.Update(ctx, &req3)
if err != nil {
log.Fatalf("Update failed: %v", err)
}
log.Printf("Update result: <%+v>\n\n", res3)
// Call ReadAll
req4 := v1.ReadAllRequest{
Api: apiVersion,
}
res4, err := c.ReadAll(ctx, &req4)
if err != nil {
log.Fatalf("ReadAll failed: %v", err)
}
log.Printf("ReadAll result: <%+v>\n\n", res4)
// Delete
req5 := v1.DeleteRequest{
Api: apiVersion,
Id: id,
}
res5, err := c.Delete(ctx, &req5)
if err != nil {
log.Fatalf("Delete failed: %v", err)
}
log.Printf("Delete result: <%+v>\n\n", res5)
}
複製代碼
以上就是客戶端全部代碼,當前項目目錄以下
最後一步驟是確保gPRC服務能跑起來
開啓一個終端build以及run gRPC服務(將下面數據庫鏈接的參數替代成你本身的數據庫配置)
cd cmd/server
go build .
server.exe -grpc-port=9090 -db-host=<HOST>:3306 -db-user=<USER> -db-password=<PASSWORD> -db-schema=<SCHEMA>
複製代碼
若是能看到
2018/09/09 08:02:16 starting gRPC server...
複製代碼
證實咱們的服務已經被啓動起來了
打開另外一個終端build和run gRPC客戶端
cd cmd/client-grpc
go build .
client-grpc.exe -server=localhost:9090
複製代碼
若是能看到如下信息:
2018/09/09 09:16:01 Create result: <api:"v1" id:13 >
2018/09/09 09:16:01 Read result: <api:"v1" toDo:<id:13 title:"title (2018-09-09T06:16:01.5755011Z)" description:"description (2018-09-09T06:16:01.5755011Z)" reminder:<seconds:1536473762 > > >
2018/09/09 09:16:01 Update result: <api:"v1" updated:1 >
2018/09/09 09:16:01 ReadAll result: <api:"v1" toDos:<id:9 title:"title (2018-09-09T04:45:16.3693282Z)" description:"description (2018-09-09T04:45:16.3693282Z)" reminder:<seconds:1536468316 > > toDos:<id:10 title:"title (2018-09-09T04:46:00.7490565Z)" description:"description (2018-09-09T04:46:00.7490565Z)" reminder:<seconds:1536468362 > > toDos:<id:13 title:"title (2018-09-09T06:16:01.5755011Z)" description:"description (2018-09-09T06:16:01.5755011Z) + updated" reminder:<seconds:1536473762 > > >
2018/09/09 09:16:01 Delete result: <api:"v1" deleted:1 >
複製代碼
全部東西都正常運做了!
以上就是Part1的所有內容了,咱們成功構建了gRPC的客戶端以及服務端
Part1的源代碼在此處
接下來 Part2是講述如何在咱們本章創建的gRPC服務上增長HTTP/REST接口,敬請期待。
感謝收看!🙏🙏🙏🙏