k8s探針實現grpc健康檢查

這篇文章教你們如何利用k8s實現grpc健康檢查

原理跟github上的grpc-ecosystem/grpc-health-probe 同樣, 這裏是簡單實現

一. 配置Liveness和Readiness探針

kubelet 使用 liveness probe（存活探針）來肯定什麼時候重啓容器。例如，當應用程序處於運行狀態但沒法作進一步操做，liveness 探針將捕獲到 deadlock，重啓處於該狀態下的容器，使應用程序在存在 bug 的狀況下依然可以繼續運行下去。

kubelet 使用 readiness probe（就緒探針）來肯定容器是否已經就緒能夠接受流量。只有當 Pod 中的容器都處於就緒狀態時 kubelet 纔會認定該 Pod處於就緒狀態。該信號的做用是控制哪些 Pod應該做爲service的後端。若是 Pod 處於非就緒狀態，那麼它們將會被從 service 的 load balancer中移除。

點擊這裏看k8s官方文檔

1.1 k8s設置livenessProbe

livenessProbe:
  exec:
    command:
    - /root/rpc_check
    - -a
    - 127.0.0.1:19000
  initialDelaySeconds: 2
  periodSeconds: 2
複製代碼

配置成功以後,k8s會每2秒執行 /root/rpc_check -a 127.0.0.1:19000, 執行成功的話表明存活,不成功的話k8s會重啓pod

二. 建立容器

2.1 建立項目 $GOPATH/src/grpc-demo

grpc健康檢查客戶端

# $GOPATH/src/grpc-demo/cmd/check/main.go
package main
import(
	"os"
	"log"
	"time"
	"errors"
	"context"
	"path/filepath"
	"google.golang.org/grpc"
	cli "gopkg.in/urfave/cli.v1"
	pb "google.golang.org/grpc/health/grpc_health_v1"
)
const (
	VERSION  = "1.0.1"
	USAGE    = "grpc health check client"
)
var app *cli.App
func init(){
	app 	    = cli.NewApp()
	app.Name 	= filepath.Base(os.Args[0])
	app.Version = VERSION
	app.Usage 	= USAGE
	app.Flags = []cli.Flag{
		cli.StringFlag{Name: "address, a", Usage: "請求地址"},
		cli.StringFlag{Name: "service, s", Usage: "請求參數service", Value: "NULL"},
	}
	app.Action 	= func(ctx *cli.Context) error {
		a := ctx.GlobalString("address")
		s := ctx.GlobalString("service")
		if a == "" {
			log.Fatalln("Missing address parameter! see --help")
			return errors.New("Missing address parameter! see --help")
		}
		conn, err := grpc.Dial(a, grpc.WithInsecure())
		if err != nil {
			log.Fatalf("did not connect: %v", err)
			return err
		}
		defer conn.Close()
		f := pb.NewHealthClient(conn)
		c, cancel := context.WithTimeout(context.Background(), time.Second * 30)
		defer cancel()
		r, err := f.Check(c, &pb.HealthCheckRequest{
			Service: s,
		})
		if err != nil {
			log.Fatalf("could not greet: %v", err)
			return err
		}
		log.Println(r)
		return nil
	}
}
func main() {
	if err := app.Run(os.Args); err != nil {
		os.Exit(1)
	}
}
複製代碼

grpc健康檢查服務端

# $GOPATH/src/grpc-demo/cmd/server/main.go
package main
import (
    "os"
	"net"
	"log"
	"strconv"
	"syscall"
	"errors"
	"context"
	"os/signal"
	"path/filepath"
	"google.golang.org/grpc"
	
	"grpc-demo/app/health"
	cli "gopkg.in/urfave/cli.v1"
	pb "google.golang.org/grpc/health/grpc_health_v1"
)
const (
	VERSION = "1.0.1"
	USAGE   = "grpc health check server"
)
var app *cli.App
func init(){
	app 	    = cli.NewApp()
	app.Name 	= filepath.Base(os.Args[0])
	app.Version = VERSION
	app.Usage 	= USAGE
	app.Flags = []cli.Flag{
		cli.UintFlag{Name: "port, p", 	Usage: "端口"},
	}
	app.Action 	= func(ctx *cli.Context) error {
		p := ctx.GlobalUint("port")
		if p == 0 {
			log.Fatalf("Missing port!")
			return errors.New("Missing port!")
		}
		grpcServer := grpc.NewServer()
		lis, err := net.Listen("tcp", ":"+strconv.Itoa(int(p)))
		if err != nil {
			log.Fatalf("Failed to listen:%+v",err)
			return err
		}
		pb.RegisterHealthServer(grpcServer, health.New())
		go func() {
			sigs := make(chan os.Signal, 1)
			signal.Notify(sigs, syscall.SIGTERM, syscall.SIGINT)
			_ = <-sigs
			grpcServer.GracefulStop()
		}()	
		log.Printf("service started")
		if err := grpcServer.Serve(lis); err != nil {
			log.Fatalf("Failed to serve: %+v", err)
			return err
		}
		return nil
	}
}
func main() {
	if err := app.Run(os.Args); err != nil {
		os.Exit(1)
	}
}
複製代碼

健康檢查實現方法 health.go

# $GOPATH/src/grpc-demo/app/health.go
package health
import(
	"log"
	"context"
	pb "google.golang.org/grpc/health/grpc_health_v1"
)
type Health struct{}
func New() *Health {
	return &Health{}
}
func (h *Health) Check(ctx context.Context, in *pb.HealthCheckRequest)(*pb.HealthCheckResponse, error){
	log.Printf("checking............%s", in.Service)
	var s pb.HealthCheckResponse_ServingStatus = 1
	return &pb.HealthCheckResponse{
		Status : s,
	}, nil
}
func (h *Health) Watch(in *pb.HealthCheckRequest, w pb.Health_WatchServer)(error){
	log.Printf("watching............%s", in.Service)
	var s pb.HealthCheckResponse_ServingStatus = 1
	r := &pb.HealthCheckResponse{
		Status : s,
	}
	for {
		w.Send(r)
	}
	return nil
}
複製代碼

編繹

go build -o rpc_srv  $GOPATH/src/grpc-demo/cmd/server/*.go
go build -o rpc_check  $GOPATH/src/grpc-demo/cmd/check/*.go
複製代碼

Dockerfile

FROM ubuntu:16.04
ADD rpc_srv /root/rpc_srv
ADD rpc_check /root/rpc_check
RUN chmod +x /root/rpc_srv && chmod +x /root/rpc_check
EXPOSE 19000
CMD /root/rpc_srv -p 19000
複製代碼

總結

k8s實現grpc健康檢查的方法跟envoy很是像, 就是調用服務中實現的health_check方法, 只是envoy集成了調用, 而k8s要本身寫調用程序或使用grpc-health-probe. k8s+nginx+consul能夠組成一個比較成熟的grpc服務發現與服務治理方案