go語言中的網絡編程主要經過net包實現,net包提供了網絡I/O接口,包括HTTP、TCP/IP、UDP、域名解析和Unix域socket等。和大多數語言同樣go可使用幾行代碼即可以啓動一個服務器,可是得益於goroutine的配合go實現的服務器擁有強大併發處理能力。web
Socket又稱"套接字",應用程序一般經過"套接字"向網絡發出請求或者應答網絡請求。編程
socket本質上就是在2臺網絡互通的電腦之間,架設一個通道,兩臺電腦經過這個通道來實現數據的互相傳遞。 咱們知道網絡 通訊 都 是基於 ip+port 方能定位到目標的具體機器上的具體服務,操做系統有0-65535個端口,每一個端口均可以獨立對外提供服務,若是 把一個公司比作一臺電腦 ,那公司的總機號碼就至關於ip地址, 每一個員工的分機號就至關於端口, 你想找公司某我的,必須 先打電話到總機,而後再轉分機 。api
go中socket編程實現起來很是方便,下面是處理流程服務器
服務器端:cookie
客戶端:網絡
服務端示例:併發
package main import ( "fmt" "net" ) func handle(conn net.Conn) { //處理鏈接方法 defer conn.Close() //關閉鏈接 for{ buf := make([]byte,100) n,err := conn.Read(buf) //讀取客戶端數據 if err!=nil { fmt.Println(err) return } fmt.Printf("read data size %d msg:%s", n, string(buf[0:n])) msg := []byte("hello,world\n") conn.Write(msg) //發送數據 } } func main() { fmt.Println("start server....") listen,err := net.Listen("tcp","0.0.0.0:3000") //建立監聽 if err != nil{ fmt.Println("listen failed! msg :" ,err) return } for{ conn,errs := listen.Accept() //接受客戶端鏈接 if errs != nil{ fmt.Println("accept failed") continue } go handle(conn) //處理鏈接 } }
客戶端示例:app
package main import ( "bufio" "fmt" "net" "os" "strings" ) func main() { conn, err := net.Dial("tcp", "127.0.0.1:3000") if err != nil { fmt.Println("err dialing:", err.Error()) return } defer conn.Close() inputReader := bufio.NewReader(os.Stdin) for { str, _ := inputReader.ReadString('\n') data := strings.Trim(str, "\n") if data == "quit" { //輸入quit退出 return } _, err := conn.Write([]byte(data)) //發送數據 if err != nil { fmt.Println("send data error:", err) return } buf := make([]byte,512) n,err := conn.Read(buf) //讀取服務端端數據 fmt.Println("from server:", string(buf[:n])) } }
conn示例還提供其餘方法:curl
type Conn interface { // Read reads data from the connection. // Read can be made to time out and return an Error with Timeout() == true // after a fixed time limit; see SetDeadline and SetReadDeadline. Read(b []byte) (n int, err error) //讀取鏈接中數據 // Write writes data to the connection. // Write can be made to time out and return an Error with Timeout() == true // after a fixed time limit; see SetDeadline and SetWriteDeadline. Write(b []byte) (n int, err error) //發送數據 // Close closes the connection. // Any blocked Read or Write operations will be unblocked and return errors. Close() error //關閉連接 // LocalAddr returns the local network address. LocalAddr() Addr //返回本地鏈接地址 // RemoteAddr returns the remote network address. RemoteAddr() Addr //返回遠程鏈接的地址 // SetDeadline sets the read and write deadlines associated // with the connection. It is equivalent to calling both // SetReadDeadline and SetWriteDeadline. // // A deadline is an absolute time after which I/O operations // fail with a timeout (see type Error) instead of // blocking. The deadline applies to all future and pending // I/O, not just the immediately following call to Read or // Write. After a deadline has been exceeded, the connection // can be refreshed by setting a deadline in the future. // // An idle timeout can be implemented by repeatedly extending // the deadline after successful Read or Write calls. // // A zero value for t means I/O operations will not time out. SetDeadline(t time.Time) error //設置連接讀取或者寫超時時間 // SetReadDeadline sets the deadline for future Read calls // and any currently-blocked Read call. // A zero value for t means Read will not time out. SetReadDeadline(t time.Time) error //單獨設置讀取超時時間 // SetWriteDeadline sets the deadline for future Write calls // and any currently-blocked Write call. // Even if write times out, it may return n > 0, indicating that // some of the data was successfully written. // A zero value for t means Write will not time out. SetWriteDeadline(t time.Time) error//單獨設置寫超時時間 }
網絡發展,不少網絡應用都是構建再 HTTP 服務基礎之上。HTTP 協議從誕生到如今,發展從1.0,1.1到2.0也不斷再進步。除去細節,理解 HTTP 構建的網絡應用只要關注兩個端---客戶端(clinet)和服務端(server),兩個端的交互來自 clinet 的 request,以及server端的response。所謂的http服務器,主要在於如何接受 clinet 的 request,並向client返回response。接收request的過程當中,最重要的莫過於路由(router
),即實現一個Multiplexer
器。Go中既可使用內置的mutilplexer --- DefautServeMux
,也能夠自定義。Multiplexer路由的目的就是爲了找處處理器函數(handler
),後者將對request進行處理,同時構建response。socket
最後簡化的請求處理流程爲:
Clinet -> Requests -> [Multiplexer(router) -> handler -> Response -> Clinet
所以,理解go中的http服務,最重要就是要理解Multiplexer和handler,Golang中的Multiplexer基於ServeMux
結構,同時也實現了Handler
接口。
對象說明:
func(w http.ResponseWriter, r *http.Requests)
簽名的函數HandlerFunc
結構包裝的handler函數
,它實現了ServeHTTP接口方法的函數。調用handler處理器的ServeHTTP方法時,即調用handler函數自己。handler處理器和handler對象的差異在於,一個是函數,另一個是結構,它們都有實現了ServeHTTP方法。不少狀況下它們的功能相似,下文就使用統稱爲handler。
Golang沒有繼承,類多態的方式能夠經過接口實現。所謂接口則是定義聲明瞭函數簽名,任何結構只要實現了與接口函數簽名相同的方法,就等同於實現了接口。go的http服務都是基於handler進行處理。
type Handler interface { ServeHTTP(ResponseWriter, *Request) }
任何結構體,只要實現了ServeHTTP方法,這個結構就能夠稱之爲handler對象。ServeMux會使用handler並調用其ServeHTTP方法處理請求並返回響應。
源碼部分:
type ServeMux struct { mu sync.RWMutex m map[string]muxEntry hosts bool } type muxEntry struct { explicit bool h Handler pattern string }
ServeMux結構中最重要的字段爲m
,這是一個map,key是一些url模式,value是一個muxEntry結構,後者裏定義存儲了具體的url模式和handler。固然,所謂的ServeMux也實現了ServeHTTP接口,也算是一個handler,不過ServeMux的ServeHTTP方法不是用來處理request和respone,而是用來找到路由註冊的handler,後面再作解釋。
除了ServeMux和Handler,還有一個結構Server須要瞭解。從http.ListenAndServe
的源碼能夠看出,它建立了一個server對象,並調用server對象的ListenAndServe方法:
func ListenAndServe(addr string, handler Handler) error { server := &Server{Addr: addr, Handler: handler} return server.ListenAndServe() }
查看server的結構以下:
type Server struct { Addr string Handler Handler ReadTimeout time.Duration WriteTimeout time.Duration TLSConfig *tls.Config MaxHeaderBytes int TLSNextProto map[string]func(*Server, *tls.Conn, Handler) ConnState func(net.Conn, ConnState) ErrorLog *log.Logger disableKeepAlives int32 nextProtoOnce sync.Once nextProtoErr error }
server結構存儲了服務器處理請求常見的字段。其中Handler字段也保留Handler接口。若是Server接口沒有提供Handler結構對象,那麼會使用DefautServeMux作multiplexer,後面再作分析。
建立一個http服務,大體須要經歷兩個過程,首先須要註冊路由,即提供url模式和handler函數的映射,其次就是實例化一個server對象,並開啓對客戶端的監聽。
http.HandleFunc("/", indexHandler) http.ListenAndServe("127.0.0.1:8000", nil) 或 server := &Server{Addr: addr, Handler: handler} server.ListenAndServe()
示例:
package main import ( "fmt" "net/http" ) func Hello(w http.ResponseWriter, r *http.Request) { fmt.Println("Hello World.") fmt.Fprintf(w, "Hello World.\n") } func main() { http.HandleFunc("/", Hello) err := http.ListenAndServe("0.0.0.0:6000", nil) if err != nil { fmt.Println("http listen failed.") } } //curl http://127.0.0.1:6000 // 結果:Hello World
net/http包暴露的註冊路由的api很簡單,http.HandleFunc選取了DefaultServeMux做爲multiplexer:
func HandleFunc(pattern string, handler func(ResponseWriter, *Request)) { DefaultServeMux.HandleFunc(pattern, handler) }
DefaultServeMux是ServeMux的一個實例。固然http包也提供了NewServeMux方法建立一個ServeMux實例,默認則建立一個DefaultServeMux:
// NewServeMux allocates and returns a new ServeMux. func NewServeMux() *ServeMux { return new(ServeMux) } // DefaultServeMux is the default ServeMux used by Serve. var DefaultServeMux = &defaultServeMux var defaultServeMux ServeMux
DefaultServeMux的HandleFunc(pattern, handler)方法實際是定義在ServeMux下的:
// HandleFunc registers the handler function for the given pattern. func (mux *ServeMux) HandleFunc(pattern string, handler func(ResponseWriter, *Request)) { mux.Handle(pattern, HandlerFunc(handler)) }
HandlerFunc是一個函數類型。同時實現了Handler接口的ServeHTTP方法。使用HandlerFunc類型包裝一下路由定義的indexHandler函數,其目的就是爲了讓這個函數也實現ServeHTTP方法,即轉變成一個handler處理器(函數)。
type HandlerFunc func(ResponseWriter, *Request) // ServeHTTP calls f(w, r). func (f HandlerFunc) ServeHTTP(w ResponseWriter, r *Request) { f(w, r) }
咱們最開始寫的例子中
http.HandleFunc("/",Indexhandler)
這樣 IndexHandler 函數也有了ServeHTTP方法。ServeMux的Handle方法,將會對pattern和handler函數作一個map映射:
func ListenAndServe(addr string, handler Handler) error { server := &Server{Addr: addr, Handler: handler} return server.ListenAndServe() } // ListenAndServe listens on the TCP network address srv.Addr and then // calls Serve to handle requests on incoming connections. // Accepted connections are configured to enable TCP keep-alives. // If srv.Addr is blank, ":http" is used. // ListenAndServe always returns a non-nil error. func (srv *Server) ListenAndServe() error { addr := srv.Addr if addr == "" { addr = ":http" } ln, err := net.Listen("tcp", addr) if err != nil { return err } return srv.Serve(tcpKeepAliveListener{ln.(*net.TCPListener)}) }
Server的ListenAndServe方法中,會初始化監聽地址Addr,同時調用Listen方法設置監聽。最後將監聽的TCP對象傳入Serve方法:
// Serve accepts incoming connections on the Listener l, creating a // new service goroutine for each. The service goroutines read requests and // then call srv.Handler to reply to them. // // For HTTP/2 support, srv.TLSConfig should be initialized to the // provided listener's TLS Config before calling Serve. If // srv.TLSConfig is non-nil and doesn't include the string "h2" in // Config.NextProtos, HTTP/2 support is not enabled. // // Serve always returns a non-nil error. After Shutdown or Close, the // returned error is ErrServerClosed. func (srv *Server) Serve(l net.Listener) error { defer l.Close() if fn := testHookServerServe; fn != nil { fn(srv, l) } var tempDelay time.Duration // how long to sleep on accept failure if err := srv.setupHTTP2_Serve(); err != nil { return err } srv.trackListener(l, true) defer srv.trackListener(l, false) baseCtx := context.Background() // base is always background, per Issue 16220 ctx := context.WithValue(baseCtx, ServerContextKey, srv) for { rw, e := l.Accept() if e != nil { select { case <-srv.getDoneChan(): return ErrServerClosed default: } if ne, ok := e.(net.Error); ok && ne.Temporary() { if tempDelay == 0 { tempDelay = 5 * time.Millisecond } else { tempDelay *= 2 } if max := 1 * time.Second; tempDelay > max { tempDelay = max } srv.logf("http: Accept error: %v; retrying in %v", e, tempDelay) time.Sleep(tempDelay) continue } return e } tempDelay = 0 c := srv.newConn(rw) c.setState(c.rwc, StateNew) // before Serve can return go c.serve(ctx) } }
監聽開啓以後,一旦客戶端請求到底,go就開啓一個協程處理請求,主要邏輯都在serve方法之中。
serve方法比較長,其主要職能就是,建立一個上下文對象,而後調用Listener的Accept方法用來 獲取鏈接數據並使用newConn方法建立鏈接對象。最後使用goroutine協程的方式處理鏈接請求。由於每個鏈接都開起了一個協程,請求的上下文都不一樣,同時又保證了go的高併發。serve也是一個長長的方法:
// Serve a new connection. func (c *conn) serve(ctx context.Context) { c.remoteAddr = c.rwc.RemoteAddr().String() ctx = context.WithValue(ctx, LocalAddrContextKey, c.rwc.LocalAddr()) defer func() { if err := recover(); err != nil && err != ErrAbortHandler { const size = 64 << 10 buf := make([]byte, size) buf = buf[:runtime.Stack(buf, false)] c.server.logf("http: panic serving %v: %v\n%s", c.remoteAddr, err, buf) } if !c.hijacked() { c.close() c.setState(c.rwc, StateClosed) } }() if tlsConn, ok := c.rwc.(*tls.Conn); ok { if d := c.server.ReadTimeout; d != 0 { c.rwc.SetReadDeadline(time.Now().Add(d)) } if d := c.server.WriteTimeout; d != 0 { c.rwc.SetWriteDeadline(time.Now().Add(d)) } if err := tlsConn.Handshake(); err != nil { c.server.logf("http: TLS handshake error from %s: %v", c.rwc.RemoteAddr(), err) return } c.tlsState = new(tls.ConnectionState) *c.tlsState = tlsConn.ConnectionState() if proto := c.tlsState.NegotiatedProtocol; validNPN(proto) { if fn := c.server.TLSNextProto[proto]; fn != nil { h := initNPNRequest{tlsConn, serverHandler{c.server}} fn(c.server, tlsConn, h) } return } } // HTTP/1.x from here on. ctx, cancelCtx := context.WithCancel(ctx) c.cancelCtx = cancelCtx defer cancelCtx() c.r = &connReader{conn: c} c.bufr = newBufioReader(c.r) c.bufw = newBufioWriterSize(checkConnErrorWriter{c}, 4<<10) for { w, err := c.readRequest(ctx) if c.r.remain != c.server.initialReadLimitSize() { // If we read any bytes off the wire, we're active. c.setState(c.rwc, StateActive) } if err != nil { const errorHeaders = "\r\nContent-Type: text/plain; charset=utf-8\r\nConnection: close\r\n\r\n" if err == errTooLarge { // Their HTTP client may or may not be // able to read this if we're // responding to them and hanging up // while they're still writing their // request. Undefined behavior. const publicErr = "431 Request Header Fields Too Large" fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr) c.closeWriteAndWait() return } if isCommonNetReadError(err) { return // don't reply } publicErr := "400 Bad Request" if v, ok := err.(badRequestError); ok { publicErr = publicErr + ": " + string(v) } fmt.Fprintf(c.rwc, "HTTP/1.1 "+publicErr+errorHeaders+publicErr) return } // Expect 100 Continue support req := w.req if req.expectsContinue() { if req.ProtoAtLeast(1, 1) && req.ContentLength != 0 { // Wrap the Body reader with one that replies on the connection req.Body = &expectContinueReader{readCloser: req.Body, resp: w} } } else if req.Header.get("Expect") != "" { w.sendExpectationFailed() return } c.curReq.Store(w) if requestBodyRemains(req.Body) { registerOnHitEOF(req.Body, w.conn.r.startBackgroundRead) } else { if w.conn.bufr.Buffered() > 0 { w.conn.r.closeNotifyFromPipelinedRequest() } w.conn.r.startBackgroundRead() } // HTTP cannot have multiple simultaneous active requests.[*] // Until the server replies to this request, it can't read another, // so we might as well run the handler in this goroutine. // [*] Not strictly true: HTTP pipelining. We could let them all process // in parallel even if their responses need to be serialized. // But we're not going to implement HTTP pipelining because it // was never deployed in the wild and the answer is HTTP/2. serverHandler{c.server}.ServeHTTP(w, w.req) w.cancelCtx() if c.hijacked() { return } w.finishRequest() if !w.shouldReuseConnection() { if w.requestBodyLimitHit || w.closedRequestBodyEarly() { c.closeWriteAndWait() } return } c.setState(c.rwc, StateIdle) c.curReq.Store((*response)(nil)) if !w.conn.server.doKeepAlives() { // We're in shutdown mode. We might've replied // to the user without "Connection: close" and // they might think they can send another // request, but such is life with HTTP/1.1. return } if d := c.server.idleTimeout(); d != 0 { c.rwc.SetReadDeadline(time.Now().Add(d)) if _, err := c.bufr.Peek(4); err != nil { return } } c.rwc.SetReadDeadline(time.Time{}) } }
使用defer定義了函數退出時,鏈接關閉相關的處理。而後就是讀取鏈接的網絡數據,並處理讀取完畢時候的狀態。接下來就是調用serverHandler{c.server}.ServeHTTP(w, w.req)
方法處理請求了。最後就是請求處理完畢的邏輯。serverHandler是一個重要的結構,它近有一個字段,即Server結構,同時它也實現了Handler接口方法ServeHTTP,並在該接口方法中作了一個重要的事情,初始化multiplexer路由多路複用器。若是server對象沒有指定Handler,則使用默認的DefaultServeMux做爲路由Multiplexer。並調用初始化Handler的ServeHTTP方法。
// serverHandler delegates to either the server's Handler or // DefaultServeMux and also handles "OPTIONS *" requests. type serverHandler struct { srv *Server } func (sh serverHandler) ServeHTTP(rw ResponseWriter, req *Request) { handler := sh.srv.Handler if handler == nil { handler = DefaultServeMux } if req.RequestURI == "*" && req.Method == "OPTIONS" { handler = globalOptionsHandler{} } handler.ServeHTTP(rw, req) }
這裏DefaultServeMux的ServeHTTP方法其實也是定義在ServeMux結構中的,相關代碼以下:
// Find a handler on a handler map given a path string. // Most-specific (longest) pattern wins. func (mux *ServeMux) match(path string) (h Handler, pattern string) { // Check for exact match first. v, ok := mux.m[path] if ok { return v.h, v.pattern } // Check for longest valid match. var n = 0 for k, v := range mux.m { if !pathMatch(k, path) { continue } if h == nil || len(k) > n { n = len(k) h = v.h pattern = v.pattern } } return } func (mux *ServeMux) Handler(r *Request) (h Handler, pattern string) { // CONNECT requests are not canonicalized. if r.Method == "CONNECT" { return mux.handler(r.Host, r.URL.Path) } // All other requests have any port stripped and path cleaned // before passing to mux.handler. host := stripHostPort(r.Host) path := cleanPath(r.URL.Path) if path != r.URL.Path { _, pattern = mux.handler(host, path) url := *r.URL url.Path = path return RedirectHandler(url.String(), StatusMovedPermanently), pattern } return mux.handler(host, r.URL.Path) } // handler is the main implementation of Handler. // The path is known to be in canonical form, except for CONNECT methods. func (mux *ServeMux) handler(host, path string) (h Handler, pattern string) { mux.mu.RLock() defer mux.mu.RUnlock() // Host-specific pattern takes precedence over generic ones if mux.hosts { h, pattern = mux.match(host + path) } if h == nil { h, pattern = mux.match(path) } if h == nil { h, pattern = NotFoundHandler(), "" } return } // ServeHTTP dispatches the request to the handler whose // pattern most closely matches the request URL. func (mux *ServeMux) ServeHTTP(w ResponseWriter, r *Request) { if r.RequestURI == "*" { if r.ProtoAtLeast(1, 1) { w.Header().Set("Connection", "close") } w.WriteHeader(StatusBadRequest) return } h, _ := mux.Handler(r) h.ServeHTTP(w, r) }
mux的ServeHTTP方法經過調用其Handler方法尋找註冊到路由上的handler函數,並調用該函數的ServeHTTP方法,本例則是IndexHandler函數。 mux的Handler方法對URL簡單的處理,而後調用handler方法,後者會建立一個鎖,同時調用match方法返回一個handler和pattern。 在match方法中,mux的m字段是map[string]muxEntry圖,後者存儲了pattern和handler處理器函數,所以經過迭代m尋找出註冊路由的patten模式與實際url匹配的handler函數並返回。 返回的結構一直傳遞到mux的ServeHTTP方法,接下來調用handler函數的ServeHTTP方法,即IndexHandler函數,而後把response寫到http.RequestWirter對象返回給客戶端。 上述函數運行結束即`serverHandler{c.server}.ServeHTTP(w, w.req)`運行結束。接下來就是對請求處理完畢以後上但願和鏈接斷開的相關邏輯。 至此,Golang中一個完整的http服務介紹完畢,包括註冊路由,開啓監聽,處理鏈接,路由處理函數。
net/http不只提供了服務端處理,還提供了客戶端處理功能。
http包中提供了Get、Post、Head、PostForm方法實現HTTP請求:
//GET func Get(url string) (resp *Response, err error) { return DefaultClient.Get(url) } //POST func Post(url string, contentType string, body io.Reader) (resp *Response, err error) { return DefaultClient.Post(url, contentType, body) } //HEAD func Head(url string) (resp *Response, err error) { return DefaultClient.Head(url) } //POSTFORM func PostForm(url string, data url.Values) (resp *Response, err error) { return DefaultClient.PostForm(url, data) }
package main import ( "fmt" "net/http" "log" "reflect" "bytes" ) func main() { resp, err := http.Get("http://www.baidu.com") if err != nil { // 錯誤處理 log.Println(err) return } defer resp.Body.Close() //關閉連接 headers := resp.Header for k, v := range headers { fmt.Printf("k=%v, v=%v\n", k, v) //全部頭信息 } fmt.Printf("resp status %s,statusCode %d\n", resp.Status, resp.StatusCode) fmt.Printf("resp Proto %s\n", resp.Proto) fmt.Printf("resp content length %d\n", resp.ContentLength) fmt.Printf("resp transfer encoding %v\n", resp.TransferEncoding) fmt.Printf("resp Uncompressed %t\n", resp.Uncompressed) fmt.Println(reflect.TypeOf(resp.Body)) buf := bytes.NewBuffer(make([]byte, 0, 512)) length, _ := buf.ReadFrom(resp.Body) fmt.Println(len(buf.Bytes())) fmt.Println(length) fmt.Println(string(buf.Bytes())) }
package main import ( "net/http" "strings" "io/ioutil" "log" "fmt" ) func main() { client := &http.Client{} req, err := http.NewRequest("POST", "http://www.baidu.com", strings.NewReader("name=xxxx&passwd=xxxx")) if err != nil { fmt.Println(err) return } req.Header.Set("Content-Type", "application/x-www-form-urlencoded; charset=UTF-8") //設置請求頭信息 resp, err := client.Do(req) defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { log.Println(err) return } var res string res = string(body[:]) fmt.Println(res) }
package main import ( "net/http" "strings" "fmt" "io/ioutil" ) func main() { resp, err := http.Post("http://www.baidu.com", "application/x-www-form-urlencoded", strings.NewReader("username=xxx&password=xxxx")) if err != nil { fmt.Println(err) return } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { fmt.Println(err) return } fmt.Println(string(body)) }
package main import ( "net/http" "fmt" "io/ioutil" "net/url" ) func main() { postParam := url.Values{ "name": {"wd"}, "password": {"1234"}, } resp, err := http.PostForm("https://cn.bing.com/", postParam) if err != nil { fmt.Println(err) return } defer resp.Body.Close() body, err := ioutil.ReadAll(resp.Body) if err != nil { fmt.Println(err) return } fmt.Println(string(body)) }