redis(REmote DIctionary Server)是一個由Salvatore Sanfilippo寫key-value存儲系統,它由C語言編寫、遵照BSD協議、支持網絡、可基於內存亦可持久化的日誌型、Key-Value類型的數據庫,並提供多種語言的API。和Memcached相似,它支持存儲的value類型相對更多,包括string(字符串)、list(鏈表)、set(集合)、zset(sorted set --有序集合)和hash(哈希類型)。這些數據類型都支持push/pop、add/remove及取交集並集和差集及更豐富的操做,並且這些操做都是原子性的。在此基礎上,redis支持各類不一樣方式的排序。與memcached同樣,爲了保證效率,數據都是緩存在內存中。區別的是redis會週期性的把更新的數據寫入磁盤或者把修改操做寫入追加的記錄文件,而且在此基礎上實現了master-slave(主從)同步,redis在3.0版本推出集羣模式。mysql
官方網站:https://redis.io/git
yum install gcc -y #安裝C依賴 wget http://download.redis.io/redis-stable.tar.gz #下載穩定版本 tar zxvf redis-stable.tar.gz #解壓 cd redis-stable make PREFIX=/opt/app/redis install #指定目錄編譯 make install mkdir /etc/redis #創建配置目錄 cp redis.conf /etc/redis/6379.conf # 拷貝配置文件 cp utils/redis_init_script /etc/init.d/redis #拷貝init啓動腳本針對6.X系統 chmod a+x /etc/init.d/redis #添加執行權限 修改配置文件: vi /etc/redis/6379.conf bind 0.0.0.0 #監聽地址 maxmemory 4294967296 #限制最大內存(4G): daemonize yes #後臺運行 ####啓動與中止 /etc/init.d/redis start /etc/init.d/redis stop
查看版本信息github
#執行客戶端工具 redis-cli #輸入命令info 127.0.0.1:6379> info # Server redis_version:4.0.10 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:cf83e9c690dbed33 redis_mode:standalone os:Linux 2.6.32-642.el6.x86_64 x86_64 arch_bits:64 multiplexing_api:epoll
golang操做redis的客戶端包有多個好比redigo、go-redis,github上Star最多的莫屬redigo。golang
github地址:https://github.com/garyburd/redigo 目前已經遷移到:https://github.com/gomodule/redigo 正則表達式
文檔:https://godoc.org/github.com/garyburd/redigo/redisredis
go get github.com/garyburd/redigo/redis import "github.com/garyburd/redigo/redis"
Conn接口是與Redis協做的主要接口,可使用Dial,DialWithTimeout或者NewConn函數來建立鏈接,當任務完成時,應用程序必須調用Close函數來完成操做。sql
package main import ( "github.com/garyburd/redigo/redis" "fmt" ) func main() { conn,err := redis.Dial("tcp","10.1.210.69:6379") if err != nil { fmt.Println("connect redis error :",err) return } defer conn.Close() }
經過使用Conn接口中的do方法執行redis命令,redis命令大全參考:http://doc.redisfans.com/數據庫
go中發送與響應對應類型:api
Do函數會必要時將參數轉化爲二進制字符串緩存
Go Type | Conversion |
---|---|
[]byte | Sent as is |
string | Sent as is |
int, int64 | strconv.FormatInt(v) |
float64 | strconv.FormatFloat(v, 'g', -1, 64) |
bool | true -> "1", false -> "0" |
nil | "" |
all other types | fmt.Print(v) |
Redis 命令響應會用如下Go類型表示:
Redis type | Go type |
---|---|
error | redis.Error |
integer | int64 |
simple string | string |
bulk string | []byte or nil if value not present. |
array | []interface{} or nil if value not present. |
可使用GO的類型斷言或者reply輔助函數將返回的interface{}轉換爲對應類型。
操做示例:
get、set
package main import ( "github.com/garyburd/redigo/redis" "fmt" ) func main() { conn,err := redis.Dial("tcp","10.1.210.69:6379") if err != nil { fmt.Println("connect redis error :",err) return } defer conn.Close() _, err = conn.Do("SET", "name", "wd") if err != nil { fmt.Println("redis set error:", err) } name, err := redis.String(conn.Do("GET", "name")) if err != nil { fmt.Println("redis get error:", err) } else { fmt.Printf("Got name: %s \n", name) } }
設置key過時時間
_, err = conn.Do("expire", "name", 10) //10秒過時 if err != nil { fmt.Println("set expire error: ", err) return }
批量獲取mget、批量設置mset
_, err = conn.Do("MSET", "name", "wd","age",22) if err != nil { fmt.Println("redis mset error:", err) } res, err := redis.Strings(conn.Do("MGET", "name","age")) if err != nil { fmt.Println("redis get error:", err) } else { res_type := reflect.TypeOf(res) fmt.Printf("res type : %s \n", res_type) fmt.Printf("MGET name: %s \n", res) fmt.Println(len(res)) } //結果: //res type : []string //MGET name: [wd 22] //2
列表操做
package main import ( "github.com/garyburd/redigo/redis" "fmt" "reflect" ) func main() { conn,err := redis.Dial("tcp","10.1.210.69:6379") if err != nil { fmt.Println("connect redis error :",err) return } defer conn.Close() _, err = conn.Do("LPUSH", "list1", "ele1","ele2","ele3") if err != nil { fmt.Println("redis mset error:", err) } res, err := redis.String(conn.Do("LPOP", "list1")) if err != nil { fmt.Println("redis POP error:", err) } else { res_type := reflect.TypeOf(res) fmt.Printf("res type : %s \n", res_type) fmt.Printf("res : %s \n", res) } } //res type : string //res : ele3
hash操做
package main import ( "github.com/garyburd/redigo/redis" "fmt" "reflect" ) func main() { conn,err := redis.Dial("tcp","10.1.210.69:6379") if err != nil { fmt.Println("connect redis error :",err) return } defer conn.Close() _, err = conn.Do("HSET", "student","name", "wd","age",22) if err != nil { fmt.Println("redis mset error:", err) } res, err := redis.Int64(conn.Do("HGET", "student","age")) if err != nil { fmt.Println("redis HGET error:", err) } else { res_type := reflect.TypeOf(res) fmt.Printf("res type : %s \n", res_type) fmt.Printf("res : %d \n", res) } } //res type : int64 //res : 22
管道操做能夠理解爲併發操做,並經過Send(),Flush(),Receive()三個方法實現。客戶端可使用send()方法一次性向服務器發送一個或多個命令,命令發送完畢時,使用flush()方法將緩衝區的命令輸入一次性發送到服務器,客戶端再使用Receive()方法依次按照先進先出的順序讀取全部命令操做結果。
Send(commandName string, args ...interface{}) error Flush() error Receive() (reply interface{}, err error)
示例:
package main import ( "github.com/garyburd/redigo/redis" "fmt" ) func main() { conn,err := redis.Dial("tcp","10.1.210.69:6379") if err != nil { fmt.Println("connect redis error :",err) return } defer conn.Close() conn.Send("HSET", "student","name", "wd","age","22") conn.Send("HSET", "student","Score","100") conn.Send("HGET", "student","age") conn.Flush() res1, err := conn.Receive() fmt.Printf("Receive res1:%v \n", res1) res2, err := conn.Receive() fmt.Printf("Receive res2:%v\n",res2) res3, err := conn.Receive() fmt.Printf("Receive res3:%s\n",res3) } //Receive res1:0 //Receive res2:0 //Receive res3:22
redis自己具備發佈訂閱的功能,其發佈訂閱功能經過命令SUBSCRIBE(訂閱)/PUBLISH(發佈)實現,而且發佈訂閱模式能夠是多對多模式還可支持正則表達式,發佈者能夠向一個或多個頻道發送消息,訂閱者可訂閱一個或者多個頻道接受消息。
示意圖:
發佈者:
訂閱者:
操做示例,示例中將使用兩個goroutine分別擔任發佈者和訂閱者角色進行演示:
package main import ( "github.com/garyburd/redigo/redis" "fmt" "time" ) func Subs() { //訂閱者 conn, err := redis.Dial("tcp", "10.1.210.69:6379") if err != nil { fmt.Println("connect redis error :", err) return } defer conn.Close() psc := redis.PubSubConn{conn} psc.Subscribe("channel1") //訂閱channel1頻道 for { switch v := psc.Receive().(type) { case redis.Message: fmt.Printf("%s: message: %s\n", v.Channel, v.Data) case redis.Subscription: fmt.Printf("%s: %s %d\n", v.Channel, v.Kind, v.Count) case error: fmt.Println(v) return } } } func Push(message string) { //發佈者 conn, _ := redis.Dial("tcp", "10.1.210.69:6379") _,err1 := conn.Do("PUBLISH", "channel1", message) if err1 != nil { fmt.Println("pub err: ", err1) return } } func main() { go Subs() go Push("this is wd") time.Sleep(time.Second*3) } //channel1: subscribe 1 //channel1: message: this is wd
MULTI, EXEC,DISCARD和WATCH是構成Redis事務的基礎,固然咱們使用go語言對redis進行事務操做的時候本質也是使用這些命令。
MULTI:開啓事務
EXEC:執行事務
DISCARD:取消事務
WATCH:監視事務中的鍵變化,一旦有改變則取消事務。
示例:
package main import ( "github.com/garyburd/redigo/redis" "fmt" ) func main() { conn,err := redis.Dial("tcp","10.1.210.69:6379") if err != nil { fmt.Println("connect redis error :",err) return } defer conn.Close() conn.Send("MULTI") conn.Send("INCR", "foo") conn.Send("INCR", "bar") r, err := conn.Do("EXEC") fmt.Println(r) } //[1, 1]
redis鏈接池是經過pool結構體實現,如下是源碼定義,相關參數說明已經備註:
type Pool struct { // Dial is an application supplied function for creating and configuring a // connection. // // The connection returned from Dial must not be in a special state // (subscribed to pubsub channel, transaction started, ...). Dial func() (Conn, error) //鏈接方法 // TestOnBorrow is an optional application supplied function for checking // the health of an idle connection before the connection is used again by // the application. Argument t is the time that the connection was returned // to the pool. If the function returns an error, then the connection is // closed. TestOnBorrow func(c Conn, t time.Time) error // Maximum number of idle connections in the pool. MaxIdle int //最大的空閒鏈接數,即便沒有redis鏈接時依然能夠保持N個空閒的鏈接,而不被清除,隨時處於待命狀態 // Maximum number of connections allocated by the pool at a given time. // When zero, there is no limit on the number of connections in the pool. MaxActive int //最大的激活鏈接數,同時最多有N個鏈接 // Close connections after remaining idle for this duration. If the value // is zero, then idle connections are not closed. Applications should set // the timeout to a value less than the server's timeout. IdleTimeout time.Duration //空閒鏈接等待時間,超過此時間後,空閒鏈接將被關閉 // If Wait is true and the pool is at the MaxActive limit, then Get() waits // for a connection to be returned to the pool before returning. Wait bool //當配置項爲true而且MaxActive參數有限制時候,使用Get方法等待一個鏈接返回給鏈接池 // Close connections older than this duration. If the value is zero, then // the pool does not close connections based on age. MaxConnLifetime time.Duration // contains filtered or unexported fields }
示例:
package main import ( "github.com/garyburd/redigo/redis" "fmt" ) var Pool redis.Pool func init() { //init 用於初始化一些參數,先於main執行 Pool = redis.Pool{ MaxIdle: 16, MaxActive: 32, IdleTimeout: 120, Dial: func() (redis.Conn, error) { return redis.Dial("tcp", "10.1.210.69:6379") }, } } func main() { conn :=Pool.Get() res,err := conn.Do("HSET","student","name","jack") fmt.Println(res,err) res1,err := redis.String(conn.Do("HGET","student","name")) fmt.Printf("res:%s,error:%v",res1,err) } //0 <nil> //res:jack,error:<nil>
mysql目前來講是使用最爲流行的關係型數據庫,golang操做mysql使用最多的包go-sql-driver/mysql。
sqlx包是做爲database/sql包的一個額外擴展包,在原有的database/sql加了不少擴展,如直接將查詢的數據轉爲結構體,大大簡化了代碼書寫,固然database/sql包中的方法一樣起做用。
github地址:
golang sql使用:
go get "github.com/go-sql-driver/mysql" go get "github.com/jmoiron/sqlx"
var Db *sqlx.DB db, err := sqlx.Open("mysql","username:password@tcp(ip:port)/database?charset=utf8") Db = db
sqlx設計和database/sql使用方法是同樣的。包含有4中主要的handle types:
全部的handler types都提供了對database/sql的兼容,意味着當你調用sqlx.DB.Query時,能夠直接替換爲sql.DB.Query.這就使得sqlx能夠很容易的加入到已有的數據庫項目中。
此外,sqlx還有兩個cursor類型:
相比database/sql方法還多了新語法,也就是實現將獲取的數據直接轉換結構體實現。
如下全部示例均已如下表結構做爲操做基礎。
CREATE TABLE `userinfo` ( `uid` INT(10) NOT NULL AUTO_INCREMENT, `username` VARCHAR(64) DEFAULT NULL, `password` VARCHAR(32) DEFAULT NULL, `department` VARCHAR(64) DEFAULT NULL, `email` varchar(64) DEFAULT NULL, PRIMARY KEY (`uid`) )ENGINE=InnoDB DEFAULT CHARSET=utf8
Exec和MustExec從鏈接池中獲取一個鏈接而後只想對應的query操做。對於不支持ad-hoc query execution的驅動,在操做執行的背後會建立一個prepared statement。在結果返回前這個connection會返回到鏈接池中。
須要注意的是不一樣的數據庫類型使用的佔位符不一樣,mysql採用?做爲佔位符號。
查詢語法使用Query後續會提到
package main import ( _ "github.com/go-sql-driver/mysql" "github.com/jmoiron/sqlx" "fmt" ) var Db *sqlx.DB func init() { db, err := sqlx.Open("mysql", "stu:1234qwer@tcp(10.0.0.241:3307)/test?charset=utf8") if err != nil { fmt.Println("open mysql failed,", err) return } Db = db } func main() { result, err := Db.Exec("INSERT INTO userinfo (username, password, department,email) VALUES (?, ?, ?,?)","wd","123","it","wd@163.com") if err != nil{ fmt.Println("insert failed,error: ", err) return } id,_ := result.LastInsertId() fmt.Println("insert id is :",id) _, err1 := Db.Exec("update userinfo set username = ? where uid = ?","jack",1) if err1 != nil{ fmt.Println("update failed error:",err1) } else { fmt.Println("update success!") } _, err2 := Db.Exec("delete from userinfo where uid = ? ", 1) if err2 != nil{ fmt.Println("delete error:",err2) }else{ fmt.Println("delete success") } } //insert id is : 1 //update success! //delete success
對於大部分的數據庫來講,當一個query執行的時候,在sql語句數據庫內部聲明已經聲明過了,其聲明是在數據庫中,咱們能夠提早進行聲明,以便在其餘地方重用。
stmt, err := db.Prepare(`SELECT * FROM place WHERE telcode=?`) row = stmt.QueryRow(65) tx, err := db.Begin() txStmt, err := tx.Prepare(`SELECT * FROM place WHERE telcode=?`) row = txStmt.QueryRow(852)
固然sqlx還提供了Preparex()進行擴展,可直接用於結構體轉換
stmt, err := db.Preparex(`SELECT * FROM place WHERE telcode=?`) var p Place err = stmt.Get(&p, 852)
Query是database/sql中執行查詢主要使用的方法,該方法返回row結果。Query返回一個sql.Rows對象和一個error對象。
在使用的時候應該吧Rows當成一個遊標而不是一系列的結果。儘管數據庫驅動緩存的方法不同,經過Next()迭代每次獲取一列結果,對於查詢結果很是巨大的狀況下,能夠有效的限制內存的使用,Scan()利用reflect把sql每一列結果映射到go語言的數據類型如string,[]byte等。若是你沒有遍歷徹底部的rows結果,必定要記得在把connection返回到鏈接池以前調用rows.Close()。
Query返回的error有多是在server準備查詢的時候發生的,也有多是在執行查詢語句的時候發生的。例如可能從鏈接池中獲取一個壞的連級(儘管數據庫會嘗試10次去發現或建立一個工做鏈接)。通常來講,錯誤主要由錯誤的sql語句,錯誤的相似匹配,錯誤的域名或表名等。
在大部分狀況下,Rows.Scan()會把從驅動獲取的數據進行拷貝,不管驅動如何使用緩存。特殊類型sql.RawBytes能夠用來從驅動返回的數據總獲取一個zero-copy的slice byte。當下一次調用Next的時候,這個值就不在有效了,由於它指向的內存已經被驅動重寫了別的數據。
Query使用的connection在全部的rows經過Next()遍歷完後或者調用rows.Close()後釋放。
示例:
package main import ( _ "github.com/go-sql-driver/mysql" "github.com/jmoiron/sqlx" "fmt" ) var Db *sqlx.DB func init() { db, err := sqlx.Open("mysql", "stu:1234qwer@tcp(10.0.0.241:3307)/test?charset=utf8") if err != nil { fmt.Println("open mysql failed,", err) return } Db = db } func main() { rows, err := Db.Query("SELECT username,password,email FROM userinfo") if err != nil{ fmt.Println("query failed,error: ", err) return } for rows.Next() { //循環結果 var username,password,email string err = rows.Scan(&username, &password, &email) println(username,password,email) } } //wd 123 wd@163.com //jack 1222 jack@165.com
Queryx和Query行爲很類似,不過返回一個sqlx.Rows對象,支持擴展的scan行爲,同時可將對數據進行結構體轉換。
示例:
package main import ( _ "github.com/go-sql-driver/mysql" "github.com/jmoiron/sqlx" "fmt" ) var Db *sqlx.DB type stu struct { Username string `db:"username"` Password string `db:"password"` Department string `db:"department"` Email string `db:"email"` } func init() { db, err := sqlx.Open("mysql", "stu:1234qwer@tcp(10.0.0.241:3307)/test?charset=utf8") if err != nil { fmt.Println("open mysql failed,", err) return } Db = db } func main() { rows, err := Db.Queryx("SELECT username,password,email FROM userinfo") if err != nil{ fmt.Println("Qeryx failed,error: ", err) return } for rows.Next() { //循環結果 var stu1 stu err = rows.StructScan(&stu1)// 轉換爲結構體 fmt.Println("stuct data:",stu1.Username,stu1.Password) } } //stuct data: wd 123 //stuct data: jack 1222
QueryRow和QueryRowx都是從數據庫中獲取一條數據,可是QueryRowx提供scan擴展,可直接將結果轉換爲結構體。
package main import ( _ "github.com/go-sql-driver/mysql" "github.com/jmoiron/sqlx" "fmt" ) var Db *sqlx.DB type stu struct { Username string `db:"username"` Password string `db:"password"` Department string `db:"department"` Email string `db:"email"` } func init() { db, err := sqlx.Open("mysql", "stu:1234qwer@tcp(10.0.0.241:3307)/test?charset=utf8") if err != nil { fmt.Println("open mysql failed,", err) return } Db = db } func main() { row := Db.QueryRow("SELECT username,password,email FROM userinfo where uid = ?",1) // QueryRow返回錯誤,錯誤經過Scan返回 var username,password,email string err :=row.Scan(&username,&password,&email) if err != nil{ fmt.Println(err) } fmt.Printf("this is QueryRow res:[%s:%s:%s]\n",username,password,email) var s stu err1 := Db.QueryRowx("SELECT username,password,email FROM userinfo where uid = ?",2).StructScan(&s) if err1 != nil{ fmt.Println("QueryRowx error :",err1) }else { fmt.Printf("this is QueryRowx res:%v",s) } } //this is QueryRow res:[wd:123:wd@163.com] //this is QueryRowx res:{jack 1222 jack@165.com}
Get和Select是一個很是省時的擴展,可直接將結果賦值給結構體,其內部封裝了StructScan進行轉化。Get用於獲取單個結果真後Scan,Select用來獲取結果切片。
示例:
package main import ( _ "github.com/go-sql-driver/mysql" "github.com/jmoiron/sqlx" "fmt" ) var Db *sqlx.DB type stu struct { Username string `db:"username"` Password string `db:"password"` Department string `db:"department"` Email string `db:"email"` } func init() { db, err := sqlx.Open("mysql", "stu:1234qwer@tcp(10.0.0.241:3307)/test?charset=utf8") if err != nil { fmt.Println("open mysql failed,", err) return } Db = db } func main() { var stus []stu err := Db.Select(&stus,"SELECT username,password,email FROM userinfo") if err != nil{ fmt.Println("Select error",err) } fmt.Printf("this is Select res:%v\n",stus) var s stu err1 := Db.Get(&s,"SELECT username,password,email FROM userinfo where uid = ?",2) if err1 != nil{ fmt.Println("GET error :",err1) }else { fmt.Printf("this is GET res:%v",s) } } //this is Select res:[{wd 123 wd@163.com} {jack 1222 jack@165.com}] //this is GET res:{jack 1222 jack@165.com}
事務操做是經過三個方法實現:
Begin():開啓事務
Commit():提交事務(執行sql)
Rollback():回滾
使用流程:
tx, err := db.Begin() err = tx.Exec(...) err = tx.Commit() //或者使用sqlx擴展的事務 tx := db.MustBegin() tx.MustExec(...) err = tx.Commit()
因爲事務是一個一直鏈接的狀態,因此Tx對象必須綁定和控制單個鏈接。一個Tx會在整個生命週期中保存一個鏈接,而後在調用commit或Rollback()的時候釋放掉。在調用這幾個函數的時候必須十分當心,不然鏈接會一直被佔用直到被垃圾回收。
使用示例:
package main import ( _ "github.com/go-sql-driver/mysql" "github.com/jmoiron/sqlx" "fmt" ) var Db *sqlx.DB func init() { db, err := sqlx.Open("mysql", "stu:1234qwer@tcp(10.0.0.241:3307)/test?charset=utf8") if err != nil { fmt.Println("open mysql failed,", err) return } Db = db } func main() { tx, err := Db.Beginx() _, err = tx.Exec("insert into userinfo(username,password) values(?,?)", "Rose","2223") if err != nil { tx.Rollback() } _, err = tx.Exec("insert into userinfo(username,password) values(?,?)", "Mick",222) if err != nil { fmt.Println("exec sql error:",err) tx.Rollback() } err = tx.Commit() if err != nil { fmt.Println("commit error") } }
默認狀況下,鏈接池增加無限制,而且只要鏈接池中沒有可用的空閒鏈接,就會建立鏈接。咱們可使用DB.SetMaxOpenConns設置池的最大大小。未使用的鏈接標記爲空閒,若是不須要則關閉。要避免創建和關閉大量鏈接,可使用DB.SetMaxIdleConns設置最大空閒鏈接。
注意:該設置方法golang版本至少爲1.2
示例:
package main import ( _ "github.com/go-sql-driver/mysql" "github.com/jmoiron/sqlx" "fmt" ) var Db *sqlx.DB func init() { db, err := sqlx.Open("mysql", "stu:1234qwer@tcp(10.0.0.241:3307)/test?charset=utf8") if err != nil { fmt.Println("open mysql failed,", err) return } Db = db Db.SetMaxOpenConns(30) Db.SetMaxIdleConns(15) }
參考:http://jmoiron.github.io/sqlx/