NoSQL和memcached相關

目錄
1.NoSQL介紹
2.Memached介紹
3.裝memcached安裝
4.查看Memcached運行狀態
5.memcached命令行
6.memcached數據導出和導入
7.php鏈接memcached
8.memcached中存儲session

1.NoSQL介紹php

非關係型數據庫就是NoSQL，關係型數據庫表明MySQL、SQLserverhtml

對於關係型數據庫來講，是須要把數據存儲到庫、表、行、字段裏，查詢的時候根據條件一行一行地去匹配，當量很是大的時候就很耗費時間和資源，尤爲是數據是須要從磁盤裏去檢索mysql

NoSQL數據庫存儲原理很是簡單（典型的數據類型爲k-v），不存在繁雜的關係鏈，好比mysql查詢的時候，須要找到對應的庫、表（一般是多個表）以及字段。linux

NoSQL數據能夠存儲在內存裏，查詢速度很是快nginx

NoSQL在性能表現上雖然能優於關係型數據庫，可是它並不能徹底替代關係型數據庫web

NoSQL由於沒有複雜的數據結構，擴展很是容易，支持分佈式redis

常見NoSQL數據庫算法

k-v形式的：memcached、redis 適合儲存用戶信息，好比會話、配置文件、參數、購物車等等。這些信息通常都和ID（鍵）掛鉤，這種情景下鍵值數據庫是個很好的選擇。sql

文檔數據庫：mongodb 將數據以文檔的形式儲存。每一個文檔都是一系列數據項的集合。每一個數據項都有一個名稱與對應的值，值既能夠是簡單的數據類型，如字符串、數字和日期等；也能夠是複雜的類型，若有序列表和關聯對象。數據存儲的最小單位是文檔，同一個表中存儲的文檔屬性能夠是不一樣的，數據可使用XML、JSON或者JSONB等多種形式存儲。mongodb

列存儲 Hbase

圖 Neo4J、Infinite Graph、OrientDB

2.Memached介紹

Memcached是國外社區網站LiveJournal團隊開發，目的是爲了經過緩存數據庫查詢結果，減小數據庫訪問次數，從而提升動態web站點性能。

官方站點 http://www.memcached.org/

數據結構簡單(k-v)，數據存放在內存裏（重啓服務或者重慶服務器數據就會消失，須要不按期的作數據落地）；多線程；基於c/s架構，協議簡單；基於libevent的事件處理；自主內存存儲處理（slab allowcation) ；數據過時方式：Lazy Expiration 和 LRU

Memcached的數據流向

Slab allocation

Slab Allocation的原理

將分配的內存分割成各類尺寸的塊（chunk），並把尺寸相同的塊分紅組（chunk的集合），每一個chunk集合被稱爲slab。

Memcached的內存分配以Page爲單位，Page默認值爲1M，能夠在啓動時經過-I參數來指定。

Slab是由多個Page組成的，Page按照指定大小切割成多個chunk。

Growth factor

Memcached在啓動時經過-f選項能夠指定 Growth Factor因子。該值控制chunk大小的差別。默認值爲1.25。

經過memcached-tool命令查看指定Memcached實例的不一樣slab狀態，能夠看到各Item所佔大小（chunk大小）差距爲1.25

命令：# memcached-tool 127.0.0.1:11211 display

3.裝memcached安裝

只須要yum安裝，啓動服務

[root@chenshi ~]# yum install -y memcached libmemcached libevent

已安裝:
  libevent.x86_64 0:2.0.21-4.el7      libmemcached.x86_64 0:1.0.16-5.el7      memcached.x86_64 0:1.4.15-10.el7_3.1     

[root@chenshi ~]# systemctl start memcached.service 
[root@chenshi ~]# ps aux|grep memcached
memcach+ 11154  0.0  0.0 344080  1664 ?        Ssl  13:27   0:00 /usr/bin/memcached -u memcached -p 11211 -m 64 -c 1024
root     11175  0.0  0.0 112720   972 pts/0    S+   13:28   0:00 grep --color=auto memcached

自定義端口和用戶

[root@chenshi ~]# vi /etc/sysconfig/memcached 

PORT="11211"
USER="memcached"
MAXCONN="1024"
CACHESIZE="64"
OPTIONS=""

[root@chenshi ~]# memcached -h
memcached 1.4.15
-p <num>      TCP port number to listen on (default: 11211)
-U <num>      UDP port number to listen on (default: 11211, 0 is off)
-s <file>     UNIX socket path to listen on (disables network support)
-a <mask>     access mask for UNIX socket, in octal (default: 0700)
-l <addr>     interface to listen on (default: INADDR_ANY, all addresses)
              <addr> may be specified as host:port. If you don't specify
              a port number, the value you specified with -p or -U is
              used. You may specify multiple addresses separated by comma
              or by using -l multiple times
-d            run as a daemon
-r            maximize core file limit
-u <username> assume identity of <username> (only when run as root)
-m <num>      max memory to use for items in megabytes (default: 64 MB)
-M            return error on memory exhausted (rather than removing items)
-c <num>      max simultaneous connections (default: 1024)
-k            lock down all paged memory.  Note that there is a
              limit on how much memory you may lock.  Trying to
              allocate more than that would fail, so be sure you
              set the limit correctly for the user you started
              the daemon with (not for -u <username> user;
              under sh this is done with 'ulimit -S -l NUM_KB').
-v            verbose (print errors/warnings while in event loop)
-vv           very verbose (also print client commands/reponses)
-vvv          extremely verbose (also print internal state transitions)
-h            print this help and exit
-i            print memcached and libevent license
-P <file>     save PID in <file>, only used with -d option
-f <factor>   chunk size growth factor (default: 1.25)
-n <bytes>    minimum space allocated for key+value+flags (default: 48)
-L            Try to use large memory pages (if available). Increasing
              the memory page size could reduce the number of TLB misses
              and improve the performance. In order to get large pages
              from the OS, memcached will allocate the total item-cache
              in one large chunk.
-D <char>     Use <char> as the delimiter between key prefixes and IDs.
              This is used for per-prefix stats reporting. The default is
              ":" (colon). If this option is specified, stats collection
              is turned on automatically; if not, then it may be turned on
              by sending the "stats detail on" command to the server.
-t <num>      number of threads to use (default: 4)
-R            Maximum number of requests per event, limits the number of
              requests process for a given connection to prevent 
              starvation (default: 20)
-C            Disable use of CAS
-b <num>      Set the backlog queue limit (default: 1024)
-B            Binding protocol - one of ascii, binary, or auto (default)
-I            Override the size of each slab page. Adjusts max item size
              (default: 1mb, min: 1k, max: 128m)
-S            Turn on Sasl authentication
-o            Comma separated list of extended or experimental options
              - (EXPERIMENTAL) maxconns_fast: immediately close new
                connections if over maxconns limit
              - hashpower: An integer multiplier for how large the hash
                table should be. Can be grown at runtime if not big enough.
                Set this based on "STAT hash_power_level" before a 
                restart.

其中-m指定memcached分配內存

-c指定最大併發數

-u指定運行memcached服務的用戶

4.查看Memcached運行狀態

須要關注get_hits 、curr_items（命中的項目）

[root@chenshi ~]# memcached-tool  127.0.0.1:11211 stats
#127.0.0.1:11211   Field       Value
         accepting_conns           1
               auth_cmds           0
             auth_errors           0
                   bytes           0
              bytes_read           7
           bytes_written           0
              cas_badval           0
                cas_hits           0
              cas_misses           0
               cmd_flush           0
                 cmd_get           0
                 cmd_set           0
               cmd_touch           0
             conn_yields           0
   connection_structures          11
        curr_connections          10
              curr_items           0
               decr_hits           0
             decr_misses           0
             delete_hits           0
           delete_misses           0
       evicted_unfetched           0
               evictions           0
       expired_unfetched           0
                get_hits           0
              get_misses           0
              hash_bytes      524288
       hash_is_expanding           0
        hash_power_level          16
               incr_hits           0
             incr_misses           0
                libevent 2.0.21-stable
          limit_maxbytes    67108864
     listen_disabled_num           0
                     pid       11154
            pointer_size          64
               reclaimed           0
            reserved_fds          20
           rusage_system    0.009883
             rusage_user    0.005929
                 threads           4
                    time  1534743302
       total_connections          11
             total_items           0
              touch_hits           0
            touch_misses           0
                  uptime         430
                 version      1.4.15

使用nc查看memcached狀態

安裝
 nmap-ncat.x86_64 2:6.40-13.el

[root@chenshi ~]# echo stats |nc 127.0.0.1 11211
STAT pid 11154
STAT uptime 2250
STAT time 1534745122
STAT version 1.4.15
STAT libevent 2.0.21-stable
STAT pointer_size 64
STAT rusage_user 0.021608
STAT rusage_system 0.038414
STAT curr_connections 10
STAT total_connections 12
STAT connection_structures 11
STAT reserved_fds 20
STAT cmd_get 0
STAT cmd_set 0
STAT cmd_flush 0
STAT cmd_touch 0
STAT get_hits 0
STAT get_misses 0
STAT delete_misses 0
STAT delete_hits 0
STAT incr_misses 0
STAT incr_hits 0
STAT decr_misses 0
STAT decr_hits 0
STAT cas_misses 0
STAT cas_hits 0
STAT cas_badval 0
STAT touch_hits 0
STAT touch_misses 0
STAT auth_cmds 0
STAT auth_errors 0
STAT bytes_read 13
STAT bytes_written 1025
STAT limit_maxbytes 67108864
STAT accepting_conns 1
STAT listen_disabled_num 0
STAT threads 4
STAT conn_yields 0
STAT hash_power_level 16
STAT hash_bytes 524288
STAT hash_is_expanding 0
STAT bytes 0
STAT curr_items 0
STAT total_items 0
STAT expired_unfetched 0
STAT evicted_unfetched 0
STAT evictions 0
STAT reclaimed 0
END

memstat查看服務

[root@chenshi ~]# memstat --servers=127.0.0.1:11211
Server: 127.0.0.1 (11211)
	 pid: 11154
	 uptime: 2340
	 time: 1534745212
	 version: 1.4.15
	 libevent: 2.0.21-stable
	 pointer_size: 64
	 rusage_user: 0.021700
	 rusage_system: 0.040989
	 curr_connections: 10
	 total_connections: 13
	 connection_structures: 11
	 reserved_fds: 20
	 cmd_get: 0
	 cmd_set: 0
	 cmd_flush: 0
	 cmd_touch: 0
	 get_hits: 0
	 get_misses: 0
	 delete_misses: 0
	 delete_hits: 0
	 incr_misses: 0
	 incr_hits: 0
	 decr_misses: 0
	 decr_hits: 0
	 cas_misses: 0
	 cas_hits: 0
	 cas_badval: 0
	 touch_hits: 0
	 touch_misses: 0
	 auth_cmds: 0
	 auth_errors: 0
	 bytes_read: 30
	 bytes_written: 2071
	 limit_maxbytes: 67108864
	 accepting_conns: 1
	 listen_disabled_num: 0
	 threads: 4
	 conn_yields: 0
	 hash_power_level: 16
	 hash_bytes: 524288
	 hash_is_expanding: 0
	 bytes: 0
	 curr_items: 0
	 total_items: 0
	 expired_unfetched: 0
	 evicted_unfetched: 0
	 evictions: 0
	 reclaimed: 0

5.memcached命令行

安裝telnet包

[root@chenshi ~]# telnet 127.0.0.1 11211
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
set key2 0 30 2
12
STORED

quit退出

格式

<command name> <key> <flags> <exptime> <bytes>\r\n <data block>\r\n

注：\r\n在windows下是Enter鍵

<command name> 能夠是set, add, replace

set表示按照相應的<key>存儲該數據，沒有的時候增長，有的時候覆蓋

add表示按照相應的<key>添加該數據,可是若是該<key>已經存在則會操做失敗

replace表示按照相應的<key>替換數據,可是若是該<key>不存在則操做失敗。

<key> 客戶端須要保存數據的key

<flags> 是一個16位的無符號的整數(以十進制的方式表示)。
該標誌將和須要存儲的數據一塊兒存儲,並在客戶端get數據時返回。
客戶端能夠將此標誌用作特殊用途，此標誌對服務器來講是不透明的。

<exptime> 爲過時的時間。
若爲0表示存儲的數據永遠不過時(但可被服務器算法：LRU 等替換)。
若是非0(unix時間或者距離此時的秒數),當過時後,服務器能夠保證用戶得不到該數據(以服務器時間爲標準)。

<bytes> 須要存儲的字節數，當用戶但願存儲空數據時<bytes>能夠爲0

<data block>須要存儲的內容，輸入完成後，最後客戶端須要加上\r\n（直接點擊Enter）做爲結束標誌。

6.memcached數據導出和導入

導出

[root@chenshi ~]# memcached-tool 127.0.0.1:11211 dump > data.txt

導入，使用nc命令

[root@chenshi ~]# nc 127.0.0.1 11211 < data.txt

7.php鏈接memcached

安裝php的memcached擴展

[root@chenshi src]# wget  http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz 
 http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz: 地址缺乏協議類型.
[root@chenshi src]# wget http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz
--2018-08-20 15:17:10--  http://www.apelearn.com/bbs/data/attachment/forum/memcache-2.2.3.tgz
正在解析主機 www.apelearn.com (www.apelearn.com)... 47.104.7.242
正在鏈接 www.apelearn.com (www.apelearn.com)|47.104.7.242|:80... 已鏈接。
已發出 HTTP 請求，正在等待迴應... 200 OK
長度：27366 (27K) [application/octet-stream]
正在保存至: 「memcache-2.2.3.tgz」

100%[============================================================>] 27,366      20.4KB/s 用時 1.3s   

2018-08-20 15:17:11 (20.4 KB/s) - 已保存 「memcache-2.2.3.tgz」 [27366/27366])

[root@chenshi src]# ls
apache-tomcat-8.5.32.tar.gz  jdk8-downloads-2133151.html                   nginx-1.8.0.tar.gz
apr-1.6.3                    jdk-8u171-linux-x64.tar.gz                    php-5.6.32
apr-1.6.3.tar.gz             mariadb-10.2.6-linux-glibc_214-x86_64.tar.gz  php-5.6.32.tar.bz2
apr-util-1.6.1               memcache-2.2.3.tgz                            php-7.1.6
apr-util-1.6.1.tar.bz2       mysql-5.6.36-linux-glibc2.5-x86_64.tar.gz     php-7.1.6.tar.bz2
httpd-2.4.33                 mysql-5.6.36.tar.gz                           phpredis-develop
httpd-2.4.33.tar.gz          nginx-1.8.0                                   phpredis-develop.zi

解壓安裝

[root@chenshi src]# tar zxvf memcache-2.2.3.tgz 
package.xml
memcache-2.2.3/config.m4
memcache-2.2.3/config9.m4
memcache-2.2.3/config.w32
memcache-2.2.3/CREDITS
memcache-2.2.3/example.php
memcache-2.2.3/memcache.c
memcache-2.2.3/memcache_queue.c
memcache-2.2.3/memcache_session.c
memcache-2.2.3/memcache_standard_hash.c
memcache-2.2.3/memcache_consistent_hash.c
memcache-2.2.3/memcache.dsp
memcache-2.2.3/php_memcache.h
memcache-2.2.3/memcache_queue.h
memcache-2.2.3/README
[root@chenshi src]# cd memcache-2.2.3/
[root@chenshi memcache-2.2.3]# 

[root@chenshi memcache-2.2.3]# /usr/local/php-fpm/bin/phpize
Configuring for:
PHP Api Version:         20131106
Zend Module Api No:      20131226
Zend Extension Api No:   220131226
您在 /var/spool/mail/root 中有新郵件
[root@chenshi memcache-2.2.3]# ./configure --with-php-config=/usr/local/php-fpm/bin/php-config

make && make install

Installing shared extensions:     /usr/local/php-fpm/lib/php/extensions/no-debug-non-zts-20131226/

編輯php配置文件，增長一行，檢測是否成功加載

[root@chenshi memcache-2.2.3]# vi /usr/local/php-fpm/etc/php.ini 

extension=memcache.so



您在 /var/spool/mail/root 中有新郵件
[root@chenshi memcache-2.2.3]# /usr/local/php-fpm/sbin/php-fpm -m
[PHP Modules]
cgi-fcgi
Core
ctype
curl
date
dom
ereg
exif
fileinfo
filter
ftp
gd
hash
iconv
json
libxml
mbstring
mcrypt
memcache
mysql
mysqli
openssl
pcre
PDO
pdo_mysql
pdo_sqlite
Phar
posix
Reflection
session
SimpleXML
soap
SPL
sqlite3
standard
tokenizer
xml
xmlreader
xmlwriter
zlib

下載測試腳本

[root@chenshi ~]# curl www.apelearn.com/study_v2/.memcache.txt > 1.php 2>/dev/null

實驗成功

[root@chenshi ~]# /usr/local/php-fpm/bin/php 1.php
Get key1 value: This is first value<br>Get key1 value: This is replace value<br>Get key2 value: Array
(
    [0] => aaa
    [1] => bbb
    [2] => ccc
    [3] => ddd
)
<br>Get key1 value: <br>Get key2 value

8.memcached中存儲session

在lamp/lnmp環境下實現

下載測試腳本，每次執行測試腳本在tmp下都會生成一個sess開頭的隨機字符文件

[root@chenshi ~]# wget http://study.lishiming.net/.mem_se.txt
[root@chenshi default]# curl localhost/1.php
1534902467<br><br>1534902467<br><br>0ucudqp5mmlg7k63mb6dd5oou5 
[root@chenshi default]# ls /tmp/
chen.sock                                                                test.log.log-20180814
mysql.sock                                                               test.log.log-20180815
php-fcgi.sock                                                            test.log.log-20180816
sess_0ucudqp5mmlg7k63mb6dd5oou5                                          test.log.log-20180817
systemd-private-944ff3d9ac0e44db867d43c34c49b6a2-chronyd.service-q2Ci44  test.log.log-20180818
test.log.log                                                             test.log.log-20180819
test.log.log-20180811                                                    test.log.log-20180820
test.log.log-20180812                                                    test.log.log-20180821
test.log.log-20180813

須要把這種文件放到memcached中

編輯php.ini文件，註釋掉

;session.save_handler = files

增長
session.save_handler = memcache
session.save_path = "tcp://192.168.1.197:11211"


重啓php

再次訪問 tmp下沒有sess文件

[root@chenshi default]# !curl
curl localhost/1.php
1534902846<br><br>1534902846<br><br>bnk199hp3v0bm7s97og8md8v83 
[root@chenshi default]# ls /tmp/
chen.sock                                                                test.log.log-20180814
mysql.sock                                                               test.log.log-20180815
php-fcgi.sock                                                            test.log.log-20180816
systemd-private-944ff3d9ac0e44db867d43c34c49b6a2-chronyd.service-q2Ci44  test.log.log-20180817
test.log.log                                                             test.log.log-20180818
test.log.log-20180811                                                    test.log.log-20180819
test.log.log-20180812                                                    test.log.log-20180820
test.log.log-20180813                                                    test.log.log-20180821

再次curl查看memcached的導出文件

[root@chenshi default]# memcached-tool 127.0.0.1:11211 dump > data.txt

[root@chenshi default]# cat data.txt 
add v01s4u0c26fuakehmtpd47hf25 0 1534904398 37
TEST|i:1534902959;TEST3|i:1534902959;
add tnit882umslssgh3p7ctpjssm0 0 1534904398 37
TEST|i:1534902958;TEST3|i:1534902958;
add rf55q291m35s2mk4q88c9ed911 0 1534904397 37
TEST|i:1534902957;TEST3|i:1534902957;
add bnk199hp3v0bm7s97og8md8v83 0 1534904286 37
TEST|i:1534902846;TEST3|i:1534902846;