經過UDFS實現Memcached與MySQL的自動更新

本文出自：http://www.mpyun.com/

UDFs是User Defined Functions的縮寫，表示MYSQL的用戶定義函數，應用程序能夠利用這些函數從MYSQL5.0以上版本的數據庫中訪問Memcached寫入或者獲取的數據。此外，MYSQL從5.1版本開始支持觸發器，從而能夠在觸發器中使用UDFs直接更新Memcached的內容，這種方式下降了應用程序設計和編寫的複雜性。

安裝UDFs須要在數據庫上安裝兩個包，分別是libmemcached和memcached_functions_mysql，而且保證Mysql爲5.1及以上版本，可經過mysql --version命令查看當前 Mysql版本。

libmemcached下載能夠訪問其官方站：
http://download.tangent.org/

memcached_functions_mysql下載地址：

http://patg.net/downloads/

第一次安裝用的是libmemcached-0.53.tar.gz，編譯memcached_functions_mysql-0.9時遇到了下面的問題：

servers.c: In function 'memc_servers_set':

servers.c:122: error: 'memcached_st' has no member named 'hosts'

servers.c:123: error: 'memcached_st' has no member named 'hosts'

servers.c:124: error: 'memcached_st' has no member named 'hosts'

最後用libmemcached-0.34.tar.gz，就沒問題了，多是軟件的兼容性問題吧。

[root@bogon home]# tar -zxvf libmemcached-0.34.tar.gz

[root@bogon home]# cd libmemcached-0.34

[root@localhost libmemcached-0.34]#  ./configure --prefix=/usr/local/libmemcached34 --with-memcached=/usr/local/bin/memcached

[root@localhost libmemcached-0.34]# make && make install

[root@bogon home]# tar zxvf memcached_functions_mysql-0.9.tar.gz

[root@localhost memcached_functions_mysql-0.9]# ./configure --prefix=/usr/local/memcache_mysql --with-mysql=/usr/local/mysql/bin/mysql_config --with-libmemcached=/usr/local/libmemcached34 #/usr/local/mysql爲MySQL安裝目錄

加上--with-libmemcached=/usr/local/libmemcached53否則會報如下錯誤

checking for libmemcached >= 0.17... configure: error: libmemcached not found

[root@localhost memcached_functions_mysql-0.9]# make && make install

安裝完畢。

首次須要使用CREATE FUNCTION來初始化用戶定義函數。有兩種方法能夠初始化全部Mysql提供的用戶定義函數：

1),在Mysql的SQL命令行中執行memcached_functions_mysql源碼目錄下的sql/install_functions.sql。

2), 運行memcached_functions_mysql源碼目錄下的utils/install.pl這個Perl腳本，把memcache function做爲UDFs加入MySQL。

這裏採用第一種辦法：

mysql> use mysql;

mysql> source /home/memcached_functions_mysql-1.1/sql/install_functions.sql

出現以下錯誤：

ERROR 1126 (HY000): Can't open shared library 'libmemcached_functions_mysql.so' (errno: 22 /usr/local/mysql/lib/mysql/plugin/libmemcached_functions_mysql.so: cannot open shared object file: No such file or directory)

解決辦法手動拷貝libmemcached_functions_mysql.so庫到 Mysql安裝目錄：

[root@localhost lib]# cp /usr/local/memcache_mysql/lib/libmemcached_functions_mysql.so* /usr/local/mysql/lib/mysql/plugin/

2.查看安裝是否成功：

mysql> select name,dl from mysql.func;

+------------------------------+---------------------------------+

| name                         | dl                              |

+------------------------------+---------------------------------+

| memc_add                     | libmemcached_functions_mysql.so |

| memc_add_by_key              | libmemcached_functions_mysql.so |

| memc_servers_set             | libmemcached_functions_mysql.so |

| memc_server_count            | libmemcached_functions_mysql.so |

| memc_set                     | libmemcached_functions_mysql.so |

| memc_set_by_key              | libmemcached_functions_mysql.so |

| memc_cas                     | libmemcached_functions_mysql.so |

| memc_cas_by_key              | libmemcached_functions_mysql.so |

| memc_get                     | libmemcached_functions_mysql.so |

| memc_get_by_key              | libmemcached_functions_mysql.so |

| memc_delete                  | libmemcached_functions_mysql.so |

| memc_delete_by_key           | libmemcached_functions_mysql.so |

| memc_append                  | libmemcached_functions_mysql.so |

| memc_append_by_key           | libmemcached_functions_mysql.so |

| memc_prepend                 | libmemcached_functions_mysql.so |

| memc_prepend_by_key          | libmemcached_functions_mysql.so |

| memc_increment               | libmemcached_functions_mysql.so |

| memc_decrement               | libmemcached_functions_mysql.so |

| memc_replace                 | libmemcached_functions_mysql.so |

| memc_replace_by_key          | libmemcached_functions_mysql.so |

| memc_servers_behavior_set    | libmemcached_functions_mysql.so |

| memc_servers_behavior_get    | libmemcached_functions_mysql.so |

| memc_behavior_set            | libmemcached_functions_mysql.so |

| memc_behavior_get            | libmemcached_functions_mysql.so |

| memc_list_behaviors          | libmemcached_functions_mysql.so |

| memc_list_hash_types         | libmemcached_functions_mysql.so |

| memc_list_distribution_types | libmemcached_functions_mysql.so |

| memc_udf_version             | libmemcached_functions_mysql.so |

| memc_libmemcached_version    | libmemcached_functions_mysql.so |

| memc_stats                   | libmemcached_functions_mysql.so |

| memc_stat_get_keys           | libmemcached_functions_mysql.so |

| memc_stat_get_value          | libmemcached_functions_mysql.so |

+------------------------------+---------------------------------+

32 rows in set (0.08 sec)

3.memcached_functions_mysql應用實例：

1).新建兩張表：urls和results,更新urls表中的內容，使系統自動更新Memcached的內容。results用來就更新Memcached失敗的記錄。

SQL代碼：

use test;

drop table if exists urls;

CREATE TABLE urls(

id int(10) NOT NULL,

url varchar(255) NOT NULL DEFAULT '',

PRIMARY KEY (id)

);

drop table if exists results;

CREATE TABLE results(

id int(10) NOT NULL,

results varchar(255) NOT NULL DEFAULT 'error',

time timestamp NULL DEFAULT CURRENT_TIMESTAMP,

PRIMARY KEY (id)

);

2).創建3個trigger

當向urls表中插入數據時，對Memcached執行set操做。trigger的代碼以下：

insert操做：

DELIMITER //

DROP TRIGGER IF EXISTS url_mem_insert;

CREATE TRIGGER url_mem_insert

BEFORE INSERT ON urls

FOR EACH ROW BEGIN

set @mm = memc_set(NEW.id,NEW.url);

if @mm <> 0 then

insert into results(id) values(NEW.id);

end if;

END //

DELIMITER;

更新時對Memcached執行replace操做：

DELIMITER //

DROP TRIGGER IF EXISTS url_mem_update;

CREATE TRIGGER url_mem_update

BEFORE UPDATE ON urls

FOR EACH ROW BEGIN

set @mm = memc_replace(OLD.id,NEW.url);

if @mm <> 0 then

insert into results(id) values(OLD.id);

end if;

END //

DELIMITER;

刪除對Memcached執行delete操做：

DELIMITER //

DROP TRIGGER IF EXISTS url_mem_delete;

CREATE TRIGGER url_mem_delete

BEFORE DELETE ON urls

FOR EACH ROW BEGIN

set @mm = memc_delete(OLD.id);

if @mm <> 0 then

insert into results(id) values(OLD.id);

end if;

END //

DELIMITER;

3).設置Memcached相關參數

設置UDFs操做Memcached服務器的IP地址和端口:

mysql> select memc_servers_set('192.168.56.2:11211');

+----------------------------------------+

| memc_servers_set('192.168.56.2:11211') |

+----------------------------------------+

|                                      0 |

+----------------------------------------+

1 row in set (0.01 sec)

mysql> select memc_server_count();

+---------------------+

| memc_server_count() |

+---------------------+

|                   1 |

+---------------------+

1 row in set (0.01 sec)

在MySQL命令行中列出能夠修改Memcache參數的行爲，以下：

mysql> select memc_list_behaviors()\G

*************************** 1. row ***************************

memc_list_behaviors():

MEMCACHED SERVER BEHAVIORS

MEMCACHED_BEHAVIOR_SUPPORT_CAS

MEMCACHED_BEHAVIOR_NO_BLOCK

MEMCACHED_BEHAVIOR_TCP_NODELAY

MEMCACHED_BEHAVIOR_HASH

MEMCACHED_BEHAVIOR_CACHE_LOOKUPS

MEMCACHED_BEHAVIOR_SOCKET_SEND_SIZE

MEMCACHED_BEHAVIOR_SOCKET_RECV_SIZE

MEMCACHED_BEHAVIOR_BUFFER_REQUESTS

MEMCACHED_BEHAVIOR_KETAMA

MEMCACHED_BEHAVIOR_POLL_TIMEOUT

MEMCACHED_BEHAVIOR_RETRY_TIMEOUT

MEMCACHED_BEHAVIOR_DISTRIBUTION

MEMCACHED_BEHAVIOR_BUFFER_REQUESTS

MEMCACHED_BEHAVIOR_USER_DATA

MEMCACHED_BEHAVIOR_SORT_HOSTS

MEMCACHED_BEHAVIOR_VERIFY_KEY

MEMCACHED_BEHAVIOR_CONNECT_TIMEOUT

MEMCACHED_BEHAVIOR_KETAMA_WEIGHTED

MEMCACHED_BEHAVIOR_KETAMA_HASH

MEMCACHED_BEHAVIOR_BINARY_PROTOCOL

MEMCACHED_BEHAVIOR_SND_TIMEOUT

MEMCACHED_BEHAVIOR_RCV_TIMEOUT

MEMCACHED_BEHAVIOR_SERVER_FAILURE_LIMIT

MEMCACHED_BEHAVIOR_IO_MSG_WATERMARK

MEMCACHED_BEHAVIOR_IO_BYTES_WATERMARK

1 row in set (0.00 sec)

設置MEMCACHED_BEHAVIOR_NO_BLOCK爲打開狀態，這樣在Memcached出現問題（不能鏈接）時，數據能夠繼續插入MySQL中，但有報錯提示：若是不設置此值，那麼Memcached失敗時，數據要等到Memcached失敗超時後才能插入到表中。

經過下面的設置，能夠避免這種狀況的發生。

mysql> select memc_servers_behavior_set('MEMCACHED_BEHAVIOR_NO_BLOCK','1');

+--------------------------------------------------------------+

| memc_servers_behavior_set('MEMCACHED_BEHAVIOR_NO_BLOCK','1') |

+--------------------------------------------------------------+

|                                                            0 |

+--------------------------------------------------------------+

1 row in set (0.01 sec)

mysql> select memc_servers_behavior_set('MEMCACHED_BEHAVIOR_TCP_NODELAY','1');

+-----------------------------------------------------------------+

| memc_servers_behavior_set('MEMCACHED_BEHAVIOR_TCP_NODELAY','1') |

+-----------------------------------------------------------------+

|                                                               0 |

+-----------------------------------------------------------------+

1 row in set (0.01 sec)

4).對memcached_functions_mysql的功能進行測試

向urls表中插入數據，而後查看Memcached是否對數據執行set操做:

mysql> use test;

Reading table information for completion of table and column names

You can turn off this feature to get a quicker startup with -A

Database changed

mysql> insert into urls(id,url) values (1,'http://www.test.com.cn');

Query OK, 1 row affected (0.09 sec)

mysql> select memc_get('1');

+------------------------+

| memc_get('1')          |

+------------------------+

| http://www.test.com.cn |

+------------------------+

1 row in set (0.01 sec)

[root@localhost ~]# telnet 192.168.56.2 11211

Trying 192.168.56.2...

Connected to 192.168.56.2 (192.168.56.2).

Escape character is '^]'.

get 1

VALUE 1 0 22

http://www.test.com.cn

END

更新測試：

mysql> update test.urls set url='http://blog.test.com.cn' where id=1;

Query OK, 1 row affected (0.05 sec)

Rows matched: 1  Changed: 1  Warnings: 0

mysql> select memc_get('1');

+-------------------------+

| memc_get('1')           |

+-------------------------+

| http://blog.test.com.cn |

+-------------------------+

1 row in set (0.01 sec)

[root@localhost ~]#  telnet 192.168.56.2 11211

Trying 192.168.56.2...

Connected to 192.168.56.2 (192.168.56.2).

Escape character is '^]'.

get 1

VALUE 1 0 23

http://blog.test.com.cn

END

刪除測試：

mysql> delete from test.urls where id=1;

Query OK, 1 row affected (0.00 sec)

mysql> select memc_get('1');

+---------------+

| memc_get('1') |

+---------------+

| NULL          |

+---------------+

1 row in set (0.01 sec)

[root@localhost ~]#  telnet 192.168.56.2 11211

Trying 192.168.56.2...

Connected to 192.168.56.2 (192.168.56.2).

Escape character is '^]'.

get 1

END