【原創】源碼安裝 Atlas-1.0.3 遇到的問題以及解決辦法

時間 2019-11-10

標籤原創源碼安裝 atlas 1.0.3 遇到問題以及解決辦法简体版

原文原文鏈接

今天 360 團隊在 GitHub 上發佈了 Atlas-2.0.0 版本。可喜可賀，但願 360 的開發團隊可以一直保持下去。藉此時機，將以前對 Atlas-1.0.3 的一些理解和使用，以博客形式總結一下。本文主要講一下在源碼安裝和剛開始運行 Atlas-1.0.3 時可能遇到的問題。

=====

按照安裝說明，Atlas-1.0.3 的安裝有以下依賴項：
glib(2.32.x)、libevent(1.4以上)、Lua(5.1.x)、OpenSSL(0.9.8以上)

基本執行命令以下：
./bootstrap.sh #可能須要修改其中的路徑
make
sudo make install

依賴項的安裝不作詳細說明，假設已經所有安裝好了。

=====

1. 執行 bootstrap.sh
下面是源碼包中提供的 bootstrap.sh 腳本的內容：

[root@Betty Atlas-1.0.3]# vi bootstrap.sh 

#!/bin/sh 
base=$(cd "$(dirname "$0")"; pwd)
cd $base
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --with-mysql=/usr --prefix=/usr/local/mysql-proxy CFLAGS="-DHAVE_LUA_H -O2" LDFLAGS="-lm -ldl -lcrypto"
LUA_CFLAGS="-I/usr/local/include/" LUA_LIBS="-L/usr/local/lib -llua"

運行該腳本會報出以下的錯誤信息： python

[root@Betty Atlas-1.0.3]# ./bootstrap.sh
checking build system type... x86_64-unknown-linux-gnu
checking host system type... x86_64-unknown-linux-gnu
checking target system type... x86_64-unknown-linux-gnu
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... /bin/mkdir -p
checking for gawk... gawk
checking whether make sets $(MAKE)... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking for style of include used by make... GNU
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking dependency style of gcc... gcc3
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking minix/config.h usability... no
checking minix/config.h presence... no
checking for minix/config.h... no
checking whether it is safe to define __EXTENSIONS__... yes
checking for library containing strerror... none required
checking for gcc... (cached) gcc
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
checking dependency style of gcc... (cached) gcc3
checking for a sed that does not truncate output... /bin/sed
checking for fgrep... /bin/grep -F
checking for ld used by gcc... /usr/bin/ld
checking if the linker (/usr/bin/ld) is GNU ld... yes
checking for a BSD-compatible install... /usr/bin/install -c
checking for gawk... (cached) gawk
checking for flex... flex
checking lex output file root... lex.yy
checking lex library... -lfl
checking whether yytext is a pointer... yes
checking how to run the C preprocessor... gcc -E
checking whether ln -s works... no, using cp -p
checking for a sed that does not truncate output... (cached) /bin/sed
checking whether make sets $(MAKE)... (cached) yes
checking whether gcc and cc understand -c and -o together... yes
checking for function prototypes... yes
checking for string.h... (cached) yes
checking whether byte ordering is bigendian... no
checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B
checking the name lister (/usr/bin/nm -B) interface... BSD nm
checking the maximum length of command line arguments... 98304
checking whether the shell understands some XSI constructs... yes
checking whether the shell understands "+="... yes
checking for /usr/bin/ld option to reload object files... -r
checking for objdump... objdump
checking how to recognize dependent libraries... pass_all
checking for ar... ar
checking for strip... strip
checking for ranlib... ranlib
checking command to parse /usr/bin/nm -B output from gcc object... ok
checking for dlfcn.h... yes
checking for objdir... .libs
checking if gcc supports -fno-rtti -fno-exceptions... no
checking for gcc option to produce PIC... -fPIC -DPIC
checking if gcc PIC flag -fPIC -DPIC works... yes
checking if gcc static flag -static works... yes
checking if gcc supports -c -o file.o... yes
checking if gcc supports -c -o file.o... (cached) yes
checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking whether -lc should be explicitly linked in... no
checking dynamic linker characteristics... GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
checking whether to build shared libraries... yes
checking whether to build static libraries... no
checking shared library path variable... LD_LIBRARY_PATH
checking lua module suffix... so
checking plugin suffix... so
checking for ANSI C header files... (cached) yes
checking for sys/wait.h that is POSIX.1 compatible... yes
checking arpa/inet.h usability... yes
checking arpa/inet.h presence... yes
checking for arpa/inet.h... yes
checking netinet/in.h usability... yes
checking netinet/in.h presence... yes
checking for netinet/in.h... yes
checking sys/filio.h usability... no
checking sys/filio.h presence... no
checking for sys/filio.h... no
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking sys/un.h usability... yes
checking sys/un.h presence... yes
checking for sys/un.h... yes
checking sys/uio.h usability... yes
checking sys/uio.h presence... yes
checking for sys/uio.h... yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking sys/resource.h usability... yes
checking sys/resource.h presence... yes
checking for sys/resource.h... yes
checking pwd.h usability... yes
checking pwd.h presence... yes
checking for pwd.h... yes
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking libproc.h usability... no
checking libproc.h presence... no
checking for libproc.h... no
checking valgrind/valgrind.h usability... no
checking valgrind/valgrind.h presence... no
checking for valgrind/valgrind.h... no
checking syslog.h usability... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking for sys/time.h... (cached) yes
checking time.h usability... yes
checking time.h presence... yes
checking for time.h... yes
checking asm/msr.h usability... no
checking asm/msr.h presence... yes
configure: WARNING: asm/msr.h: present but cannot be compiled
configure: WARNING: asm/msr.h: check for missing prerequisite headers?
configure: WARNING: asm/msr.h: see the Autoconf documentation
configure: WARNING: asm/msr.h: section "Present But Cannot Be Compiled"
configure: WARNING: asm/msr.h: proceeding with the preprocessor's result
configure: WARNING: asm/msr.h: in the future, the compiler will take precedence
configure: WARNING: ## ------------------------------------------------------ ##
configure: WARNING: ## Report this to mysql-proxy-discuss@lists.launchpad.net ##
configure: WARNING: ## ------------------------------------------------------ ##
checking for asm/msr.h... yes
checking sys/timeb.h usability... yes
checking sys/timeb.h presence... yes
checking for sys/timeb.h... yes
checking sys/times.h usability... yes
checking sys/times.h presence... yes
checking for sys/times.h... yes
checking ia64intrin.h usability... no
checking ia64intrin.h presence... no
checking for ia64intrin.h... no
checking for rdtscll... no
checking for ftime... yes
checking for times... yes
checking for clock_gettime... no
checking for get_hrtime... no
checking for read_real_time... no
checking for gettimeofday... yes
checking for time... yes
checking for wait4... yes
checking cc is Sun CC... no
checking for an ANSI C-conforming const... yes
checking for inline... inline
checking whether char is unsigned... no
checking return type of signal handlers... void
checking whether lstat dereferences a symlink specified with a trailing slash... no
checking whether stat accepts an empty string... no
checking for strftime... yes
checking for library containing socket... none required
checking for library containing gethostbyname... none required
checking for library containing hstrerror... none required
checking for socklen_t... yes
checking for ulong... yes
checking for ulong_t... no
checking for MySQL support... yes
checking for MySQL includes at... -I/usr/include/mysql
checking errmsg.h usability... no
checking errmsg.h presence... no
checking for errmsg.h... no
checking mysql.h usability... no
checking mysql.h presence... no
checking for mysql.h... no
configure: error: mysql.h is required, please install the mysql header package
[root@Betty Atlas-1.0.3]#

失敗緣由：找不到 mysql.h 。
解決辦法：在 bootstrap.sh 文件中添加 CPPFLAGS="-I/usr/local/mysql/include/" 。 mysql

修改後 bootstrap.sh 文件以下：

[root@Betty Atlas-1.0.3]# vi bootstrap.sh  

#!/bin/sh 
base=$(cd "$(dirname "$0")"; pwd)
cd $base
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --with-mysql=/usr --prefix=/usr/local/mysql-proxy CFLAGS="-DHAVE_LUA_H -O2" CPPFLAGS="-I/usr/local/mysql/include/" LDFLAGS="-lm -ldl -lcrypto" LUA_CFLAGS="-I/usr/local/include/" LUA_LIBS="-L/usr/local/lib -llua"

從新執行 bootstrap.sh 成功（即執行常規操做中的 configure ）。 linux

2. 執行 make

[root@Betty Atlas-1.0.3]# make
...
libtool: link: gcc -shared  .libs/libproxy_la-proxy-plugin.o   -Wl,-rpath -Wl,/root/workstation/mnt/Atlas-1.0.3/src/.libs -Wl,-rpath -Wl,/usr/local/lib -Wl,-rpath -Wl,/usr/local/mysql-proxy/lib -Wl,-rpath -Wl,/usr/local/lib -L/root/workstation/mnt/Atlas-1.0.3/src/.libs -L/usr/local/lib -L/usr/lib64/mysql -lmysqlclient -lz -lcrypt -L/usr/lib64 -lssl ../../src/.libs/libmysql-proxy.so /root/workstation/mnt/Atlas-1.0.3/src/.libs/libmysql-chassis.so ../../src/.libs/libsql-tokenizer.so /usr/local/lib/libevent.so -lnsl -lresolv -llua /usr/local/lib/libgmodule-2.0.so /usr/local/lib/libgthread-2.0.so /root/workstation/mnt/Atlas-1.0.3/src/.libs/libmysql-chassis-timing.so /root/workstation/mnt/Atlas-1.0.3/src/.libs/libmysql-chassis-glibext.so -lm -ldl -lcrypto /usr/local/lib/libglib-2.0.so -lpthread -lrt  -Wl,--export-dynamic -pthread   -pthread -Wl,-soname -Wl,libproxy.so -o .libs/libproxy.so
/usr/bin/ld: cannot find -lmysqlclient
collect2: ld returned 1 exit status
make[3]: *** [libproxy.la] Error 1
make[3]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3/plugins/proxy'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3/plugins'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3'
make: *** [all] Error 2
[root@Betty Atlas-1.0.3]#

失敗緣由：找不到 mysqlclient 庫。
解決辦法：在 bootstrap.sh 文件中修改 LDFLAGS 的內容，添加 "-L/usr/local/mysql/lib/" 。 git

修改後 bootstrap.sh 文件以下：

[root@Betty Atlas-1.0.3]# vi bootstrap.sh 

#!/bin/sh 
base=$(cd "$(dirname "$0")"; pwd)
cd $base
PKG_CONFIG_PATH=/usr/local/lib/pkgconfig ./configure --with-mysql=/usr --prefix=/usr/local/mysql-proxy CFLAGS="-DHAVE_LUA_H -O2" CPPFLAGS="-I/usr/local/mysql
/include/" LDFLAGS="-L/usr/local/mysql/lib/ -lm -ldl -lcrypto" LUA_CFLAGS="-I/usr/local/include/" LUA_LIBS="-L/usr/local/lib -llua"

從新 bootstrap.sh && make 成功。 github

3. 執行 make install

[root@Betty Atlas-1.0.3]# make install
Making install in src
make[1]: Entering directory `/root/workstation/mnt/Atlas-1.0.3/src'
make  install-am installdirs installscript
make[2]: Entering directory `/root/workstation/mnt/Atlas-1.0.3/src'
/bin/sh ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I/usr/include/mysql  -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include   -I/usr/local/include/ -pthread -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include   -pthread -I/usr/local/include/glib-2.0 -I/usr/local/lib/glib-2.0/include   -I../lib/ -I/usr/local/mysql/include/  -DHAVE_LUA_H -O2 -MT libsql_tokenizer_la-sql-tokenizer-keywords.lo -MD -MP -MF .deps/libsql_tokenizer_la-sql-tokenizer-keywords.Tpo -c -o libsql_tokenizer_la-sql-tokenizer-keywords.lo `test -f '../lib/sql-tokenizer-keywords.c' || echo './'`../lib/sql-tokenizer-keywords.c
...
test -z "/usr/local/mysql-proxy/lib/pkgconfig" || /bin/mkdir -p "/usr/local/mysql-proxy/lib/pkgconfig"
 /usr/bin/install -c -m 644 mysql-proxy.pc mysql-chassis.pc '/usr/local/mysql-proxy/lib/pkgconfig'
make[2]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3'
make[1]: Leaving directory `/root/workstation/mnt/Atlas-1.0.3'
[root@Betty Atlas-1.0.3]#

執行成功。 sql

此時你會發現，可執行程序 mysql-proxy 和 shell 腳本 mysql-proxyd 並無安裝到系統的標準可執行文件所在目錄。能夠經過在環境變量 PATH 中添加的方式進行解決。

4. 運行可執行程序 mysql-proxy
嘗試執行 mysql-proxy 看有什麼輸出（如下給出的 log 信息中存在本人添加的調試打印，故與原代碼略有差別）。

[root@Betty ~]# mysql-proxy
2013-11-25 14:41:59: (debug) chassis-stats.c:35: created new global chassis stats at 0x41a3f40
2013-11-25 14:41:59: (debug) chassis-timings.c:124: created new global chassis timer info at 0x41a4dd0
2013-11-25 14:41:59: (debug) mysql-proxy-cli.c:329: [MoDB] print_version = 0    default_file = (NULL)
2013-11-25 14:41:59: (debug) mysql-proxy-cli.c:432: [MoDB] base_dir = /usr/local/mysql-proxy
2013-11-25 14:41:59: (critical) chassis-frontend.c:122: Failed to get log directory, please set by --log-path
2013-11-25 14:41:59: (message) Initiating shutdown, requested from mysql-proxy-cli.c:443
2013-11-25 14:41:59: (message) shutting down normally, exit code is: 1
[root@Betty ~]#

出現一條 critical 日誌，大體意思是「沒法找到保存日誌文件的路徑位置，須要經過 --log-path 進行指定」。而執行 mysql-proxyd 腳本獲得輸出以下： shell

[root@Betty ~]# mysql-proxyd 
Usage: /usr/local/mysql-proxy/bin/mysql-proxyd instance {start|stop|restart|status}

綜上，想要運行起來 Atlas-1.0.3 ，至少要了解如何設置日誌目錄，以及 instance 所表明的含義。查看 GitHub 上的信息，發現能夠經過自定義配置文件解決這兩個問題。個人測試配置文件以下：數據庫

[mysql-proxy]

plugins = admin, proxy

admin-username = moooofly
admin-password = moooofly
admin-lua-script = /usr/local/mysql-proxy/lib/mysql-proxy/lua/admin.lua

proxy-backend-addresses = 127.0.0.1:3306
#proxy-read-only-backend-addresses = 127.0.0.1:3305@1

daemon = true
keepalive = true

event-threads = 4

log-level = debug
log-path = /usr/local/mysql-proxy/log

instance = modb

proxy-address = 0.0.0.0:1234
admin-address = 0.0.0.0:2345

min-idle-connections = 128

#tables = person.mt.id.3
#pwds = user1:+jKsgB3YAG8=, user2:GS+tr4TPgqc=

charset = utf8

#client-ips = 127.0.0.1, 192.168.1
#lvs-ips = 192.168.1.1

上述配置中，不少選項目前不須要啓用，因此都註釋掉了。惟一須要說明的是，Atlas-1.0.3 對配置文件的名字有以下約束：要求配置文件名字和配置文件中的 instance 的名字要保持一致。例如個人配置文件名字爲 modb.cnf ，因此 instance 的名字爲 modb 。 bootstrap

從新使用 mysql-proxyd 腳本啓動服務程序，輸出信息以下：

[root@Betty ~]# mysql-proxyd modb status
MySQL-Proxy of modb is NOT running
[root@Betty ~]# mysql-proxyd modb start
OK: MySQL-Proxy of modb is started
[root@Betty ~]# mysql-proxyd modb status
MySQL-Proxy of modb is running (7186)
MySQL-Proxy of modb is running (7187)
[root@Betty ~]# ps aux|grep mysql-proxy
root      7186  0.0  0.0  57960  1028 ?        S    14:54   0:00 /usr/local/mysql-proxy/bin/mysql-proxy --defaults-file=/usr/local/mysql-proxy/conf/modb.cnf
root      7187  0.0  0.0 111524  2356 ?        Sl   14:54   0:00 /usr/local/mysql-proxy/bin/mysql-proxy --defaults-file=/usr/local/mysql-proxy/conf/modb.cnf
root      7207  0.0  0.0  61180   752 pts/4    S+   14:54   0:00 grep mysql-proxy
[root@Betty ~]#

能夠看出，服務器程序已經正確啓動。此時在 log 目錄下會多出以下文件：服務器

[root@Betty log]# pwd
/usr/local/mysql-proxy/log
[root@Betty log]# 
[root@Betty log]# ll
總計 8
-rw-r----- 1 root root 1233 11-25 14:54 modb.log
-rw------- 1 root root    4 11-25 14:54 modb.pid
-rw-rw-rw- 1 root root    0 11-25 14:54 sql_modb.log
[root@Betty log]#

其中 modb.log 是服務器的通用日誌；sql_modb.log 是經由 Atlas 進行處理、轉發的 sql 日誌。
啓動 Atlas 時 modb.log 中的日誌內容：

[root@Betty log]# tail -f modb.log 
2013-11-25 14:54:06: (message) mysql-proxy 0.8.2 started - instance: modb
2013-11-25 14:54:06: (message) max open file-descriptors = 1024
2013-11-25 14:54:06: (debug) mysql-proxy-cli.c:649: [MoDB] enter mainloop!!!
2013-11-25 14:54:06: (message) proxy listening on port 0.0.0.0:1234
2013-11-25 14:54:06: (message) added read/write backend: 127.0.0.1:3306
2013-11-25 14:54:06: (message) proxy-plugin.c:2929: [MoDB] current charset is set to utf8
2013-11-25 14:54:06: (message) proxy-plugin.c:2932: [MoDB] min_idle_connections is set to 32
2013-11-25 14:54:06: (message) chassis-event-thread.c:319: starting 3 worker threads(not include main thread)
2013-11-25 14:54:06: (message) chassis-unix-daemon.c:136: [angel] we try to keep PID=7187 alive
2013-11-25 14:54:06: (debug) chassis-unix-daemon.c:155: waiting for 7187

經過 MySQL 客戶端經由 Atlas 訪問 MySQL 數據庫時，sql_modb.log 中的日誌內容：

[root@Betty log]# tail -f sql_modb.log 


[11/25/2013 14:58:54] C:172.16.80.111 S:127.0.0.1 OK 0.155 "SET NAMES utf8"
[11/25/2013 14:58:54] C:172.16.80.111 S:127.0.0.1 OK 0.298 "SELECT @@character_set_database, @@collation_database"
[11/25/2013 14:58:54] C:172.16.80.111 S:127.0.0.1 OK 0.471 "SHOW FULL TABLES WHERE Table_type != 'VIEW'"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.124 "SET NAMES utf8"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 19.297 "SHOW TABLE STATUS LIKE 'test'"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.308 "SHOW CREATE TABLE `test`"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 24.835 "SELECT TABLE_NAME, PARTITION_NAME, SUBPARTITION_NAME, PARTITION_METHOD, SUBPARTITION_METHOD, PARTITION_EXPRESSION, SUBPARTITION_EXPRESSION, PARTITION_DESCRIPTION, PARTITION_COMMENT, NODEGROUP, TABLESPACE_NAME FROM information_schema.PARTITIONS WHERE TABLE_SCHEMA LIKE 'python' AND NOT ISNULL(PARTITION_NAME) AND TABLE_NAME LIKE 'test' ORDER BY TABLE_NAME, PARTITION_NAME, PARTITION_ORDINAL_POSITION, SUBPARTITION_ORDINAL_POSITION"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.279 "SHOW CREATE TABLE `test`"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 1.221 "SELECT * FROM `test` LIMIT 0, 1000"
[11/25/2013 14:59:01] C:172.16.80.111 S:127.0.0.1 OK 0.707 "SHOW COLUMNS FROM `python`.`test`"

至此，一個源碼安裝的、可用的、具備基本配置的 Atlas-1.0.3 完成了。