數據庫不定時重啓排錯

問題表現： 數據庫不定時重啓

懷疑：

1，mysql   佔用服務器內存過多， 致使操做系統kill掉了佔用內存最大mysql程序，以保證服務器的穩定運行。
   2，mysql 寫入瞬時寫入過多，致使io不正常，從而被系統重啓。
   3，數據庫被人爲重啓
   4，數據庫被計劃任務重啓

問題排查：
1，查看mysql-error.log，發現一切正常，mysql可以正常的寫入 重啓日誌。 從而可以排除數據庫 core的可能性。
2，從/var/log/messages發現相似日誌

Feb  2 01:27:01 ip-172-30-0-106 kernel: [21129733.556112] audit: type=1400 audit(1517506021.319:806): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=6041 comm="apparmor_parser"Feb  2 01:27:01 ip-172-30-0-106 kernel: [21129733.808389] audit: type=1400 audit(1517506021.571:807): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=6078 comm="apparmor_parser"Feb  2 14:51:52 ip-172-30-0-106 sz[3618]: [root] mysql-bin.010762/ZMODEM: got ZSKIPFeb  2 14:51:52 ip-172-30-0-106 sz[3618]: [root] mysql-bin.010762/ZMODEM: 335 Bytes, 192 BPS

從而懷疑是不是apparmor重啓了數據庫，或者是apparmor沒有配置好相關權限，致使數據庫讀取/ 寫入不了東西，從而重啓了數據庫.. 開放權限之後，幾天後仍是被自動重啓,因此問題仍是沒有被找到。

3, 從 journalctl 命令中查找系統的全部日誌信息發現以下： 系統日誌

Feb 02 01:26:17 ip-172-30-0-106 systemd[1]: Starting Daily apt activities...Feb 02 01:26:30 ip-172-30-0-106 systemd[1]: Reloading.Feb 02 01:26:30 ip-172-30-0-106 systemd[1]: snapd.refresh.timer: Adding 3h 12min 20.171706s random time.Feb 02 01:26:30 ip-172-30-0-106 systemd[1]: Started ACPI event daemon.Feb 02 01:26:30 ip-172-30-0-106 systemd[1]: Stopping MySQL Community Server...Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: Stopped MySQL Community Server.Feb 02 01:27:01 ip-172-30-0-106 audit[6041]: AVC apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=6041 comm="apparmor_parser"Feb 02 01:27:01 ip-172-30-0-106 kernel: audit: type=1400 audit(1517506021.319:806): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=6041 comm="apparmor_parser"Feb 02 01:27:01 ip-172-30-0-106 CRON[6060]: pam_unix(cron:session): session opened for user root by (uid=0)Feb 02 01:27:01 ip-172-30-0-106 CRON[6061]: (root) CMD (/usr/bin/tsar --cron > /dev/null 2>&1)Feb 02 01:27:01 ip-172-30-0-106 CRON[6060]: pam_unix(cron:session): session closed for user rootFeb 02 01:27:01 ip-172-30-0-106 audit[6078]: AVC apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=6078 comm="apparmor_parser"Feb 02 01:27:01 ip-172-30-0-106 kernel: audit: type=1400 audit(1517506021.571:807): apparmor="STATUS" operation="profile_replace" profile="unconfined" name="/usr/sbin/mysqld" pid=6078 comm="apparmor_parser"Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: Reloading.Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: snapd.refresh.timer: Adding 46min 21.323606s random time.Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: Started ACPI event daemon.Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: Reloading.Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: snapd.refresh.timer: Adding 4h 51min 8.862102s random time.Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: Started ACPI event daemon.Feb 02 01:27:01 ip-172-30-0-106 systemd[1]: Starting MySQL Community Server...Feb 02 01:27:06 ip-172-30-0-106 systemd[1]: Started MySQL Community Server.Feb 02 01:27:06 ip-172-30-0-106 systemd[1]: Reloading.Feb 02 01:27:06 ip-172-30-0-106 systemd[1]: snapd.refresh.timer: Adding 4h 32.769528s random time.Feb 02 01:27:06 ip-172-30-0-106 systemd[1]: Started ACPI event daemon.Feb 02 01:27:06 ip-172-30-0-106 systemd[1]: Stopping MySQL Community Server...Feb 02 01:27:10 ip-172-30-0-106 systemd[1]: Stopped MySQL Community Server.Feb 02 01:27:14 ip-172-30-0-106 systemd[1]: Reloading.Feb 02 01:27:14 ip-172-30-0-106 systemd[1]: snapd.refresh.timer: Adding 3h 43min 30.596261s random time.Feb 02 01:27:14 ip-172-30-0-106 systemd[1]: Started ACPI event daemon.Feb 02 01:27:14 ip-172-30-0-106 systemd[1]: Starting MySQL Community Server...Feb 02 01:27:17 ip-172-30-0-106 systemd[1]: Started MySQL Community Server.Feb 02 01:27:21 ip-172-30-0-106 systemd[1]: Started Daily apt activities.Feb 02 01:27:21 ip-172-30-0-106 systemd[1]: apt-daily.timer: Adding 5h 3min 16.296108s random time.Feb 02 01:27:21 ip-172-30-0-106 systemd[1]: apt-daily.timer: Adding 3h 1min 23.389954s random time.
Feb 02 01:26:17 ip-172-30-0-106 systemd[1]: Starting Daily apt activities...

發現每次數據庫重啓以前都有一個這個。
從而懷疑是系統的某些平常檢查，平常更新有影響到

經過命令：
yinni99@ip-172-30-0-106:/var/log/apt$ cat /etc/apt/apt.conf.d/10periodic
APT::Periodic::Update-Package-Lists "1";
APT::Periodic::Download-Upgradeable-Packages "0";
APT::Periodic::AutocleanInterval "0";

肯定系統沒有開啓自動更新。。。
問題又回到了原點！！！ %>_<%

4，不放棄，感受問題就在剛剛日誌裏面，從日誌裏面能夠看到apt activities這幾個關鍵字，百度。發現一堆羅嗦。 懷疑是 apt 自動更新底層依賴庫，致使數據庫重啓，去/var/log/apt/ 查看日誌發現

看着文件更新時間，感受離真相愈來愈近，分別查看history.log 與 term.log 內容以下：

history.log
Start-Date: 2018-02-02  01:26:29Commandline: /usr/bin/unattended-upgradeUpgrade: curl:amd64 (7.47.0-1ubuntu2.5, 7.47.0-1ubuntu2.6), libcurl3-gnutls:amd64 (7.47.0-1ubuntu2.5, 7.47.0-1ubuntu2.6)Error: Sub-process /usr/bin/dpkg returned an error code (1)End-Date: 2018-02-02  01:27:17

term.log

Log started: 2018-02-02  01:26:29(Reading database ... 400278 files and directories currently installed.)Preparing to unpack .../curl_7.47.0-1ubuntu2.6_amd64.deb ...Unpacking curl (7.47.0-1ubuntu2.6) over (7.47.0-1ubuntu2.5) ...Preparing to unpack .../libcurl3-gnutls_7.47.0-1ubuntu2.6_amd64.deb ...Unpacking libcurl3-gnutls:amd64 (7.47.0-1ubuntu2.6) over (7.47.0-1ubuntu2.5) ...Processing triggers for man-db (2.7.5-1) ...Processing triggers for libc-bin (2.23-0ubuntu10) ...Setting up mysql-server-5.7 (5.7.21-0ubuntu0.16.04.1) ...mysql_upgrade: Got error: 1045: Access denied for user 'debian-sys-maint'@'localhost' (using password: YES) while connecting to the MySQL serverUpgrade process encountered error and will not continue.mysql_upgrade failed with exit status 11dpkg: error processing package mysql-server-5.7 (--configure): subprocess installed post-installation script returned error exit status 1dpkg: dependency problems prevent configuration of mysql-server: mysql-server depends on mysql-server-5.7; however:  Package mysql-server-5.7 is not configured yet.dpkg: error processing package mysql-server (--configure): dependency problems - leaving unconfiguredSetting up libcurl3-gnutls:amd64 (7.47.0-1ubuntu2.6) ...Setting up curl (7.47.0-1ubuntu2.6) ...Processing triggers for libc-bin (2.23-0ubuntu10) ...Errors were encountered while processing: mysql-server-5.7 mysql-serverLog ended: 2018-02-02  01:27:17

問題已然肯定，是apt的更新，致使數據庫重啓，查找相關資料：
http://wiki.ubuntu.org.cn/Ubu...:AutomaticSecurityUpdates/zh
https://help.ubuntu.com/commu...

(it's an interactive dialog) which will create /etc/apt/apt.conf.d/20auto-upgrades with the following contents:

打開 /etc/apt/apt.conf.d/20auto-upgrades 修改 APT::Periodic::Unattended-Upgrade "1";選項爲0.

so:最終結果 系統安全更新，影響到mysql