Linux下監控文件系統

時間 2019-11-26

標籤 linux 監控文件系統欄目 Linux 简体版

原文原文鏈接

Linux下監控文件系統

Linux的後臺程序一般在機器沒有問題的狀況下，須要長期運行（好比說數個月，甚至是數年）。可是，程序的配置文件有時候是須要按期做調整。爲了避免影響程序對外服務（不重啓），動態加載配置文件是一種很是常見的需求。經過監控某個文件的建立、刪除和修改等事件，能夠很方便作出對應的動做（好比說reload）。html

1. Linux下監控文件系統的經常使用方法

監控配置文件或配置文件目錄的變化，一種可行的方法是程序啓動的時候記錄下文件（或目錄）的修改時間，週期性檢查（好比說一秒一次）文件是否已經被修改，來決定是否須要從新加載配置文件。node

另外一種更爲優雅的辦法是使用Linux系統從內核層面支持的系統API dnotify、inotify或者fanotify。inotify API提供一個文件描述符，能夠在該文件描述符上註冊對指定的文件或者目錄的文件系統事件（文件刪除、文件修改和文件建立），而後經過read系統調用讀取該文件描述法上的事件。linux

2. 使用stat或fstat監控Linux文件系統

經過週期性地獲取被監控文件的狀態，stat和fstat能夠幫助用戶監控指定文件的狀態。ios

int stat(const char *path, struct stat *buf);
    int fstat(int fd, struct stat *buf);
    struct stat {
           dev_t     st_dev;     /* ID of device containing file */
           ino_t     st_ino;     /* inode number */
           mode_t    st_mode;    /* protection */
           nlink_t   st_nlink;   /* number of hard links */
           uid_t     st_uid;     /* user ID of owner */
           gid_t     st_gid;     /* group ID of owner */
           dev_t     st_rdev;    /* device ID (if special file) */
           off_t     st_size;    /* total size, in bytes */
           blksize_t st_blksize; /* blocksize for file system I/O */
           blkcnt_t  st_blocks;  /* number of 512B blocks allocated */
           time_t    st_atime;   /* time of last access */
           time_t    st_mtime;   /* time of last modification */
           time_t    st_ctime;   /* time of last status change */
   };

文件狀態結構體struct stat的st_mtime字段記錄了path對應的文件的最後修改時間，只用週期性地檢查st_mtime的值，就能夠監控文件是否被修改了。c++

開啓一個單獨的線程，並在線程中註冊一個回調函數，當文件改變了之後，就調用註冊的回調函數從新加載監控的目標文件。一個簡單的例子以下：ubuntu

#include <iostream>
#include <string>
#include <thread>
#include <functional>
#include <ev.h>

#include <sys/stat.h>
#include <sys/types.h>
#include <unistd.h>
#include <string.h>
#include <errno.h>

typedef std::function<void (std::string &)> Notifier;
class ConfWatcher {
public:
    explicit ConfWatcher(const std::string &conf_path, Notifier notifier)
    : conf_path_(conf_path), notifier_(notifier), wait_sec_(1), stop_(false)
    {  }

    bool Init() {
        if (access(conf_path_.c_str(), F_OK) != 0) {
            std::cout << "conf file " << conf_path_ << " doesn't exist" << std::endl;
            return false;
        }
        if (!update_last_mod_time()) {
            return false;
        }
        thread_ = std::thread([this] () {
            std::cout << "starting..." << std::endl;
            time_t now = 0;
            while (!stop_) {
                now = time(nullptr);
                time_t delay = now - last_detect_time_;
                if (delay < wait_sec_) {
                    //std::cout << "sleep " << wait_sec_ - delay << " seconds" << std::endl;
                    sleep(wait_sec_ - delay);
                }
                time_t mod_time = last_mod_time_;
                update_last_mod_time();
                if (mod_time != last_mod_time_) {
                    std::cout << "target file " << conf_path_ <<
                    " has been modified, so reload it" << std::endl;
                    notifier_(conf_path_);
                }
            }
            std::cout << "stopping..." << std::endl;
        });
        return true;
    }
    bool Stop() { stop_ = true; }
    bool Wait() { thread_.join(); }

private:
    bool update_last_mod_time() {
        struct stat f_stat = {0};
        if (stat(conf_path_.c_str(), &f_stat) == -1) {
            std::cout << "update_last_mod_time() failed on conf_file "
                << conf_path_ << " ,error: " << strerror(errno) << std::endl;
            return false;
        }
        last_mod_time_ = f_stat.st_mtime;
        last_detect_time_ = time(nullptr);
        return true;
    }

private:
    std::string conf_path_;
    int wait_sec_;
    bool stop_;
    time_t last_mod_time_;
    time_t last_detect_time_;
    std::thread thread_;
    Notifier notifier_;
};

int main(int argc, char **argv) {
    if (argc != 2) {
        std::cout << "Usage: " << argv[0] << " target-file-path" << std::endl;
        return -1;
    }
    auto notifier = [] (const std::string &conf_file_path) {
        std::cout << "watched file " << conf_file_path
            << " has been modified, maybe you need reload it" << std::endl;
    };
    ConfWatcher watcher(argv[1], notifier);
    if (!watcher.Init()) {
        return -1;
    }
    watcher.Wait();
    return 0;
}

編譯：g++ -std=c++11 -lpthread main.cppcookie

3. dnotify監控Linux文件系統

dnofigy是Linux kernel 2.4.0開始支持的一個系統API。它提供了很是有限的方式來和內核交互，以便獲取指定的目錄下文件的修改事件。dnotify的文件監控室經過fcntl的F_NOTIFY選項來實現的。所以它須要使用一個用open系統調用返回的一個文件描述符。可監控的事件有：文件讀取、文件建立、文件內容修改、文件屬性修改和文件重命名。dnotify的事件一般是經過SIGIO信號來獲得通知的。函數

dnotify有如下缺陷，所以使用起來不是很方便：沒法監控單個文件；被監控文件夾所在的文件系統分區在進程退出前沒法卸載；文件事件發生後，須要調用stat系統調用來檢查究竟是那個文件被修改了。oop

4. 使用inotify監控Linux文件系統

inotify是Linux 2.6.13引入的一個inode監控系統。它的API提供了監控單個文件或者目錄的機制，當監控目錄時，它能夠返回目錄自己的事件和目錄下文件的事件。所以inotify能夠徹底替代dnotify。ui

inotigy能夠監控如下事件（在此只列出部分，更詳細信息能夠man 7 inotify查看）：

IN_ACCESS         File was accessed (read) (*). 
IN_ATTRIB         Metadata changed, e.g., permissions, timestamps, extended attributes, link count (since Linux 2.6.25), UID, GID, etc. (*).
IN_CLOSE_WRITE    File opened for writing was closed (*).
IN_CLOSE_NOWRITE  File not opened for writing was closed (*).
IN_CREATE         File/directory created in watched directory (*).
IN_DELETE         File/directory deleted from watched directory (*).
IN_DELETE_SELF    Watched file/directory was itself deleted.
IN_MODIFY         File was modified (*).
IN_MOVE_SELF      Watched file/directory was itself moved.
IN_MOVED_FROM     Generated for the directory containing the old filename when a file is renamed (*).
IN_MOVED_TO       Generated for the directory containing the new filename when a file is renamed (*).
IN_OPEN           File was opened (*).

inotify的接口inotify_init()返回一個文件描述符，而後調用inotify_add_watch()來添加監控的目錄或文件及其對應的感興趣的事件。對於不感興趣的文件或目錄，可使用inotify_rm_watch()來取消關注。註冊了文件監控後，可使用select、poll或者epoll來檢測inotify_init()返回的文件描述符是否有可讀事件，而後使用read系統調用將可讀事件讀取到struct inotify_event裏。

struct inotify_event {
    int      wd;       /* Watch descriptor */
    uint32_t mask;     /* Mask of events */
    uint32_t cookie;   /* Unique cookie associating related events (for rename(2)) */
    uint32_t len;      /* Size of name field */
    char     name[];   /* Optional null-terminated name */
};

正常狀況下，read能夠讀取到若干個struct inotify_event，而後逐個遍歷就能知道是哪一個文件（或者目錄）上發生了什麼事件。下面是一個簡單的使用方法：

//#pragma once
#include <stdlib.h>
#include <errno.h>
#include <string.h>
#include <sys/types.h>
#include <unistd.h>
#include <sys/inotify.h>
#include <string>
#include <unordered_map>
#include <vector>
#include <iostream>
#include <thread>
#include <ev.h>

struct fs_io {
    struct ev_io io;
    void * data;
};

class FsMonitor {
public:
    ~FsMonitor() {
        if (fd_ != -1) {
            close(fd_);
        }
    }

    // register file/directory path and event mask.
    bool Init(const std::vector<std::pair<std::string,uint32_t>> &item_list);
    bool Run();
    bool Stop();

private:
    static void HandleEvents(struct ev_loop *loop, struct ev_io *watcher, int revents);
    bool RemoveItem(const std::string &item_name);
    bool ModifyItem(const std::string &item_name, uint32_t mask);

private:
    std::unordered_map<std::string,int> item_wd_;
    int fd_;

    struct ev_loop *loop_;
    struct fs_io watcher_;
    std::thread thread_;
};


#define EVENT_SIZE  (sizeof (struct inotify_event))
#define BUF_LEN     (1024 * (EVENT_SIZE + 16))

bool FsMonitor::Init(const std::vector<std::pair<std::string,uint32_t>> &item_list)
{
    fd_ = inotify_init1(IN_NONBLOCK);
    if (fd_ < 0) {
        perror("inotify_init1");
        return false;
    }

    for (auto item : item_list) {
        int wd = inotify_add_watch(fd_, item.first.c_str(), item.second);
        if (wd == -1) {
            std::cout << "inotify_add_watch() item " << item.first << " failed:" << strerror(errno)  << std::endl;
            return false;
        }
        item_wd_.insert(std::pair<std::string,int>(item.first, wd));
    }

    loop_ = EV_DEFAULT;

    watcher_.data = this;
    ev_io_init (&watcher_.io, HandleEvents, fd_, EV_READ);
    ev_io_start (loop_, &watcher_.io);

    return true;
}

bool FsMonitor::Run()
{
    thread_ = std::thread([this] () {
        ev_run(loop_, 0);
    });
    return true;
}

bool FsMonitor::Stop()
{
    ev_break(loop_, EVBREAK_ALL);
    return true;
}

void FsMonitor::HandleEvents(struct ev_loop *loop, struct ev_io *watcher, int revents)
{
    struct fs_io * fs_watcher = (struct fs_io *)watcher;
    FsMonitor * fs_monitor = (FsMonitor *)fs_watcher->data;

    int length, i = 0;
    char buffer[BUF_LEN];
    length = read(fs_monitor->fd_, buffer, BUF_LEN);
    if (length < 0) {
        perror("read");
        return;
    }

    while (i < length) {
        struct inotify_event *event = (struct inotify_event *) &buffer[i];
        if (event->len) {
            if (event->mask & IN_CREATE) {
                if (event->mask & IN_ISDIR) {
                    std::cout <<"The directory " << event->name <<" was created." << std::endl;
                }
                else {
                    std::cout <<"The file " << event->name << " was created." << std::endl;
                }
            }
            else if (event->mask & IN_DELETE) {
                if (event->mask & IN_ISDIR) {
                    std::cout <<"The directory " << event->name << " was deleted." << std::endl;
                }
                else {
                    std::cout <<"The file " << event->name << " was deleted." << std::endl;
                }
            }
            else if (event->mask & IN_MODIFY) {
                if (event->mask & IN_ISDIR) {
                    std::cout <<"The directory " << event->name << " was modified." << std::endl;
                }
                else {
                    std::cout <<"The file " << event->name << " was modified." << std::endl;
                }
            }
        }
        i += EVENT_SIZE + event->len;
    }
}

bool FsMonitor::RemoveItem(const std::string &item_name) {
    auto find_item = item_wd_.find(item_name);
    if (find_item == item_wd_.end()) {
        std::cout << "no item " << item_name << " to remove" << std::endl;
        return false;
    }
    int wd = find_item->second;
    inotify_rm_watch(fd_, wd);
    return true;
}

bool FsMonitor::ModifyItem(const std::string &item_name, uint32_t mask) {
    // remove and add
    return true;
}

int main(int argc, char **argv) {
    FsMonitor fs_monitor;
    std::vector<std::pair<std::string,uint32_t>> item_list;
    item_list.emplace_back(std::pair<std::string,uint32_t>("/tmp", IN_MODIFY | IN_CREATE | IN_DELETE));
    if (fs_monitor.Init(item_list) == false) {
        return -1;
    }
    std::cout << "Starting..." << std::endl;
    fs_monitor.Run();
    while (true) {
        usleep(1000 * 1000);
    }
    std::cout << "End..." << std::endl;

    return 0;
}