渣渣的 ElasticSearch 源碼解析 —— 啓動流程（上）

時間 2019-12-06

原文原文鏈接

關注我

轉載請務必註明原創地址爲：http://www.54tianzhisheng.cn/2018/08/11/es-code02/java

前提

上篇文章寫了 ElasticSearch 源碼解析 —— 環境搭建，其中裏面說了啓動打開 server 模塊下的 Elasticsearch 類：org.elasticsearch.bootstrap.Elasticsearch，運行裏面的 main 函數就能夠啓動 ElasticSearch 了，這篇文章講講啓動流程，由於篇幅會不少，因此分了兩篇來寫。node

啓動流程

main 方法入口

能夠看到入口實際上是一個 main 方法，方法裏面先是檢查權限，而後是一個錯誤日誌監聽器（確保在日誌配置以前狀態日誌沒有出現 error），而後是 Elasticsearch 對象的建立，而後調用了靜態方法 main 方法（18 行），並把建立的對象和參數以及 Terminal 默認值傳進去。靜態的 main 方法裏面調用 elasticsearch.main 方法。json

public static void main(final String[] args) throws Exception {         //一、入口
    // we want the JVM to think there is a security manager installed so that if internal policy decisions that would be based on the
    // presence of a security manager or lack thereof act as if there is a security manager present (e.g., DNS cache policy)
    System.setSecurityManager(new SecurityManager() {
        @Override
        public void checkPermission(Permission perm) {
            // grant all permissions so that we can later set the security manager to the one that we want
        }
    });
    LogConfigurator.registerErrorListener();                            //
    final Elasticsearch elasticsearch = new Elasticsearch();
    int status = main(args, elasticsearch, Terminal.DEFAULT); //二、調用Elasticsearch.main方法
    if (status != ExitCodes.OK) {
        exit(status);
    }
}

static int main(final String[] args, final Elasticsearch elasticsearch, final Terminal terminal) throws Exception {
    return elasticsearch.main(args, terminal);  //三、command main
}

由於 Elasticsearch 類是繼承了 EnvironmentAwareCommand 類，EnvironmentAwareCommand 類繼承了 Command 類，可是 Elasticsearch 類並無重寫 main 方法，因此上面調用的 elasticsearch.main 實際上是調用了 Command 的 main 方法，代碼以下：bootstrap

/** Parses options for this command from args and executes it. */
public final int main(String[] args, Terminal terminal) throws Exception {
    if (addShutdownHook()) {                                                //利用Runtime.getRuntime().addShutdownHook方法加入一個Hook，在程序退出時觸發該Hook
        shutdownHookThread = new Thread(() -> {
            try {
                this.close();
            } catch (final IOException e) {
                try (
                    StringWriter sw = new StringWriter();
                    PrintWriter pw = new PrintWriter(sw)) {
                    e.printStackTrace(pw);
                    terminal.println(sw.toString());
                } catch (final IOException impossible) {
                    // StringWriter#close declares a checked IOException from the Closeable interface but the Javadocs for StringWriter
                    // say that an exception here is impossible
                    throw new AssertionError(impossible);
                }
            }
        });
        Runtime.getRuntime().addShutdownHook(shutdownHookThread);
    }

    beforeMain.run();

    try {
        mainWithoutErrorHandling(args, terminal);//四、mainWithoutErrorHandling
    } catch (OptionException e) {
        printHelp(terminal);
        terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage());
        return ExitCodes.USAGE;
    } catch (UserException e) {
        if (e.exitCode == ExitCodes.USAGE) {
            printHelp(terminal);
        }
        terminal.println(Terminal.Verbosity.SILENT, "ERROR: " + e.getMessage());
        return e.exitCode;
    }
    return ExitCodes.OK;
}

上面代碼一開始利用一個勾子函數，在程序退出時觸發該 Hook，該方法主要代碼是 mainWithoutErrorHandling() 方法，而後下面的是 catch 住方法拋出的異常，方法代碼以下：安全

/*** Executes the command, but all errors are thrown. */
void mainWithoutErrorHandling(String[] args, Terminal terminal) throws Exception {
    final OptionSet options = parser.parse(args);
    if (options.has(helpOption)) {
        printHelp(terminal);
        return;
    }
    if (options.has(silentOption)) {
        terminal.setVerbosity(Terminal.Verbosity.SILENT);
    } else if (options.has(verboseOption)) {
        terminal.setVerbosity(Terminal.Verbosity.VERBOSE);
    } else {
        terminal.setVerbosity(Terminal.Verbosity.NORMAL);
    }
    execute(terminal, options);//五、執行 EnvironmentAwareCommand 中的 execute()，（重寫了command裏面抽象的execute方法）
}

上面的代碼從 3 ～ 14 行是解析傳進來的參數並配置 terminal，重要的 execute() 方法，執行的是 EnvironmentAwareCommand 中的 execute() （重寫了 Command 類裏面的抽象 execute 方法），從上面那個繼承圖能夠看到 EnvironmentAwareCommand 繼承了 Command，重寫的 execute 方法代碼以下：app

@Override
protected void execute(Terminal terminal, OptionSet options) throws Exception {
    final Map<String, String> settings = new HashMap<>();
    for (final KeyValuePair kvp : settingOption.values(options)) {
        if (kvp.value.isEmpty()) {
            throw new UserException(ExitCodes.USAGE, "setting [" + kvp.key + "] must not be empty");
        }
        if (settings.containsKey(kvp.key)) {
            final String message = String.format(
                Locale.ROOT, "setting [%s] already set, saw [%s] and [%s]",
                kvp.key, settings.get(kvp.key), kvp.value);
            throw new UserException(ExitCodes.USAGE, message);
        }
        settings.put(kvp.key, kvp.value);
    }
    //六、根據咱們ide配置的 vm options 進行設置path.data、path.home、path.logs
    putSystemPropertyIfSettingIsMissing(settings, "path.data", "es.path.data");
    putSystemPropertyIfSettingIsMissing(settings, "path.home", "es.path.home");
    putSystemPropertyIfSettingIsMissing(settings, "path.logs", "es.path.logs");

    execute(terminal, options, createEnv(terminal, settings));//七、先調用 createEnv 建立環境
    //九、執行elasticsearch的execute方法，elasticsearch中重寫了EnvironmentAwareCommand中的抽象execute方法
}

方法前面是根據傳參去判斷配置的，若是配置爲空，就會直接跳到執行 putSystemPropertyIfSettingIsMissing 方法，這裏會配置三個屬性：path.data、path.home、path.logs 設置 es 的 data、home、logs 目錄，它這裏是根據咱們 ide 配置的 vm options 進行設置的，這也是爲何咱們上篇文章說的配置信息，若是不配置的話就會直接報錯。下面看看 putSystemPropertyIfSettingIsMissing 方法代碼裏面怎麼作到的：jvm

/** Ensure the given setting exists, reading it from system properties if not already set. */
private static void putSystemPropertyIfSettingIsMissing(final Map<String, String> settings, final String setting, final String key) {
    final String value = System.getProperty(key);//獲取key（es.path.data）找系統設置
    if (value != null) {
        if (settings.containsKey(setting)) {
            final String message =
                String.format(
                Locale.ROOT,
                "duplicate setting [%s] found via command-line [%s] and system property [%s]",
                setting, settings.get(setting), value);
            throw new IllegalArgumentException(message);
        } else {
            settings.put(setting, value);
        }
    }
}

執行這三個方法後：elasticsearch

跳出此方法，繼續看會發現 execute 方法調用了方法，ide

execute(terminal, options, createEnv(terminal, settings));

這裏咱們先看看 createEnv(terminal, settings) 方法：函數

protected Environment createEnv(final Terminal terminal, final Map<String, String> settings) throws UserException {
    final String esPathConf = System.getProperty("es.path.conf");//八、讀取咱們 vm options 中配置的 es.path.conf
    if (esPathConf == null) {
        throw new UserException(ExitCodes.CONFIG, "the system property [es.path.conf] must be set");
    }
    return InternalSettingsPreparer.prepareEnvironment(Settings.EMPTY, terminal, settings, getConfigPath(esPathConf));  //八、準備環境 prepareEnvironment
}

讀取咱們 ide vm options 中配置的 es.path.conf，同上篇文章也講了這個必定要配置的，由於 es 啓動的時候會加載咱們的配置和一些插件。這裏繼續看下上面代碼第 6 行的 prepareEnvironment 方法：

public static Environment prepareEnvironment(Settings input, Terminal terminal, Map<String, String> properties, Path configPath) {
    // just create enough settings to build the environment, to get the config dir
    Settings.Builder output = Settings.builder();
    initializeSettings(output, input, properties);
    Environment environment = new Environment(output.build(), configPath);

    //查看 es.path.conf 目錄下的配置文件是否是 yml 格式的，若是不是則拋出一個異常
    if (Files.exists(environment.configFile().resolve("elasticsearch.yaml"))) {
        throw new SettingsException("elasticsearch.yaml was deprecated in 5.5.0 and must be renamed to elasticsearch.yml");
    }

    if (Files.exists(environment.configFile().resolve("elasticsearch.json"))) {
        throw new SettingsException("elasticsearch.json was deprecated in 5.5.0 and must be converted to elasticsearch.yml");
    }

    output = Settings.builder(); // start with a fresh output
    Path path = environment.configFile().resolve("elasticsearch.yml");
    if (Files.exists(path)) {
        try {
            output.loadFromPath(path);  //加載文件並讀取配置文件內容
        } catch (IOException e) {
            throw new SettingsException("Failed to load settings from " + path.toString(), e);
        }
    }

    // re-initialize settings now that the config file has been loaded
    initializeSettings(output, input, properties);          //再一次初始化設置
    finalizeSettings(output, terminal);

    environment = new Environment(output.build(), configPath);

    // we put back the path.logs so we can use it in the logging configuration file
    output.put(Environment.PATH_LOGS_SETTING.getKey(), environment.logsFile().toAbsolutePath().normalize().toString());
    return new Environment(output.build(), configPath);
}

準備的環境如上圖，經過構建的環境查看配置文件 elasticsearch.yml 是否是以 yml 結尾，若是是 yaml 或者 json 結尾的則拋出異常（在 5.5.0 版本其餘兩種格式過時了，只能使用 yml 格式），而後加載該配置文件並讀取裏面的內容（KV結構）。

跳出 createEnv 方法，咱們繼續看 execute 方法吧。

EnvironmentAwareCommand 類的 execute 方法代碼以下：

protected abstract void execute(Terminal terminal, OptionSet options, Environment env) throws Exception;

這是個抽象方法，那麼它的實現方法在 Elasticsearch 類中，代碼以下：

@Override
protected void execute(Terminal terminal, OptionSet options, Environment env) throws UserException {
    if (options.nonOptionArguments().isEmpty() == false) {
        throw new UserException(ExitCodes.USAGE, "Positional arguments not allowed, found " + options.nonOptionArguments());
    }
    if (options.has(versionOption)) {
        final String versionOutput = String.format(
            Locale.ROOT,
            "Version: %s, Build: %s/%s/%s/%s, JVM: %s",
            Version.displayVersion(Version.CURRENT, Build.CURRENT.isSnapshot()),
            Build.CURRENT.flavor().displayName(),
            Build.CURRENT.type().displayName(),
            Build.CURRENT.shortHash(),
            Build.CURRENT.date(),
            JvmInfo.jvmInfo().version());
        terminal.println(versionOutput);
        return;
    }

    final boolean daemonize = options.has(daemonizeOption);
    final Path pidFile = pidfileOption.value(options);
    final boolean quiet = options.has(quietOption);

    // a misconfigured java.io.tmpdir can cause hard-to-diagnose problems later, so reject it immediately
    try {
        env.validateTmpFile();
    } catch (IOException e) {
        throw new UserException(ExitCodes.CONFIG, e.getMessage());
    }
    try {
        init(daemonize, pidFile, quiet, env);    //十、初始化
    } catch (NodeValidationException e) {
        throw new UserException(ExitCodes.CONFIG, e.getMessage());
    }
}

上面代碼裏主要仍是看看 init(daemonize, pidFile, quiet, env); 初始化方法吧。

void init(final boolean daemonize, final Path pidFile, final boolean quiet, Environment initialEnv)
    throws NodeValidationException, UserException {
    try {
        Bootstrap.init(!daemonize, pidFile, quiet, initialEnv); //十一、執行 Bootstrap 中的 init 方法
    } catch (BootstrapException | RuntimeException e) {
        // format exceptions to the console in a special way
        // to avoid 2MB stacktraces from guice, etc.
        throw new StartupException(e);
    }
}

init 方法

Bootstrap 中的靜態 init 方法以下：

static void init(
    final boolean foreground,
    final Path pidFile,
    final boolean quiet,
    final Environment initialEnv) throws BootstrapException, NodeValidationException, UserException {
    // force the class initializer for BootstrapInfo to run before
    // the security manager is installed
    BootstrapInfo.init();

    INSTANCE = new Bootstrap();   //十二、建立一個 Bootstrap 實例

    final SecureSettings keystore = loadSecureSettings(initialEnv);//若是註冊了安全模塊則將相關配置加載進來
    final Environment environment = createEnvironment(foreground, pidFile, keystore, initialEnv.settings(), initialEnv.configFile());   //幹以前幹過的事情
    try {
        LogConfigurator.configure(environment);   //1三、log 配置環境
    } catch (IOException e) {
        throw new BootstrapException(e);
    }
    if (environment.pidFile() != null) {
        try {
            PidFile.create(environment.pidFile(), true);
        } catch (IOException e) {
            throw new BootstrapException(e);
        }
    }

    final boolean closeStandardStreams = (foreground == false) || quiet;
    try {
        if (closeStandardStreams) {
            final Logger rootLogger = ESLoggerFactory.getRootLogger();
            final Appender maybeConsoleAppender = Loggers.findAppender(rootLogger, ConsoleAppender.class);
            if (maybeConsoleAppender != null) {
                Loggers.removeAppender(rootLogger, maybeConsoleAppender);
            }
            closeSystOut();
        }

        // fail if somebody replaced the lucene jars
        checkLucene();             //1四、檢查Lucene版本

// install the default uncaught exception handler; must be done before security is initialized as we do not want to grant the runtime permission setDefaultUncaughtExceptionHandler
        Thread.setDefaultUncaughtExceptionHandler(
            new ElasticsearchUncaughtExceptionHandler(() -> Node.NODE_NAME_SETTING.get(environment.settings())));

        INSTANCE.setup(true, environment);      //1五、調用 setup 方法

        try {
            // any secure settings must be read during node construction
            IOUtils.close(keystore);
        } catch (IOException e) {
            throw new BootstrapException(e);
        }

        INSTANCE.start();         //2六、調用 start 方法

        if (closeStandardStreams) {
            closeSysError();
        }
    } catch (NodeValidationException | RuntimeException e) {
        // disable console logging, so user does not see the exception twice (jvm will show it already)
        final Logger rootLogger = ESLoggerFactory.getRootLogger();
        final Appender maybeConsoleAppender = Loggers.findAppender(rootLogger, ConsoleAppender.class);
        if (foreground && maybeConsoleAppender != null) {
            Loggers.removeAppender(rootLogger, maybeConsoleAppender);
        }
        Logger logger = Loggers.getLogger(Bootstrap.class);
        if (INSTANCE.node != null) {
            logger = Loggers.getLogger(Bootstrap.class, Node.NODE_NAME_SETTING.get(INSTANCE.node.settings()));
        }
        // HACK, it sucks to do this, but we will run users out of disk space otherwise
        if (e instanceof CreationException) {
            // guice: log the shortened exc to the log file
            ByteArrayOutputStream os = new ByteArrayOutputStream();
            PrintStream ps = null;
            try {
                ps = new PrintStream(os, false, "UTF-8");
            } catch (UnsupportedEncodingException uee) {
                assert false;
                e.addSuppressed(uee);
            }
            new StartupException(e).printStackTrace(ps);
            ps.flush();
            try {
                logger.error("Guice Exception: {}", os.toString("UTF-8"));
            } catch (UnsupportedEncodingException uee) {
                assert false;
                e.addSuppressed(uee);
            }
        } else if (e instanceof NodeValidationException) {
            logger.error("node validation exception\n{}", e.getMessage());
        } else {
            // full exception
            logger.error("Exception", e);
        }
        // re-enable it if appropriate, so they can see any logging during the shutdown process
        if (foreground && maybeConsoleAppender != null) {
            Loggers.addAppender(rootLogger, maybeConsoleAppender);
        }
        throw e;
    }
}

該方法主要有：

一、建立 Bootstrap 實例

二、若是註冊了安全模塊則將相關配置加載進來

三、建立 Elasticsearch 運行的必須環境以及相關配置, 如將 config、scripts、plugins、modules、logs、lib、bin 等配置目錄加載到運行環境中

四、log 配置環境，建立日誌上下文

五、檢查是否存在 PID 文件，若是不存在，建立 PID 文件

六、檢查 Lucene 版本

七、調用 setup 方法（用當前環境來建立一個節點）

setup 方法

private void setup(boolean addShutdownHook, Environment environment) throws BootstrapException {
    Settings settings = environment.settings();//根據環境獲得配置
    try {
        spawner.spawnNativeControllers(environment);
    } catch (IOException e) {
        throw new BootstrapException(e);
    }
    initializeNatives(
        environment.tmpFile(),
        BootstrapSettings.MEMORY_LOCK_SETTING.get(settings),
        BootstrapSettings.SYSTEM_CALL_FILTER_SETTING.get(settings),
        BootstrapSettings.CTRLHANDLER_SETTING.get(settings));
    // initialize probes before the security manager is installed
    initializeProbes();
    if (addShutdownHook) {
        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
                try {
                    IOUtils.close(node, spawner);
                    LoggerContext context = (LoggerContext) LogManager.getContext(false);
                    Configurator.shutdown(context);
                } catch (IOException ex) {
                    throw new ElasticsearchException("failed to stop node", ex);
                }
            }
        });
    }
    try {
        // look for jar hell
        final Logger logger = ESLoggerFactory.getLogger(JarHell.class);
        JarHell.checkJarHell(logger::debug);
    } catch (IOException | URISyntaxException e) {
        throw new BootstrapException(e);
    }
    // Log ifconfig output before SecurityManager is installed
    IfConfig.logIfNecessary();
    // install SM after natives, shutdown hooks, etc.
    try {
        Security.configure(environment, BootstrapSettings.SECURITY_FILTER_BAD_DEFAULTS_SETTING.get(settings));
    } catch (IOException | NoSuchAlgorithmException e) {
        throw new BootstrapException(e);
    }
    node = new Node(environment) {              //1六、新建節點
        @Override
        protected void validateNodeBeforeAcceptingRequests(
            final BootstrapContext context,
            final BoundTransportAddress boundTransportAddress, List<BootstrapCheck> checks) throws NodeValidationException {
            BootstrapChecks.check(context, boundTransportAddress, checks);
        }
    };
}

上面代碼最後就是 Node 節點的建立，這篇文章就不講 Node 的建立了，下篇文章會好好講一下 Node 節點的建立和正式啓動 ES 節點的。