聊聊hikari鏈接池的fixed pool design

序

本文主要研究一下hikari鏈接池的fixed pool design

fixed pool design

hikari的做者比較傾向於fixed pool design的理念，即建議minimumIdle與maximumPoolSize設置成同樣，當作固定鏈接大小的鏈接池。做者認爲minimumIdle小於maximumPoolSize的話，在流量激增的時候須要額外的鏈接，此時在請求方法裏頭再去處理新建鏈接會形成性能損失，即會致使數據庫一方面下降鏈接創建的速度，另外一方面也會影響既有的鏈接事務的完成，間接影響了這些既有鏈接歸還到鏈接池的速度。

做者認爲minimumIdle與maximumPoolSize設置成同樣，多餘的空閒鏈接不會對總體的性能有什麼嚴重影響，若是說設置minimumIdle小於maximumPoolSize是爲了在必要的時候能夠釋放鏈接以釋放內存給其餘功能用，可是在高峯時期，鏈接池可能也會到達maximumPoolSize，於是這個目的彷佛沒起到效果。

HikariPool.houseKeeperTask

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

private final long HOUSEKEEPING_PERIOD_MS = Long.getLong("com.zaxxer.hikari.housekeeping.periodMs", SECONDS.toMillis(30));

   /**
    * Construct a HikariPool with the specified configuration.
    *
    * @param config a HikariConfig instance
    */
   public HikariPool(final HikariConfig config)
   {
      super(config);

      this.connectionBag = new ConcurrentBag<>(this);
      this.suspendResumeLock = config.isAllowPoolSuspension() ? new SuspendResumeLock() : SuspendResumeLock.FAUX_LOCK;

      this.houseKeepingExecutorService = initializeHouseKeepingExecutorService();

      checkFailFast();

      if (config.getMetricsTrackerFactory() != null) {
         setMetricsTrackerFactory(config.getMetricsTrackerFactory());
      }
      else {
         setMetricRegistry(config.getMetricRegistry());
      }

      setHealthCheckRegistry(config.getHealthCheckRegistry());

      registerMBeans(this);

      ThreadFactory threadFactory = config.getThreadFactory();

      LinkedBlockingQueue<Runnable> addConnectionQueue = new LinkedBlockingQueue<>(config.getMaximumPoolSize());
      this.addConnectionQueue = unmodifiableCollection(addConnectionQueue);
      this.addConnectionExecutor = createThreadPoolExecutor(addConnectionQueue, poolName + " connection adder", threadFactory, new ThreadPoolExecutor.DiscardPolicy());
      this.closeConnectionExecutor = createThreadPoolExecutor(config.getMaximumPoolSize(), poolName + " connection closer", threadFactory, new ThreadPoolExecutor.CallerRunsPolicy());

      this.leakTaskFactory = new ProxyLeakTaskFactory(config.getLeakDetectionThreshold(), houseKeepingExecutorService);

      this.houseKeeperTask = houseKeepingExecutorService.scheduleWithFixedDelay(new HouseKeeper(), 100L, HOUSEKEEPING_PERIOD_MS, MILLISECONDS);
   }

在初始化HikariPool的時候會初始化houseKeeperTask

HikariPool.HouseKeeper

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

/**
    * The house keeping task to retire and maintain minimum idle connections.
    */
   private final class HouseKeeper implements Runnable
   {
      private volatile long previous = plusMillis(currentTime(), -HOUSEKEEPING_PERIOD_MS);

      @Override
      public void run()
      {
         try {
            // refresh timeouts in case they changed via MBean
            connectionTimeout = config.getConnectionTimeout();
            validationTimeout = config.getValidationTimeout();
            leakTaskFactory.updateLeakDetectionThreshold(config.getLeakDetectionThreshold());

            final long idleTimeout = config.getIdleTimeout();
            final long now = currentTime();

            // Detect retrograde time, allowing +128ms as per NTP spec.
            if (plusMillis(now, 128) < plusMillis(previous, HOUSEKEEPING_PERIOD_MS)) {
               LOGGER.warn("{} - Retrograde clock change detected (housekeeper delta={}), soft-evicting connections from pool.",
                           poolName, elapsedDisplayString(previous, now));
               previous = now;
               softEvictConnections();
               return;
            }
            else if (now > plusMillis(previous, (3 * HOUSEKEEPING_PERIOD_MS) / 2)) {
               // No point evicting for forward clock motion, this merely accelerates connection retirement anyway
               LOGGER.warn("{} - Thread starvation or clock leap detected (housekeeper delta={}).", poolName, elapsedDisplayString(previous, now));
            }

            previous = now;

            String afterPrefix = "Pool ";
            if (idleTimeout > 0L && config.getMinimumIdle() < config.getMaximumPoolSize()) {
               logPoolState("Before cleanup ");
               afterPrefix = "After cleanup  ";

               final List<PoolEntry> notInUse = connectionBag.values(STATE_NOT_IN_USE);
               int toRemove = notInUse.size() - config.getMinimumIdle();
               for (PoolEntry entry : notInUse) {
                  if (toRemove > 0 && elapsedMillis(entry.lastAccessed, now) > idleTimeout && connectionBag.reserve(entry)) {
                     closeConnection(entry, "(connection has passed idleTimeout)");
                     toRemove--;
                  }
               }
            }

            logPoolState(afterPrefix);

            fillPool(); // Try to maintain minimum connections
         }
         catch (Exception e) {
            LOGGER.error("Unexpected exception in housekeeping task", e);
         }
      }
   }

假設minimumIdle與maximumPoolSize設置成同樣，那麼這個task在第一次執行的時候，直接執行fillPool

fillPool

/**
    * Fill pool up from current idle connections (as they are perceived at the point of execution) to minimumIdle connections.
    */
   private synchronized void fillPool()
   {
      final int connectionsToAdd = Math.min(config.getMaximumPoolSize() - getTotalConnections(), config.getMinimumIdle() - getIdleConnections())
                                   - addConnectionQueue.size();
      for (int i = 0; i < connectionsToAdd; i++) {
         addConnectionExecutor.submit((i < connectionsToAdd - 1) ? POOL_ENTRY_CREATOR : POST_FILL_POOL_ENTRY_CREATOR);
      }
   }

這個fillPool，在初始化時刻，minimumIdle與maximumPoolSize值同樣，totalConnections與idleConnections都爲0，那麼connectionsToAdd的值就是maximumPoolSize
也就是說這個task會添加maximumPoolSize大小鏈接

小結

• tomcat jdbc pool

有個initial-size參數來指定最開始的時候初始化多少個鏈接，有min-idle及max-idle來控制空閒鏈接的最小值及最大值，有max-active來控制鏈接池總大小。min-evictable-idle-time-millis用來指定空閒鏈接的時長，time-between-eviction-runs-millis用來指定清理空閒鏈接的任務的調度時間間隔。

• hikari connection pool

有minIdle來指定空閒鏈接的最小數量，maxPoolSize指定鏈接池鏈接最大值，默認初始化的時候，是初始化minIdle大小的鏈接，若是minIdle與maxPoolSize值相等那就是初始化時把鏈接池填滿。idleTimeout用來指定空閒鏈接的時長，maxLifetime用來指定全部鏈接的時長。com.zaxxer.hikari.housekeeping.periodMs用來指定鏈接池空閒鏈接處理及鏈接池數補充的HouseKeeper任務的調度時間間隔。

也就是說hikari比tomcat jdbc pool多了個maxLifetime，也就是全部的鏈接在maxLifetime以後都得重連一次，保證鏈接池的活性。

doc

• About Pool Sizing
• Re: Connection pooling - Number of connections
• Hikari connection pool grows to maximum size at start