聊聊hikari鏈接池的idleTimeout及minimumIdle屬性

序

本文主要研究一個hikari鏈接池的idleTimeout及minimumIdle屬性

idleTimeout

默認是600000毫秒，即10分鐘。若是idleTimeout+1秒>maxLifetime 且 maxLifetime>0，則會被重置爲0；若是idleTimeout!=0且小於10秒，則會被重置爲10秒。若是idleTimeout=0則表示空閒的鏈接在鏈接池中永遠不被移除。

只有當minimumIdle小於maximumPoolSize時，這個參數才生效，當空閒鏈接數超過minimumIdle，並且空閒時間超過idleTimeout，則會被移除。

minimumIdle

控制鏈接池空閒鏈接的最小數量，當鏈接池空閒鏈接少於minimumIdle，並且總共鏈接數不大於maximumPoolSize時，HikariCP會盡力補充新的鏈接。爲了性能考慮，不建議設置此值，而是讓HikariCP把鏈接池當作固定大小的處理，默認minimumIdle與maximumPoolSize同樣。

當minIdle<0或者minIdle>maxPoolSize,則被重置爲maxPoolSize，該值默認爲10。

HikariPool.HouseKeeper

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

private final long HOUSEKEEPING_PERIOD_MS = Long.getLong("com.zaxxer.hikari.housekeeping.periodMs", SECONDS.toMillis(30));

    this.houseKeeperTask = houseKeepingExecutorService.scheduleWithFixedDelay(new HouseKeeper(), 100L, HOUSEKEEPING_PERIOD_MS, MILLISECONDS);

   /**
    * The house keeping task to retire and maintain minimum idle connections.
    */
   private final class HouseKeeper implements Runnable
   {
      private volatile long previous = plusMillis(currentTime(), -HOUSEKEEPING_PERIOD_MS);

      @Override
      public void run()
      {
         try {
            // refresh timeouts in case they changed via MBean
            connectionTimeout = config.getConnectionTimeout();
            validationTimeout = config.getValidationTimeout();
            leakTaskFactory.updateLeakDetectionThreshold(config.getLeakDetectionThreshold());

            final long idleTimeout = config.getIdleTimeout();
            final long now = currentTime();

            // Detect retrograde time, allowing +128ms as per NTP spec.
            if (plusMillis(now, 128) < plusMillis(previous, HOUSEKEEPING_PERIOD_MS)) {
               LOGGER.warn("{} - Retrograde clock change detected (housekeeper delta={}), soft-evicting connections from pool.",
                           poolName, elapsedDisplayString(previous, now));
               previous = now;
               softEvictConnections();
               return;
            }
            else if (now > plusMillis(previous, (3 * HOUSEKEEPING_PERIOD_MS) / 2)) {
               // No point evicting for forward clock motion, this merely accelerates connection retirement anyway
               LOGGER.warn("{} - Thread starvation or clock leap detected (housekeeper delta={}).", poolName, elapsedDisplayString(previous, now));
            }

            previous = now;

            String afterPrefix = "Pool ";
            if (idleTimeout > 0L && config.getMinimumIdle() < config.getMaximumPoolSize()) {
               logPoolState("Before cleanup ");
               afterPrefix = "After cleanup  ";

               final List<PoolEntry> notInUse = connectionBag.values(STATE_NOT_IN_USE);
               int toRemove = notInUse.size() - config.getMinimumIdle();
               for (PoolEntry entry : notInUse) {
                  if (toRemove > 0 && elapsedMillis(entry.lastAccessed, now) > idleTimeout && connectionBag.reserve(entry)) {
                     closeConnection(entry, "(connection has passed idleTimeout)");
                     toRemove--;
                  }
               }
            }

            logPoolState(afterPrefix);

            fillPool(); // Try to maintain minimum connections
         }
         catch (Exception e) {
            LOGGER.error("Unexpected exception in housekeeping task", e);
         }
      }
   }

這個HouseKeeper是一個定時任務，在HikariPool構造器裏頭初始化，默認的是初始化後100毫秒執行，以後每執行完一次以後隔HOUSEKEEPING_PERIOD_MS( 30秒)時間執行。
這個定時任務的做用就是根據idleTimeout的值，移除掉空閒超時的鏈接。
首先檢測時鐘是否倒退，若是倒退了則當即對過時的鏈接進行標記evict；以後當idleTimeout>0且配置的minimumIdle<maximumPoolSize時纔開始處理超時的空閒鏈接。
取出狀態是STATE_NOT_IN_USE的鏈接數，若是大於minimumIdle，則遍歷STATE_NOT_IN_USE的鏈接的鏈接，將空閒超時達到idleTimeout的鏈接從connectionBag移除掉，若移除成功則關閉該鏈接，而後toRemove--。
在空閒鏈接移除以後，再調用fillPool，嘗試補充空間鏈接數到minimumIdle值

HikariPool.fillPool

HikariCP-2.7.6-sources.jar!/com/zaxxer/hikari/pool/HikariPool.java

private final PoolEntryCreator POOL_ENTRY_CREATOR = new PoolEntryCreator(null /*logging prefix*/);
      private final PoolEntryCreator POST_FILL_POOL_ENTRY_CREATOR = new PoolEntryCreator("After adding ");
      LinkedBlockingQueue<Runnable> addConnectionQueue = new LinkedBlockingQueue<>(config.getMaximumPoolSize());
      this.addConnectionQueue = unmodifiableCollection(addConnectionQueue);
      this.addConnectionExecutor = createThreadPoolExecutor(addConnectionQueue, poolName + " connection adder", threadFactory, new ThreadPoolExecutor.DiscardPolicy());

   /**
    * Fill pool up from current idle connections (as they are perceived at the point of execution) to minimumIdle connections.
    */
   private synchronized void fillPool()
   {
      final int connectionsToAdd = Math.min(config.getMaximumPoolSize() - getTotalConnections(), config.getMinimumIdle() - getIdleConnections())
                                   - addConnectionQueue.size();
      for (int i = 0; i < connectionsToAdd; i++) {
         addConnectionExecutor.submit((i < connectionsToAdd - 1) ? POOL_ENTRY_CREATOR : POST_FILL_POOL_ENTRY_CREATOR);
      }
   }

PoolEntryCreator

/**
    * Creating and adding poolEntries (connections) to the pool.
    */
   private final class PoolEntryCreator implements Callable<Boolean>
   {
      private final String loggingPrefix;

      PoolEntryCreator(String loggingPrefix)
      {
         this.loggingPrefix = loggingPrefix;
      }

      @Override
      public Boolean call() throws Exception
      {
         long sleepBackoff = 250L;
         while (poolState == POOL_NORMAL && shouldCreateAnotherConnection()) {
            final PoolEntry poolEntry = createPoolEntry();
            if (poolEntry != null) {
               connectionBag.add(poolEntry);
               LOGGER.debug("{} - Added connection {}", poolName, poolEntry.connection);
               if (loggingPrefix != null) {
                  logPoolState(loggingPrefix);
               }
               return Boolean.TRUE;
            }

            // failed to get connection from db, sleep and retry
            quietlySleep(sleepBackoff);
            sleepBackoff = Math.min(SECONDS.toMillis(10), Math.min(connectionTimeout, (long) (sleepBackoff * 1.5)));
         }
         // Pool is suspended or shutdown or at max size
         return Boolean.FALSE;
      }

      /**
       * We only create connections if we need another idle connection or have threads still waiting
       * for a new connection.  Otherwise we bail out of the request to create.
       *
       * @return true if we should create a connection, false if the need has disappeared
       */
      private boolean shouldCreateAnotherConnection() {
         return getTotalConnections() < config.getMaximumPoolSize() &&
            (connectionBag.getWaitingThreadCount() > 0 || getIdleConnections() < config.getMinimumIdle());
      }
   }

shouldCreateAnotherConnection方法決定了是否須要添加新的鏈接

createPoolEntry

/**
    * Creating new poolEntry.  If maxLifetime is configured, create a future End-of-life task with 2.5% variance from
    * the maxLifetime time to ensure there is no massive die-off of Connections in the pool.
    */
   private PoolEntry createPoolEntry()
   {
      try {
         final PoolEntry poolEntry = newPoolEntry();

         final long maxLifetime = config.getMaxLifetime();
         if (maxLifetime > 0) {
            // variance up to 2.5% of the maxlifetime
            final long variance = maxLifetime > 10_000 ? ThreadLocalRandom.current().nextLong( maxLifetime / 40 ) : 0;
            final long lifetime = maxLifetime - variance;
            poolEntry.setFutureEol(houseKeepingExecutorService.schedule(
               () -> {
                  if (softEvictConnection(poolEntry, "(connection has passed maxLifetime)", false /* not owner */)) {
                     addBagItem(connectionBag.getWaitingThreadCount());
                  }
               },
               lifetime, MILLISECONDS));
         }

         return poolEntry;
      }
      catch (Exception e) {
         if (poolState == POOL_NORMAL) { // we check POOL_NORMAL to avoid a flood of messages if shutdown() is running concurrently
            LOGGER.debug("{} - Cannot acquire connection from data source", poolName, (e instanceof ConnectionSetupException ? e.getCause() : e));
         }
         return null;
      }
   }

createPoolEntry方法建立一個poolEntry，同時給它的lifetime過時設定了一個延時任務。

小結

• HouseKeeper是一個定時任務，在HikariPool構造器裏頭初始化，默認的是初始化後100毫秒執行，以後每執行完一次以後隔HOUSEKEEPING_PERIOD_MS(30秒)時間執行。
• 若是發現時鐘倒退，則當即標記evict鏈接，而後退出；不然都會執行fillPool，來試圖維持空閒鏈接到minimumIdle的數值
• 當idleTimeout>0且配置的minimumIdle<maximumPoolSize時，會移除超過idleTimeout的空閒鏈接，不然不作操做，繼續往下執行fillPool
• 當minIdle<0或者minIdle>maxPoolSize,則minIdle被重置爲maxPoolSize，該值默認爲10，官方建議設置爲一致，當作固定大小的鏈接池處理提升性能

idleTimeout有點相似tomcat jdbc pool裏頭的min-evictable-idle-time-millis參數。不一樣的是tomcat jdbc pool的鏈接泄露檢測以及空閒鏈接清除的工做都放在一個名爲PoolCleaner的timerTask中處理，該任務的執行間隔爲timeBetweenEvictionRunsMillis，默認爲5秒；而hikari的鏈接泄露是每次getConnection的時候單獨觸發一個延時任務來處理，而空閒鏈接的清除則是使用HouseKeeper定時任務來處理，其運行間隔由com.zaxxer.hikari.housekeeping.periodMs環境變量控制，默認爲30秒。

doc

• configuration-knobs-baby