在上一篇博客咱們介紹瞭如何在本身的項目中從無到有的添加了Quartz定時調度引擎,其實就是一個Quartz 和Spring的整合過程,很容易實現,可是咱們如今企業中項目一般都是部署在集羣環境中的,這樣咱們以前的定時調度就會出現問題了,由於咱們的定時任務都加載在內存中的,每一個集羣節點中的調度器都會去執行,這就會存在重複執行和資源競爭的問題,那麼如何來解決這樣的問題呢,往下面看吧...html
在通常的企業中解決相似的問題通常都是在一個note上部署Quartz其餘note不部署(或者是在其餘幾個機器加IP地址過濾),可是這樣集羣對於定時任務來講就沒有什麼意義了,並且存在着單點故障的隱患,也就是這臺部署着Quartz的機器一旦掛了,咱們的定時任務就中止服務了,這絕對不是咱們想要的。java
Quartz自己是支持集羣的,咱們經過Quartz的集羣方式來解決這樣的問題。mysql
雖然單個 Quartz 實例能給予你很好的 Job調度能力,但它不能令典型的企業需求,如可伸縮性、高可靠性知足。假如你須要故障轉移的能力並能運行日益增多的 Job,Quartz 集羣勢必成爲你方言的一部分了,而且即便是其中一臺機器在最糟的時間崩潰了也能確保全部的 Job 獲得執行。 QuartzJob Scheduling Frameworkspring
瞭解了Quartz集羣的好處,接下來就對咱們以前的工程進行改造,增長Quartz集羣特性。sql
Quartz集羣中節點依賴於數據庫來傳播 Scheduler 實例的狀態,你只能在使用 JDBC JobStore 時應用 Quartz 集羣數據庫
因此咱們集羣的第一步就是創建Quartz所須要的12張表:apache
在quartz核心包裏面經過quartz提供的建表語句創建相關表結構tomcat
生成的表結構以下oracle
這幾張表是用於存儲任務信息,觸發器,調度器,集羣節點等信息app
詳細解釋:
QRTZ_CRON_TRIGGERS 存儲Cron Trigger,包括Cron 表達式和時區信息
QRTZ_PAUSED_TRIGGER_GRPS 存儲已暫停的Trigger 組的信息
QRTZ_LOCKS 存儲程序的非觀鎖的信息(假如使用了悲觀鎖)
QRTZ_JOB_LISTENERS 存儲有關已配置的JobListener 的信息
QRTZ_BLOG_TRIGGERS Trigger 做爲Blob 類型存儲(用於Quartz 用戶用JDBC 建立他們本身定製的Trigger 類型,JobStore並不知道如何存儲實例的時候)
QRTZ_TRIGGERS 存儲已配置的Trigger 的信息
全部的表默認之前綴QRTZ_開始。能夠經過在quartz.properties配置修改(org.quartz.jobStore.tablePrefix= QRTZ_)。
共12張表的SQL腳本
1 # # Quartz seems to work best with the driver mm.mysql-2.0.7-bin.jar # # In your Quartz properties file, you'll need to set 2 # org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate 3 # 4 5 DROP TABLE IF EXISTS QRTZ_JOB_LISTENERS; DROP TABLE IF EXISTS QRTZ_TRIGGER_LISTENERS; DROP TABLE IF EXISTS QRTZ_FIRED_TRIGGERS; DROP TABLE IF EXISTS QRTZ_PAUSED_TRIGGER_GRPS; DROP TABLE IF EXISTS QRTZ_SCHEDULER_STATE; DROP TABLE IF EXISTS QRTZ_LOCKS; DROP TABLE IF EXISTS QRTZ_SIMPLE_TRIGGERS; DROP TABLE IF EXISTS QRTZ_CRON_TRIGGERS; DROP TABLE IF EXISTS QRTZ_BLOB_TRIGGERS; DROP TABLE IF EXISTS QRTZ_TRIGGERS; DROP TABLE IF EXISTS QRTZ_JOB_DETAILS; DROP TABLE IF EXISTS QRTZ_CALENDARS; CREATE TABLE QRTZ_JOB_DETAILS 6 ( 7 JOB_NAME VARCHAR(200) NOT NULL, 8 JOB_GROUP VARCHAR(200) NOT NULL, 9 DESCRIPTION VARCHAR(250) NULL, 10 JOB_CLASS_NAME VARCHAR(250) NOT NULL, 11 IS_DURABLE VARCHAR(1) NOT NULL, 12 IS_VOLATILE VARCHAR(1) NOT NULL, 13 IS_STATEFUL VARCHAR(1) NOT NULL, 14 REQUESTS_RECOVERY VARCHAR(1) NOT NULL, 15 JOB_DATA BLOB NULL, 16 PRIMARY KEY (JOB_NAME,JOB_GROUP) 17 ); CREATE TABLE QRTZ_JOB_LISTENERS 18 ( 19 JOB_NAME VARCHAR(200) NOT NULL, 20 JOB_GROUP VARCHAR(200) NOT NULL, 21 JOB_LISTENER VARCHAR(200) NOT NULL, 22 PRIMARY KEY (JOB_NAME,JOB_GROUP,JOB_LISTENER), 23 FOREIGN KEY (JOB_NAME,JOB_GROUP) 24 REFERENCES QRTZ_JOB_DETAILS(JOB_NAME,JOB_GROUP) 25 ); CREATE TABLE QRTZ_TRIGGERS 26 ( 27 TRIGGER_NAME VARCHAR(200) NOT NULL, 28 TRIGGER_GROUP VARCHAR(200) NOT NULL, 29 JOB_NAME VARCHAR(200) NOT NULL, 30 JOB_GROUP VARCHAR(200) NOT NULL, 31 IS_VOLATILE VARCHAR(1) NOT NULL, 32 DESCRIPTION VARCHAR(250) NULL, 33 NEXT_FIRE_TIME BIGINT(13) NULL, 34 PREV_FIRE_TIME BIGINT(13) NULL, 35 PRIORITY INTEGER NULL, 36 TRIGGER_STATE VARCHAR(16) NOT NULL, 37 TRIGGER_TYPE VARCHAR(8) NOT NULL, 38 START_TIME BIGINT(13) NOT NULL, 39 END_TIME BIGINT(13) NULL, 40 CALENDAR_NAME VARCHAR(200) NULL, 41 MISFIRE_INSTR SMALLINT(2) NULL, 42 JOB_DATA BLOB NULL, 43 PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP), 44 FOREIGN KEY (JOB_NAME,JOB_GROUP) 45 REFERENCES QRTZ_JOB_DETAILS(JOB_NAME,JOB_GROUP) 46 ); CREATE TABLE QRTZ_SIMPLE_TRIGGERS 47 ( 48 TRIGGER_NAME VARCHAR(200) NOT NULL, 49 TRIGGER_GROUP VARCHAR(200) NOT NULL, 50 REPEAT_COUNT BIGINT(7) NOT NULL, 51 REPEAT_INTERVAL BIGINT(12) NOT NULL, 52 TIMES_TRIGGERED BIGINT(10) NOT NULL, 53 PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP), 54 FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP) 55 REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP) 56 ); CREATE TABLE QRTZ_CRON_TRIGGERS 57 ( 58 TRIGGER_NAME VARCHAR(200) NOT NULL, 59 TRIGGER_GROUP VARCHAR(200) NOT NULL, 60 CRON_EXPRESSION VARCHAR(200) NOT NULL, 61 TIME_ZONE_ID VARCHAR(80), 62 PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP), 63 FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP) 64 REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP) 65 ); CREATE TABLE QRTZ_BLOB_TRIGGERS 66 ( 67 TRIGGER_NAME VARCHAR(200) NOT NULL, 68 TRIGGER_GROUP VARCHAR(200) NOT NULL, 69 BLOB_DATA BLOB NULL, 70 PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP), 71 FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP) 72 REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP) 73 ); CREATE TABLE QRTZ_TRIGGER_LISTENERS 74 ( 75 TRIGGER_NAME VARCHAR(200) NOT NULL, 76 TRIGGER_GROUP VARCHAR(200) NOT NULL, 77 TRIGGER_LISTENER VARCHAR(200) NOT NULL, 78 PRIMARY KEY (TRIGGER_NAME,TRIGGER_GROUP,TRIGGER_LISTENER), 79 FOREIGN KEY (TRIGGER_NAME,TRIGGER_GROUP) 80 REFERENCES QRTZ_TRIGGERS(TRIGGER_NAME,TRIGGER_GROUP) 81 ); CREATE TABLE QRTZ_CALENDARS 82 ( 83 CALENDAR_NAME VARCHAR(200) NOT NULL, 84 CALENDAR BLOB NOT NULL, 85 PRIMARY KEY (CALENDAR_NAME) 86 ); CREATE TABLE QRTZ_PAUSED_TRIGGER_GRPS 87 ( 88 TRIGGER_GROUP VARCHAR(200) NOT NULL, 89 PRIMARY KEY (TRIGGER_GROUP) 90 ); CREATE TABLE QRTZ_FIRED_TRIGGERS 91 ( 92 ENTRY_ID VARCHAR(95) NOT NULL, 93 TRIGGER_NAME VARCHAR(200) NOT NULL, 94 TRIGGER_GROUP VARCHAR(200) NOT NULL, 95 IS_VOLATILE VARCHAR(1) NOT NULL, 96 INSTANCE_NAME VARCHAR(200) NOT NULL, 97 FIRED_TIME BIGINT(13) NOT NULL, 98 PRIORITY INTEGER NOT NULL, 99 STATE VARCHAR(16) NOT NULL, 100 JOB_NAME VARCHAR(200) NULL, 101 JOB_GROUP VARCHAR(200) NULL, 102 IS_STATEFUL VARCHAR(1) NULL, 103 REQUESTS_RECOVERY VARCHAR(1) NULL, 104 PRIMARY KEY (ENTRY_ID) 105 ); CREATE TABLE QRTZ_SCHEDULER_STATE 106 ( 107 INSTANCE_NAME VARCHAR(200) NOT NULL, 108 LAST_CHECKIN_TIME BIGINT(13) NOT NULL, 109 CHECKIN_INTERVAL BIGINT(13) NOT NULL, 110 PRIMARY KEY (INSTANCE_NAME) 111 ); CREATE TABLE QRTZ_LOCKS 112 ( 113 LOCK_NAME VARCHAR(40) NOT NULL, 114 PRIMARY KEY (LOCK_NAME) 115 ); INSERT INTO QRTZ_LOCKS values('TRIGGER_ACCESS'); INSERT INTO QRTZ_LOCKS values('JOB_ACCESS'); INSERT INTO QRTZ_LOCKS values('CALENDAR_ACCESS'); INSERT INTO QRTZ_LOCKS values('STATE_ACCESS'); INSERT INTO QRTZ_LOCKS values('MISFIRE_ACCESS'); commit; 116 --------------------- 117 做者:默默同窗 118 來源:CSDN 119 原文:https://blog.csdn.net/u012909738/article/details/74454810
創建 quartz.properties文件把它放在工程的 src 目錄下,內容以下:
1 #============================================================================ 2 3 # Configure Main Scheduler Properties 4 5 #============================================================================ 6 7 8 9 org.quartz.scheduler.instanceName = Mscheduler 10 11 org.quartz.scheduler.instanceId = AUTO 12 13 org.quartz.jobStore.clusterCheckinInterval=20000 14 15 16 17 #============================================================================ 18 19 # Configure ThreadPool 20 21 #============================================================================ 22 23 24 25 org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool 26 27 org.quartz.threadPool.threadCount = 3 28 29 org.quartz.threadPool.threadPriority = 5 30 31 32 33 #============================================================================ 34 35 # Configure JobStore 36 37 #============================================================================ 38 39 40 41 #org.quartz.jobStore.class = org.quartz.simpl.RAMJobStore 42 43 44 45 org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX 46 47 #org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.StdJDBCDelegate 48 49 org.quartz.jobStore.useProperties = true 50 51 #org.quartz.jobStore.dataSource = myDS 52 53 org.quartz.jobStore.tablePrefix = QRTZ_ 54 55 org.quartz.jobStore.isClustered = true 56 57 org.quartz.jobStore.maxMisfiresToHandleAtATime=1 58 59 #============================================================================ 60 61 # Configure Datasources 62 63 #============================================================================ 64 65 66 67 #mysql 68 69 #org.quartz.dataSource.myDS.driver = com.ibm.db2.jcc.DB2Driver 70 71 #org.quartz.dataSource.myDS.URL = jdbc:db2://localhost:50000/db 72 73 #org.quartz.dataSource.myDS.user = db2 74 75 #org.quartz.dataSource.myDS.password = db2 76 77 #org.quartz.dataSource.myDS.maxConnections = 5 78 79 80 81 #oracle 82 83 #org.quartz.dataSource.myDS.driver = oracle.jdbc.driver.OracleDriver 84 85 #org.quartz.dataSource.myDS.URL = jdbc:oracle:thin:@localhost:1521:orcl 86 87 #org.quartz.dataSource.myDS.user = scott 88 89 #org.quartz.dataSource.myDS.password = shao 90 91 #org.quartz.dataSource.myDS.maxConnections = 5 92 93 94 95 #For Tomcat 96 97 org.quartz.jobStore.driverDelegateClass =org.quartz.impl.jdbcjobstore.oracle.OracleDelegate 98 99 #For Weblogic & Websphere 100 101 #org.quartz.jobStore.driverDelegateClass = org.quartz.impl.jdbcjobstore.WebLogicDelegate 102 103 org.quartz.jobStore.useProperties = false 104 105 org.quartz.jobStore.dataSource = myDS 106 107 108 109 110 111 #JNDI MODE 112 113 #For Tomcat 114 115 org.quartz.dataSource.myDS.jndiURL=java:comp/env/jdbc/oracle 116 117 #For Weblogic & Websphere 118 119 #org.quartz.dataSource.myDS.jndiURL=jdbc/oracle 120 121 122 123 124 125 #============================================================================ 126 127 # Configure Plugins 128 129 #============================================================================ 130 131 132 133 #org.quartz.plugin.triggHistory.class = org.quartz.plugins.history.LoggingJobHistoryPlugin 134 135 136 137 #org.quartz.plugin.jobInitializer.class = org.quartz.plugins.xml.JobInitializationPlugin 138 139 #org.quartz.plugin.jobInitializer.fileNames = jobs.xml 140 141 #org.quartz.plugin.jobInitializer.overWriteExistingJobs = true 142 143 #org.quartz.plugin.jobInitializer.failOnFileNotFound = true 144 145 #org.quartz.plugin.jobInitializer.scanInterval = 10 146 147 #org.quartz.plugin.jobInitializer.wrapInUserTransaction = false
紅色加粗部分是集羣須要的配置
核心配置解釋以下:
org.quartz.jobStore.class 屬性爲JobStoreTX,
將任務持久化到數據中。由於集羣中節點依賴於數據庫來傳播Scheduler實例的狀態,你只能在使用JDBC JobStore 時應用Quartz 集羣。
org.quartz.jobStore.isClustered 屬性爲true,通知Scheduler實例要它參與到一個集羣當中。
org.quartz.jobStore.clusterCheckinInterval
屬性定義了Scheduler實例檢入到數據庫中的頻率(單位:毫秒)。
Scheduler 檢查是否其餘的實例到了它們應當檢入的時候未檢入;
這能指出一個失敗的Scheduler 實例,且當前Scheduler 會以此來接管任何執行失敗並可恢復的Job。
經過檢入操做,Scheduler也會更新自身的狀態記錄。clusterChedkinInterval越小,Scheduler節點檢查失敗的Scheduler 實例就越頻繁。默認值是 20000 (即20 秒)
1 <?xmlversion="1.0"encoding="UTF-8"?>
2 <!DOCTYPEbeansPUBLIC"-//SPRING//DTD BEAN//EN"
3 "http://www.springframework.org/dtd/spring-beans.dtd">
4 <beans>
5
6 <!-- 調度器lazy-init='false'那麼容器啓動就會執行調度程序 -->
7 <beanid="startQuertz"lazy-init="false"autowire="no"class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
8 <property name="dataSource" ref="dataSource"/> 9 <property name="configLocation" value="classpath:quartz.properties" />
10 <propertyname="triggers">
11 <list>
12 <refbean="doTime"/>
13 </list>
14 </property>
15 <!-- 容許在Quartz上下文中使用Spring實例工廠 -->
16 <propertyname="applicationContextSchedulerContextKey"value="applicationContext"/>
17 </bean>
18
19 <!-- 觸發器 -->
20 <beanid="doTime"class="org.springframework.scheduling.quartz.CronTriggerBean">
21 <propertyname="jobDetail"ref="jobtask"></property>
22 <!-- cron表達式 -->
23 <propertyname="cronExpression"value="10,15,20,25,30,35,40,45,50,55 * * * * ?"></property>
24 </bean>
25
26 <!-- 任務 -->
27 <beanid="jobtask"class="org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean">
28 <propertyname="targetObject"ref="synUsersJob"></property>
29 <propertyname="targetMethod"value="execute"></property>
30 </bean>
31
32 <!-- 要調用的工做類 -->
33 <beanid="synUsersJob"class="org.leopard.core.quartz.job.SynUsersJob"></bean>
34
35 </beans>
增長紅色加粗部分代碼,注入數據源和加載quartz.properties文件
OK Quartz集羣的配置只有這幾步,咱們來啓動項目。。。
咱們啓着啓着….報錯了!
17:00:59,718 ERROR ContextLoader:215 - Context initialization failed org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'startQuertz' defined in class path resource [config/spring/spring-time.xml]: Invocation of init method failed; nested exception is org.quartz.JobPersistenceException:Couldn't store job: Unable to serialize JobDataMap for insertion into database because the value of property 'methodInvoker' is not serializable: org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean [See nested exception: java.io.NotSerializableException: Unable to serialize JobDataMap for insertion into database because the value of property 'methodInvoker' is not serializable: org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean] |
咱們主要來看紅色部分,主要緣由就是這個MethodInvokingJobDetailFactoryBean 類中的 methodInvoking 方法,是不支持序列化的,所以在把 quartz 的 task 序列化進入數據庫時就會拋這個serializable的錯誤
網上查了一下,解決這個問題,目前主要有兩種方案:
博客地址:http://jira.springframework.org/browse/SPR-3797
做者重寫了MethodInvokingJobDetailFactoryBean
博客地址:http://blog.csdn.net/lifetragedy/article/details/6212831
根據 QuartzJobBean 來重寫一個本身的類,而後使用 SPRING 把這個重寫的類(咱們就名命它爲: MyDetailQuartzJobBean )注入 appContext 中後,再使用 AOP 技術反射出原有的 quartzJobx( 就是開發人員原來已經作好的用於執行 QUARTZ 的 JOB 的執行類 ) 。
兩種方式我都進行了測試,均可以解決問題,咱們這裏先經過第二種方式解決這個bug,沒有修改任何Spring的源碼
1 package org.leopard.core.quartz; 2 3 import java.lang.reflect.Method; 4 5 import org.apache.commons.logging.Log; 6 import org.apache.commons.logging.LogFactory; 7 import org.quartz.JobExecutionContext; 8 import org.quartz.JobExecutionException; 9 import org.springframework.context.ApplicationContext; 10 import org.springframework.scheduling.quartz.QuartzJobBean; 11 12 /** 13 * 解決Spring和Quartz整合bug 14 * 15 */ 16 public class MyDetailQuartzJobBean extends QuartzJobBean { 17 protected final Log logger = LogFactory.getLog(getClass()); 18 19 private String targetObject; 20 private String targetMethod; 21 private ApplicationContext ctx; 22 23 protected void executeInternal(JobExecutionContext context) throws JobExecutionException { 24 try { 25 26 logger.info("execute [" + targetObject + "] at once>>>>>>"); 27 Object otargetObject = ctx.getBean(targetObject); 28 Method m = null; 29 try { 30 m = otargetObject.getClass().getMethod(targetMethod, new Class[] {}); 31 32 m.invoke(otargetObject, new Object[] {}); 33 } catch (SecurityException e) { 34 logger.error(e); 35 } catch (NoSuchMethodException e) { 36 logger.error(e); 37 } 38 39 } catch (Exception e) { 40 throw new JobExecutionException(e); 41 } 42 43 } 44 45 public void setApplicationContext(ApplicationContext applicationContext) { 46 this.ctx = applicationContext; 47 } 48 49 public void setTargetObject(String targetObject) { 50 this.targetObject = targetObject; 51 } 52 53 public void setTargetMethod(String targetMethod) { 54 this.targetMethod = targetMethod; 55 } 56 57 }
修改後的spring-time.xml文件內容以下:
1 <?xmlversion="1.0"encoding="UTF-8"?>
2 <!DOCTYPEbeansPUBLIC"-//SPRING//DTD BEAN//EN"
3 "http://www.springframework.org/dtd/spring-beans.dtd">
4 <beans>
5
6 <!-- 調度器lazy-init='false'那麼容器啓動就會執行調度程序 -->
7 <beanid="startQuertz"lazy-init="false"autowire="no"class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
8 <propertyname="dataSource"ref="dataSource"/>
9 <propertyname="configLocation"value="classpath:quartz.properties"/>
10 <propertyname="triggers">
11 <list>
12 <refbean="doTime"/>
13 </list>
14 </property>
15 <!--這個是必須的,QuartzScheduler延時啓動,應用啓動完後 QuartzScheduler再啓動-->
16 <propertyname="startupDelay"value="30"/>
17 <!--這個是可選,QuartzScheduler啓動時更新己存在的Job,這樣就不用每次修改targetObject後刪除qrtz_job_details表對應記錄了-->
18 <propertyname="overwriteExistingJobs"value="true"/>
19 <!-- 容許在Quartz上下文中使用Spring實例工廠 -->
20 <propertyname="applicationContextSchedulerContextKey"value="applicationContext"/>
21 </bean>
22
23 <!-- 觸發器 -->
24 <beanid="doTime"class="org.springframework.scheduling.quartz.CronTriggerBean">
25 <propertyname="jobDetail"ref="jobtask"></property>
26 <!-- cron表達式 -->
27 <propertyname="cronExpression"value="10,15,20,25,30,35,40,45,50,55 * * * * ?"></property>
28 </bean>
29
30 <!-- 任務 -->
31 <beanid="jobtask"class="org.springframework.scheduling.quartz.JobDetailBean">
32 <propertyname="jobClass"> 33 <value>org.leopard.core.quartz.MyDetailQuartzJobBean</value> 34 </property>
35 <propertyname="jobDataAsMap">
36 <map>
37 <entrykey="targetObject"value="synUsersJob"/>
38 <entrykey="targetMethod"value="execute"/>
39 </map>
40 </property>
41 </bean>
42
43 <!-- 要調用的工做類 -->
44 <beanid="synUsersJob"class="org.leopard.core.quartz.job.SynUsersJob"></bean>
45
46 </beans>
主要看紅色加粗部分...
Ok 配置完成,咱們把oa_ssh部署到兩臺tomcat上面,分別啓動。
能夠看到咱們先啓動的tomcat控制檯打印出日誌
另一臺沒有打印日誌
這時咱們把執行定時任務的那臺tomcat中止,能夠看到等了一會以後,咱們的另一臺tomcat會把以前tomcat執行的定時任務接管過來繼續執行,咱們的集羣是成功的。
參考網址: