Springboot quartz集羣(3) — 多節點發送郵件

本期將提供quartz集羣能力

• 集羣案例分析： 
  上一期的郵件發送功能，若在服務須要部署多節點，但定時任務不支持集羣，所以，多節點定時任務勢必會同時運行， 
  若向用戶發送郵件通知，這種狀況下會向用戶發送兩次如出一轍的郵件，N個節點會發送N次郵件，嚴重不符合業務場景， 
  若提供集羣能力，則多節點間應分擔郵件發送的工做而不是各節點作重複的工做，所以在部署多節點的時候定時任務也須要提供集羣能力。
• 我的看法： 
  
  1. quartz集羣分爲水平集羣和垂直集羣，水平集羣即將定時任務節點部署在不一樣的服務器，水平集羣最大的問題就是時鐘同步問題， 
     quartz集羣強烈要求時鐘同步，若時鐘不能同步，則會致使集羣中各個節點狀態紊亂，形成不可預知的後果，請自行搜索服務器時鐘同步, 
     若能保證時鐘同步，水平集羣能保證服務的可靠性，其中一個節點掛掉或其中一個服務器宕機，其餘節點依然正常服務；垂直集羣則是集羣各節點部署在同一臺服務器， 
     時鐘同步天然不是問題，但存在單點故障問題，服務器宕機會嚴重影響服務的可用性。所以，要結合實際狀況來考慮集羣方案
  2. 因爲集羣中強烈要求時鐘同步，所以無論是垂直集羣仍是水平集羣，本地開發決不能鏈接線上環境（本地也是集羣模式），這樣的話勢必會破壞集羣，但本地如果非集羣模式， 
     則能夠依狀況來鏈接線上環境。
  3. quartz集羣和redis這樣的集羣實現方式不同，redis集羣須要節點之間通訊，各節點須要知道其餘節點的情況，而quartz集羣的實現 
     方式在於11張表，集羣節點相互之間不通訊，而是經過定時任務持久化加鎖的方式來實現集羣。
  4. 破壞集羣后果通常是死鎖或者狀態紊亂每一個節點都不可用或其中某些節點能用部分或所有的定時任務

1. 建立集羣須要的11張表

t_b_qrtz_blob_triggers
t_b_qrtz_calendars
t_b_qrtz_cron_triggers
t_b_qrtz_fired_triggers
t_b_qrtz_job_details
t_b_qrtz_locks
t_b_qrtz_paused_trigger_grps
t_b_qrtz_scheduler_state
t_b_qrtz_simple_triggers
t_b_qrtz_simprop_triggers
t_b_qrtz_triggers

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11

2. 集羣建表sql

drop table if exists t_b_qrtz_fired_triggers; drop table if exists t_b_qrtz_paused_trigger_grps; drop table if exists t_b_qrtz_scheduler_state; drop table if exists t_b_qrtz_locks; drop table if exists t_b_qrtz_simple_triggers; drop table if exists t_b_qrtz_simprop_triggers; drop table if exists t_b_qrtz_cron_triggers; drop table if exists t_b_qrtz_blob_triggers; drop table if exists t_b_qrtz_triggers; drop table if exists t_b_qrtz_job_details; drop table if exists t_b_qrtz_calendars; create table t_b_qrtz_job_details( sched_name varchar(120) not null, job_name varchar(200) not null, job_group varchar(200) not null, description varchar(250) null, job_class_name varchar(250) not null, is_durable varchar(1) not null, is_nonconcurrent varchar(1) not null, is_update_data varchar(1) not null, requests_recovery varchar(1) not null, job_data blob null, primary key (sched_name,job_name,job_group)) engine=innodb; create table t_b_qrtz_triggers ( sched_name varchar(120) not null, trigger_name varchar(200) not null, trigger_group varchar(200) not null, job_name varchar(200) not null, job_group varchar(200) not null, description varchar(250) null, next_fire_time bigint(13) null, prev_fire_time bigint(13) null, priority integer null, trigger_state varchar(16) not null, trigger_type varchar(8) not null, start_time bigint(13) not null, end_time bigint(13) null, calendar_name varchar(200) null, misfire_instr smallint(2) null, job_data blob null, primary key (sched_name,trigger_name,trigger_group), foreign key (sched_name,job_name,job_group) references t_b_qrtz_job_details(sched_name,job_name,job_group)) engine=innodb; create table t_b_qrtz_simple_triggers ( sched_name varchar(120) not null, trigger_name varchar(200) not null, trigger_group varchar(200) not null, repeat_count bigint(7) not null, repeat_interval bigint(12) not null, times_triggered bigint(10) not null, primary key (sched_name,trigger_name,trigger_group), foreign key (sched_name,trigger_name,trigger_group) references t_b_qrtz_triggers(sched_name,trigger_name,trigger_group)) engine=innodb; create table t_b_qrtz_cron_triggers ( sched_name varchar(120) not null, trigger_name varchar(200) not null, trigger_group varchar(200) not null, cron_expression varchar(120) not null, time_zone_id varchar(80), primary key (sched_name,trigger_name,trigger_group), foreign key (sched_name,trigger_name,trigger_group) references t_b_qrtz_triggers(sched_name,trigger_name,trigger_group)) engine=innodb; create table t_b_qrtz_simprop_triggers ( sched_name varchar(120) not null, trigger_name varchar(200) not null, trigger_group varchar(200) not null, str_prop_1 varchar(512) null, str_prop_2 varchar(512) null, str_prop_3 varchar(512) null, int_prop_1 int null, int_prop_2 int null, long_prop_1 bigint null, long_prop_2 bigint null, dec_prop_1 numeric(13,4) null, dec_prop_2 numeric(13,4) null, bool_prop_1 varchar(1) null, bool_prop_2 varchar(1) null, primary key (sched_name,trigger_name,trigger_group), foreign key (sched_name,trigger_name,trigger_group) references t_b_qrtz_triggers(sched_name,trigger_name,trigger_group)) engine=innodb; create table t_b_qrtz_blob_triggers ( sched_name varchar(120) not null, trigger_name varchar(200) not null, trigger_group varchar(200) not null, blob_data blob null, primary key (sched_name,trigger_name,trigger_group), index (sched_name,trigger_name, trigger_group), foreign key (sched_name,trigger_name,trigger_group) references t_b_qrtz_triggers(sched_name,trigger_name,trigger_group)) engine=innodb; create table t_b_qrtz_calendars ( sched_name varchar(120) not null, calendar_name varchar(200) not null, calendar blob not null, primary key (sched_name,calendar_name)) engine=innodb; create table t_b_qrtz_paused_trigger_grps ( sched_name varchar(120) not null, trigger_group varchar(200) not null, primary key (sched_name,trigger_group)) engine=innodb; create table t_b_qrtz_fired_triggers ( sched_name varchar(120) not null, entry_id varchar(95) not null, trigger_name varchar(200) not null, trigger_group varchar(200) not null, instance_name varchar(200) not null, fired_time bigint(13) not null, sched_time bigint(13) not null, priority integer not null, state varchar(16) not null, job_name varchar(200) null, job_group varchar(200) null, is_nonconcurrent varchar(1) null, requests_recovery varchar(1) null, primary key (sched_name,entry_id)) engine=innodb; create table t_b_qrtz_scheduler_state ( sched_name varchar(120) not null, instance_name varchar(200) not null, last_checkin_time bigint(13) not null, checkin_interval bigint(13) not null, primary key (sched_name,instance_name)) engine=innodb; create table t_b_qrtz_locks ( sched_name varchar(120) not null, lock_name varchar(40) not null, primary key (sched_name,lock_name)) engine=innodb; create index idx_qrtz_j_req_recovery on t_b_qrtz_job_details(sched_name,requests_recovery); create index idx_qrtz_j_grp on t_b_qrtz_job_details(sched_name,job_group); create index idx_qrtz_t_j on t_b_qrtz_triggers(sched_name,job_name,job_group); create index idx_qrtz_t_jg on t_b_qrtz_triggers(sched_name,job_group); create index idx_qrtz_t_c on t_b_qrtz_triggers(sched_name,calendar_name); create index idx_qrtz_t_g on t_b_qrtz_triggers(sched_name,trigger_group); create index idx_qrtz_t_state on t_b_qrtz_triggers(sched_name,trigger_state); create index idx_qrtz_t_n_state on t_b_qrtz_triggers(sched_name,trigger_name,trigger_group,trigger_state); create index idx_qrtz_t_n_g_state on t_b_qrtz_triggers(sched_name,trigger_group,trigger_state); create index idx_qrtz_t_next_fire_time on t_b_qrtz_triggers(sched_name,next_fire_time); create index idx_qrtz_t_nft_st on t_b_qrtz_triggers(sched_name,trigger_state,next_fire_time); create index idx_qrtz_t_nft_misfire on t_b_qrtz_triggers(sched_name,misfire_instr,next_fire_time); create index idx_qrtz_t_nft_st_misfire on t_b_qrtz_triggers(sched_name,misfire_instr,next_fire_time,trigger_state); create index idx_qrtz_t_nft_st_misfire_grp on t_b_qrtz_triggers(sched_name,misfire_instr,next_fire_time,trigger_group,trigger_state); create index idx_qrtz_ft_trig_inst_name on t_b_qrtz_fired_triggers(sched_name,instance_name); create index idx_qrtz_ft_inst_job_req_rcvry on t_b_qrtz_fired_triggers(sched_name,instance_name,requests_recovery); create index idx_qrtz_ft_j_g on t_b_qrtz_fired_triggers(sched_name,job_name,job_group); create index idx_qrtz_ft_jg on t_b_qrtz_fired_triggers(sched_name,job_group); create index idx_qrtz_ft_t_g on t_b_qrtz_fired_triggers(sched_name,trigger_name,trigger_group); create index idx_qrtz_ft_tg on t_b_qrtz_fired_triggers(sched_name,trigger_group); commit;

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15
• 16
• 17
• 18
• 19
• 20
• 21
• 22
• 23
• 24
• 25
• 26
• 27
• 28
• 29
• 30
• 31
• 32
• 33
• 34
• 35
• 36
• 37
• 38
• 39
• 40
• 41
• 42
• 43
• 44
• 45
• 46
• 47
• 48
• 49
• 50
• 51
• 52
• 53
• 54
• 55
• 56
• 57
• 58
• 59
• 60
• 61
• 62
• 63
• 64
• 65
• 66
• 67
• 68
• 69
• 70
• 71
• 72
• 73
• 74
• 75
• 76
• 77
• 78
• 79
• 80
• 81
• 82
• 83
• 84
• 85
• 86
• 87
• 88
• 89
• 90
• 91
• 92
• 93
• 94
• 95
• 96
• 97
• 98
• 99
• 100
• 101
• 102
• 103
• 104
• 105
• 106
• 107
• 108
• 109
• 110
• 111
• 112
• 113
• 114
• 115
• 116
• 117
• 118
• 119
• 120
• 121
• 122
• 123
• 124
• 125
• 126
• 127
• 128
• 129
• 130
• 131
• 132
• 133
• 134
• 135
• 136
• 137
• 138
• 139
• 140
• 141
• 142
• 143
• 144
• 145
• 146
• 147
• 148
• 149
• 150
• 151
• 152
• 153
• 154
• 155
• 156
• 157
• 158
• 159
• 160
• 161
• 162
• 163
• 164
• 165
• 166
• 167
• 168
• 169
• 170
• 171

3. 工程中application.properties添加集羣配置

quartz.scheduler.instanceName=CnlmScheduler
org.quartz.dataSource.myDS.driver=com.mysql.cj.jdbc.Driver
org.quartz.dataSource.myDS.URL=jdbc:mysql://120.77.172.111:3306/touch6?useUnicode=true&characterEncoding=utf8
org.quartz.dataSource.myDS.user=cnlm.me
org.quartz.dataSource.myDS.password=123456
org.quartz.dataSource.myDS.maxConnections=10

• 1
• 2
• 3
• 4
• 5
• 6

4. 調度工廠配置集羣參數支持集羣能力，下面是支持集羣能力的調度工廠類

package me.cnlm.springboot.quartz.config; import org.quartz.Trigger; import org.springframework.beans.factory.annotation.Qualifier; import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.scheduling.quartz.SchedulerFactoryBean; import java.io.IOException; import java.util.Properties; @Configuration public class SchedulerFactoryBeanConfig { @Value("${quartz.scheduler.instanceName}") private String quartzInstanceName; @Value("${org.quartz.dataSource.myDS.driver}") private String myDSDriver; @Value("${org.quartz.dataSource.myDS.URL}") private String myDSURL; @Value("${org.quartz.dataSource.myDS.user}") private String myDSUser; @Value("${org.quartz.dataSource.myDS.password}") private String myDSPassword; @Value("${org.quartz.dataSource.myDS.maxConnections}") private String myDSMaxConnections; /** * 定時任務集羣配置 * 設置屬性 * * @return * @throws IOException */ private Properties quartzProperties() throws IOException { Properties prop = new Properties(); prop.put("quartz.scheduler.instanceName", quartzInstanceName); prop.put("org.quartz.scheduler.instanceId", "AUTO"); prop.put("org.quartz.scheduler.skipUpdateCheck", "true"); prop.put("org.quartz.scheduler.jmx.export", "true"); prop.put("org.quartz.jobStore.class", "org.quartz.impl.jdbcjobstore.JobStoreTX"); prop.put("org.quartz.jobStore.driverDelegateClass", "org.quartz.impl.jdbcjobstore.StdJDBCDelegate"); prop.put("org.quartz.jobStore.dataSource", "quartzDataSource"); prop.put("org.quartz.jobStore.tablePrefix", "T_B_QRTZ_"); prop.put("org.quartz.jobStore.isClustered", "true"); prop.put("org.quartz.jobStore.clusterCheckinInterval", "20000"); prop.put("org.quartz.jobStore.dataSource", "myDS"); prop.put("org.quartz.jobStore.maxMisfiresToHandleAtATime", "1"); prop.put("org.quartz.jobStore.misfireThreshold", "120000"); prop.put("org.quartz.jobStore.txIsolationLevelSerializable", "true"); prop.put("org.quartz.jobStore.selectWithLockSQL", "SELECT * FROM {0}LOCKS WHERE LOCK_NAME = ? FOR UPDATE"); prop.put("org.quartz.threadPool.class", "org.quartz.simpl.SimpleThreadPool"); prop.put("org.quartz.threadPool.threadCount", "10"); prop.put("org.quartz.threadPool.threadPriority", "5"); prop.put("org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread", "true"); prop.put("org.quartz.dataSource.myDS.driver", myDSDriver); prop.put("org.quartz.dataSource.myDS.URL", myDSURL); prop.put("org.quartz.dataSource.myDS.user", myDSUser); prop.put("org.quartz.dataSource.myDS.password", myDSPassword); prop.put("org.quartz.dataSource.myDS.maxConnections", myDSMaxConnections); prop.put("org.quartz.plugin.triggHistory.class", "org.quartz.plugins.history.LoggingJobHistoryPlugin"); prop.put("org.quartz.plugin.shutdownhook.class", "org.quartz.plugins.management.ShutdownHookPlugin"); prop.put("org.quartz.plugin.shutdownhook.cleanShutdown", "true"); return prop; } @Bean public SchedulerFactoryBean schedulerFactoryBean(@Qualifier("sendEmailTrigger") Trigger sendEmailTrigger) throws IOException { SchedulerFactoryBean factory = new SchedulerFactoryBean(); factory.setOverwriteExistingJobs(true); //用於quartz集羣,加載quartz數據源 //factory.setDataSource(dataSource); factory.setStartupDelay(10); // factory.setQuartzProperties(quartzProperties()); factory.setAutoStartup(true); factory.setApplicationContextSchedulerContextKey("applicationContext"); //註冊觸發器 factory.setTriggers( sendEmailTrigger ); return factory; } }

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15
• 16
• 17
• 18
• 19
• 20
• 21
• 22
• 23
• 24
• 25
• 26
• 27
• 28
• 29
• 30
• 31
• 32
• 33
• 34
• 35
• 36
• 37
• 38
• 39
• 40
• 41
• 42
• 43
• 44
• 45
• 46
• 47
• 48
• 49
• 50
• 51
• 52
• 53
• 54
• 55
• 56
• 57
• 58
• 59
• 60
• 61
• 62
• 63
• 64
• 65
• 66
• 67
• 68
• 69
• 70
• 71
• 72
• 73
• 74
• 75
• 76
• 77
• 78
• 79
• 80
• 81
• 82
• 83
• 84
• 85
• 86
• 87
• 88
• 89
• 90
• 91
• 92
• 93
• 94
• 95

5. 測試驗證

略（分別運行兩個項目節點，能夠發現上述11張表中添加了2個節點的信息和定時任務運行狀態，此處可自行驗證）

本期已結束，至此項目支持定時任務分佈式集羣模式

歡迎加入技術交流QQ羣566654343