《淺入淺出》-RocketMQ

<div class="output_wrapper" id="output_wrapper_id" style="font-size: 15px; line-height: 1.8; word-spacing: 2px; letter-spacing: 2px; font-family: 'Helvetica Neue', Helvetica, 'Hiragino Sans GB', 'Microsoft YaHei', Arial, sans-serif; background-image: linear-gradient(90deg, rgba(50, 0, 0, 0.05) 3%, rgba(0, 0, 0, 0) 3%), linear-gradient(360deg, rgba(50, 0, 0, 0.05) 3%, rgba(0, 0, 0, 0) 3%); background-size: 20px 20px; background-position: center center;"><blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">你知道的越多,你不知道的越多</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">點贊再看,養成習慣</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">本文<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">GitHub</strong> <a href="https://github.com/AobingJava/JavaFamily" style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; text-decoration: none; color: rgb(30, 107, 184); overflow-wrap: break-word;">https://github.com/JavaFamily</a> 已收錄,有一線大廠面試點腦圖、我的聯繫方式和技術交流羣,歡迎Star和指教</p> </blockquote>html

<div class="output_wrapper" id="output_wrapper_id" style="font-size: 15px; line-height: 1.8; word-spacing: 2px; letter-spacing: 2px; font-family: 'Helvetica Neue', Helvetica, 'Hiragino Sans GB', 'Microsoft YaHei', Arial, sans-serif; background-image: linear-gradient(90deg, rgba(50, 0, 0, 0.05) 3%, rgba(0, 0, 0, 0) 3%), linear-gradient(360deg, rgba(50, 0, 0, 0.05) 3%, rgba(0, 0, 0, 0) 3%); background-size: 20px 20px; background-position: center center;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);"><a href="https://www.aliyun.com/minisite/goods?userCode=tybhsgp5&amp;share_source=copy_link" style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; text-decoration: none; color: rgb(30, 107, 184); overflow-wrap: break-word;">我幫阿里雲推廣服務器89/年,229/3年,買來送本身,送女友立刻過年再合適不過了,買了搭建個項目給面試官看也香,還能夠熟悉技術棧,我明天會出一個服務器搭建我的項目的教程。掃碼或者點擊購買</a></strong></p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9jmxonfwfj302s02sgld.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure></div> <h2 id="h" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.4em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">前言</span></h2> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">消息隊列</strong>在互聯網技術存儲方面使用如此普遍,幾乎全部的後端技術面試官都要在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">消息隊列</strong>的使用和原理方面對小夥伴們進行360°的刁難。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">做爲一個在互聯網公司面一次拿一次Offer的麪霸,戰勝了無數競爭對手,每次都只能看到無數落寞的身影失望的離開,略感愧疚(<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">請容許我使用一下誇張的修辭手法</strong>)。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">因而在一個寂寞難耐的夜晚,我痛定思痛,決定開始寫<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">《吊打面試官》</strong>系列,但願能幫助各位讀者之後面試勢如破竹,對面試官進行360°的反擊,吊打問你的面試官,讓一同面試的同僚瞠目結舌,瘋狂收割大廠Offer!</p> <h2 id="h-1" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.4em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">撈一下</span></h2> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">消息隊列系列前面兩章分別講了<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">消息隊列</strong>的基礎知識,還有比較常見的問題和常見分佈式事務解決方案,那麼在實際開發過程當中,咱們使用頻率比較高的消息隊列中間件有哪些呢?</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">帥丙我工做以來接觸的消息隊列中間件有<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Kafka</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">自研</strong>,是的由於我主要接觸的都是電商公司,相對而言業務體量還有場景來講都是他們比較適合,再加上杭州阿里系公司偏多,身邊同事或者公司老大基本都是阿里出來創業的,那在使用技術棧的時候<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">阿里系的開源框架</strong>也就成了首選。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">就算是自研的中間件多多少少也是借鑑RocketMQ、Kafka的優勢自研的,那我後面兩章就分別簡單的介紹下二者,他們分別在業務場景和大數據領域各自發光發熱。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">那究竟是道德的淪喪,仍是人性的泯滅,讓咱們跟着敖丙走進RocketMQ的心裏世界。</p> <h2 id="h-2" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.4em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">正文</span></h2> <h3 id="hrocketmq" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">RocketMQ簡介</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">RocketMQ是一個純Java、分佈式、隊列模型的開源消息中間件,前身是MetaQ,是阿里參考Kafka特色研發的一個隊列模型的消息中間件,後開源給apache基金會成爲了apache的頂級開源項目,具備高性能、高可靠、高實時、分佈式特色。</p> </blockquote> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hjvwekvkg309j06makn.gif" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">咱們再看下阿里給他取的名字哈:<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Rocket</strong> 火箭 阿里這是但願他上天呀,不過我以爲這個名字確實挺酷的。</p> <h3 id="h-3" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">咱們先看看他最新的官網</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g9953w22ogj30rr0o646c.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <h3 id="h-4" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">回顧一下他的心路歷程</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">2007年</strong>:淘寶實施了「五彩石」項目,「五彩石」用於將交易系統從單機變成分佈式,也是在這個過程當中產生了阿里巴巴第一代消息引擎——Notify。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">2010年</strong>:阿里巴巴B2B部門基於ActiveMQ的5.1版本也開發了本身的一款消息引擎,稱爲Napoli,這款消息引擎在B2B裏面普遍地被使用,不只僅是在交易領域,在不少的後臺異步解耦等方面也獲得了普遍的應用。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">2011年</strong>:業界出現瞭如今被不少大數據領域所推崇的Kafka消息引擎,阿里巴巴在研究了Kafka的總體機制和架構設計以後,基於Kafka的設計使用Java進行了徹底重寫並推出了<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">MetaQ 1.0</strong>版本,主要是用於解決順序消息和海量堆積的問題。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">2012年</strong>:阿里巴巴開源其自研的第三代分佈式消息中間件——<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">通過幾年的技術打磨,阿里稱基於RocketMQ技術,目前雙十一當天消息容量可達到萬億級。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">2016年11月</strong>:阿里將RocketMQ捐獻給<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Apache</strong>軟件基金會,正式成爲孵化項目。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">阿里稱會將其打形成頂級項目。<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">這是阿里邁出的一大步</strong>,由於加入到開源軟件基金會須要通過評審方的考覈與觀察。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">坦率而言,業界還對國人的代碼開源參與度仍保持着刻板印象;而Apache基金會中的342個項目中,暫時還只有Kylin、CarbonData、Eagle 、Dubbo和 RocketMQ 共計五個中國技術人主導的項目。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">2017年2月20日</strong>:RocketMQ正式發佈4.0版本,專家稱新版本適用於電商領域,金融領域,大數據領域,兼有物聯網領域的編程模型。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">以上就是RocketMQ的總體發展歷史,其實在阿里巴巴內部圍繞着RocketMQ內核打造了三款產品,分別是<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">MetaQ</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Notify</strong>和<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Aliware MQ</strong>。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">這三者分別採用了不一樣的模型,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">MetaQ</strong>主要使用了拉模型,解決了順序消息和海量堆積問題;<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Notify</strong>主要使用了推模型,解決了事務消息;而云產品<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Aliware MQ</strong>則是提供了商業化的版本。</p> <h3 id="h11" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">經歷屢次雙11洗禮的英雄</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">在備戰2016年雙十一時,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMq</strong>團隊重點作了<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">兩件事情</strong>,優化慢請求與統一存儲引擎。</p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">優化慢請求</strong>:這裏主要是解決在海量高併發場景降低低慢請求對整個集羣帶來的抖動,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">毛刺問題</strong>。這是一個極具挑戰的技術活,團隊同窗通過長達1個多月的跟進調優,從雙十一的覆盤狀況來看,99.996%的延遲落在了10ms之內,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">而99.6%的延遲在1ms之內</strong>。優化主要集中在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>存儲層算法優化、JVM與操做系統調優。更多的細節你們能夠參考《萬億級數據洪峯下的分佈式消息引擎》。</li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">統一存儲引擎</strong>:主要解決的消息引擎的高可用,成本問題。在多代消息引擎共存的前提下,咱們對Notify的存儲模塊進行了全面移植與替換。</li> </ul> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ天生爲金融互聯網領域而生</strong>,追求高可靠、高可用、高併發、低延遲,是一個阿里巴巴由內而外成功孕育的典範,除了阿里集團上千個應用外,根據咱們不徹底統計,國內至少有上百家單位、科研教育機構在使用。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>在阿里集團也被普遍應用在訂單,交易,充值,流計算,消息推送,日誌流式處理,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">binglog</strong>分發等場景。</p> <h3 id="h-5" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">他所擁有的功能</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">咱們直接去<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">GitHub</strong>上看<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Apache</strong>對他的描述可能會好點</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g98ystiybgj30np0jagpg.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">是的功能完整到爆炸基本上開發徹底夠用,什麼?看不懂專業詞彙的英文?</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">帥丙是暖男來的嘛,中文功能以下 ↓ </p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">發佈/訂閱消息傳遞模型</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">財務級交易消息</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">各類跨語言客戶端,例如Java,C / C ++,Python,Go</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">可插拔的傳輸協議,例如TCP,SSL,AIO</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">內置的消息跟蹤功能,還支持開放式跟蹤</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">多功能的大數據和流生態系統集成</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">按時間或偏移量追溯消息</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">可靠的FIFO和嚴格的有序消息傳遞在同一隊列中</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">高效的推拉消費模型</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">單個隊列中的百萬級消息累積容量</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">多種消息傳遞協議,例如JMS和OpenMessaging</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">靈活的分佈式橫向擴展部署架構</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">快如閃電的批量消息交換系統</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">各類消息過濾器機制,例如SQL和Tag</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">用於隔離測試和雲隔離羣集的Docker映像</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">功能豐富的管理儀表板,用於配置,指標和監視</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">認證與受權</span></li> </ul> <h3 id="h-6" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">他的項目結構組成是怎麼樣子的?</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">GitHub地址:https://github.com/apache/rocketmq</p> </blockquote> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g994wb4851j30b60jgn28.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">他的核心模塊:</p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-broker:接受生產者發來的消息並存儲(經過調用rocketmq-store),消費者從這裏取得消息</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-client:提供發送、接受消息的客戶端API。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-namesrv:NameServer,相似於Zookeeper,這裏保存着消息的TopicName,隊列等運行時的元信息。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-common:通用的一些類,方法,數據結構等。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-remoting:基於Netty4的client/server + fastjson序列化 + 自定義二進制協議。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-store:消息、索引存儲等。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-filtersrv:消息過濾器Server,須要注意的是,要實現這種過濾,須要上傳代碼到MQ!(通常而言,咱們利用Tag足以知足大部分的過濾需求,若是更靈活更復雜的過濾需求,能夠考慮filtersrv組件)。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">rocketmq-tools:命令行工具。</span></li> </ul> <h3 id="h-7" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">他的架構組成,或者理解爲爲何他這麼快?這麼強?這麼厲害?</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">他主要有四大核心組成部分:<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NameServer</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong>以及<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>四部分。</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g9908d8kj5j30nb0fwdh9.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Tip</strong>:咱們能夠看到<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>啥都是<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">集羣</strong>部署的,這是他<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">吞吐量大</strong>,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">高可用</strong>的緣由之一,集羣的模式也很花哨,能夠支持多master 模式、多master多slave異步複製模式、多 master多slave同步雙寫模式。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">並且這個模式好像Kafka啊!(我這裏是廢話,自己就是阿里基於Kafka的不少特性研發的)。</p> <h3 id="h-8" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">分別介紹下各個集羣組成部分吧</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <h3 id="hnameserver" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">NameServer:</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">主要負責對於源數據的管理,包括了對於<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong>和路由信息的管理。</p> </blockquote> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NameServer</strong>是一個功能齊全的服務器,其角色相似Dubbo中的Zookeeper,但NameServer與Zookeeper相比更輕量。主要是由於每一個NameServer節點互相之間是獨立的,沒有任何信息交互。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NameServer</strong>壓力不會太大,平時主要開銷是在維持心跳和提供Topic-Broker的關係數據。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">但有一點須要注意,Broker向NameServer發心跳時, 會帶上當前本身所負責的全部<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong>信息,若是<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong>個數太多(萬級別),會致使一次心跳中,就Topic的數據就幾十M,網絡狀況差的話, 網絡傳輸失敗,心跳失敗,致使NameServer誤認爲Broker心跳失敗。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NameServer</strong> 被設計成幾乎無狀態的,能夠橫向擴展,節點之間相互之間無通訊,經過部署多臺機器來標記本身是一個僞集羣。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">每一個 Broker 在啓動的時候會到 NameServer 註冊,Producer 在發送消息前會根據 Topic 到 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NameServer</strong> 獲取到 Broker 的路由信息,Consumer 也會定時獲取 Topic 的路由信息。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">因此從功能上看NameServer應該是和 ZooKeeper 差很少,聽說 RocketMQ 的早期版本確實是使用的 ZooKeeper ,後來改成了本身實現的 NameServer 。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">咱們看一下<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Dubbo</strong>中註冊中心的角色,是否是真的一毛同樣,師出同門類似點真的不少:</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hk6880vrj30qw0k5abb.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <h3 id="hproducer" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">Producer</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">消息生產者,負責產生消息,通常由業務系統負責產生消息。</p> </blockquote> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong>由用戶進行分佈式部署,消息由<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong>經過多種負載均衡模式發送到<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>集羣,發送低延時,支持快速失敗。</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong> 提供了三種方式發送消息:同步、異步和單向</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">同步發送</strong>:同步發送指消息發送方發出數據後會在收到接收方發回響應以後才發下一個數據包。通常用於重要通知消息,例如重要通知郵件、營銷短信。</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">異步發送</strong>:異步發送指發送方發出數據後,不等接收方發回響應,接着發送下個數據包,通常用於可能鏈路耗時較長而對響應時間敏感的業務場景,例如用戶視頻上傳後通知啓動轉碼服務。</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">單向發送</strong>:單向發送是指只負責發送消息而不等待服務器迴應且沒有回調函數觸發,適用於某些耗時很是短但對可靠性要求並不高的場景,例如日誌收集。</p></li> </ul> <h3 id="hbroker" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">Broker</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">消息中轉角色,負責<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">存儲消息</strong>,轉發消息。</p> </blockquote> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>是具體提供業務的服務器,單個Broker節點與全部的NameServer節點保持長鏈接及心跳,並會定時將<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong>信息註冊到NameServer,順帶一提底層的通訊和鏈接都是<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">基於Netty實現</strong>的。</li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>負責消息存儲,以Topic爲緯度支持輕量級的隊列,單機能夠支撐上萬隊列規模,支持消息推拉模型。</li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;">官網上有數據顯示:具備<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">上億級消息堆積能力</strong>,同時可<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">嚴格保證消息的有序性</strong>。</li> </ul> <h3 id="hconsumer" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">Consumer</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">消息消費者,負責消費消息,通常是後臺系統負責異步消費。</p> </blockquote> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>也由用戶部署,支持PUSH和PULL兩種消費模式,支持<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">集羣消費</strong>和<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">廣播消息</strong>,提供<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">實時的消息訂閱機制</strong>。</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Pull</strong>:拉取型消費者(Pull Consumer)主動從消息服務器拉取信息,只要批量拉取到消息,用戶應用就會啓動消費過程,因此 Pull 稱爲主動消費型。</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Push</strong>:推送型消費者(Push Consumer)封裝了消息的拉取、消費進度和其餘的內部維護工做,將消息到達時執行的回調接口留給用戶應用程序來實現。因此 Push 稱爲被動消費類型,但從實現上看仍是從消息服務器中拉取消息,不一樣於 Pull 的是 Push 首先要註冊消費監聽器,當監聽器處觸發後纔開始消費消息。</p></li> </ul> <h3 id="h-9" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">消息領域模型</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://img-blog.csdn.net/20180129202958831" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <h4 id="hmessage" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Message</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Message</strong>(消息)就是要傳輸的信息。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">一條消息必須有一個主題(Topic),主題能夠看作是你的信件要郵寄的地址。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">一條消息也能夠擁有一個可選的標籤(Tag)和額處的鍵值對,它們能夠用於設置一個業務 Key 並在 Broker 上查找此消息以便在開發期間查找問題。</p> <h4 id="htopic" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Topic</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong>(主題)能夠看作消息的規類,它是消息的第一級類型。好比一個電商系統能夠分爲:交易消息、物流消息等,一條消息必須有一個 Topic 。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong> 與生產者和消費者的關係很是鬆散,一個 Topic 能夠有0個、1個、多個生產者向其發送消息,一個生產者也能夠同時向不一樣的 Topic 發送消息。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">一個 Topic 也能夠被 0個、1個、多個消費者訂閱。</p> <h4 id="htag" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Tag</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Tag</strong>(標籤)能夠看做子主題,它是消息的第二級類型,用於爲用戶提供額外的靈活性。使用標籤,同一業務模塊不一樣目的的消息就能夠用相同 Topic 而不一樣的 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Tag</strong> 來標識。好比交易消息又能夠分爲:交易建立消息、交易完成消息等,一條消息能夠沒有 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Tag</strong> 。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">標籤有助於保持您的代碼乾淨和連貫,而且還能夠爲 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong> 提供的查詢系統提供幫助。</p> <h4 id="hgroup" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Group</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">分組,一個組能夠訂閱多個Topic。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">分爲ProducerGroup,ConsumerGroup,表明某一類的生產者和消費者,通常來講同一個服務能夠做爲Group,同一個Group通常來講發送和消費的消息都是同樣的</p> <h4 id="hqueue" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Queue</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Kafka</strong>中叫Partition,每一個Queue內部是有序的,在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>中分爲讀和寫兩種隊列,通常來講讀寫隊列數量一致,若是不一致就會出現不少問題。</p> <h4 id="hmessagequeue" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Message Queue</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Message Queue</strong>(消息隊列),主題被劃分爲一個或多個子主題,即消息隊列。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">一個 Topic 下能夠設置多個消息隊列,發送消息時執行該消息的 Topic ,RocketMQ 會輪詢該 Topic 下的全部隊列將消息發出去。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">消息的物理管理單位。一個Topic下能夠有多個Queue,Queue的引入使得消息的存儲能夠分佈式集羣化,具備了水平擴展能力。</p> <h4 id="hoffset" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Offset</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong> 中,全部消息隊列都是持久化,長度無限的數據結構,所謂長度無限是指隊列中的每一個存儲單元都是定長,訪問其中的存儲單元使用Offset 來訪問,Offset 爲 java long 類型,64 位,理論上在 100年內不會溢出,因此認爲是長度無限。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">也能夠認爲 Message Queue 是一個長度無限的數組,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Offset</strong> 就是下標。</p> <h4 id="h-10" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">消息消費模式</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">消息消費模式有兩種:<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Clustering</strong>(集羣消費)和<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broadcasting</strong>(廣播消費)。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">默認狀況下就是集羣消費,該模式下一個消費者集羣共同消費一個主題的多個隊列,一個隊列只會被一個消費者消費,若是某個消費者掛掉,分組內其它消費者會接替掛掉的消費者繼續消費。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">而廣播消費消息會發給消費者組中的每個消費者進行消費。</p> <h4 id="hmessageorder" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Message Order</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Message Order</strong>(消息順序)有兩種:<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Orderly</strong>(順序消費)和<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Concurrently</strong>(並行消費)。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">順序消費表示消息消費的順序同生產者爲每一個消息隊列發送的順序一致,因此若是正在處理全局順序是強制性的場景,須要確保使用的主題只有一個消息隊列。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">並行消費再也不保證消息順序,消費的最大並行數量受每一個消費者客戶端指定的線程池限制。</p> <h3 id="h-11" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">一次完整的通訊流程是怎樣的?</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">Producer 與 NameServer集羣中的其中一個節點(隨機選擇)創建長鏈接,按期從 NameServer 獲取 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong> 路由信息,並向提供 Topic 服務的 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker Master</strong> 創建長鏈接,且定時向 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong> 發送心跳。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong> 只能將消息發送到 Broker master,可是 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong> 則不同,它同時和提供 Topic 服務的 Master 和 Slave創建長鏈接,既能夠從 Broker Master 訂閱消息,也能夠從 Broker Slave 訂閱消息。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">具體以下圖:</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g990ne2odpj30bj07kjrq.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">我上面說過他跟<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Dubbo</strong>像不是我瞎說的,就連他的註冊過程都很像<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Dubbo</strong>的服務暴露過程。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">是否是以爲很簡單,可是你同時也產生了好奇心,每一步是怎麼初始化啓動的呢?</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">帥丙呀就知道你們都是<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">求知慾極強的人才</strong>,這不我都準備好了,咱們一步步分析一下。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">主要是人才羣裏的仔要求我寫出來。。。(文末有進羣方式)</p> <h4 id="hnameservice" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">NameService啓動流程</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">在org.apache.rocketmq.namesrv目錄下的<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NamesrvStartup</strong>這個啓動類基本上描述了他的啓動過程咱們能夠看一下代碼:</p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">第一步是初始化配置</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">建立<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NamesrvController</strong>實例,並開啓兩個定時任務:</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">每隔10s掃描一次<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>,移除處於不激活的<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>;</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">每隔10s打印一次KV配置。</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hk0fn5l5j315u0na7c0.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">第三步註冊鉤子函數,啓動服務器並監聽Broker。</p></li> </ul> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NameService</strong>還有不少東西的哈我這裏就介紹他的啓動流程,你們還能夠去看看代碼,仍是頗有意思的,好比路由註冊會發送心跳包,還有<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">心跳包的處理流程</strong>,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">路由刪除</strong>,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">路由發現</strong>等等。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Tip</strong>:原本我想貼不少源碼的,後面跟<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">歪歪(Java3y)</strong>討論了好久作出了不貼的決定,你們理解過程爲主!我主要是作只是掃盲還有一些痛點分析嘛,深究仍是得你們花時間,我要啥都介紹篇幅就不夠了。</p> <h4 id="hproducer-1" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Producer</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">鏈路很長涉及的細節也多,我就發一下鏈路圖。</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hixadhgfj30wl0icabx.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong>是消息發送方,那他怎麼發送的呢?</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">經過輪訓,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong>輪訓某個<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong>下面的全部隊列實現發送方的負載均衡</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g9983lv0bqj30fj076mxg.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <h4 id="hbroker-1" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Broker</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>在RocketMQ中是進行處理Producer發送消息請求,Consumer消費消息的請求,而且進行消息的持久化,以及HA策略和服務端過濾,就是集羣中很重的工做都是交給了<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>進行處理。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>模塊是經過BrokerStartup進行啓動的,會實例化BrokerController,而且調用其初始化方法</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g998c3slw4j30sg0gpwhf.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">你們去看<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>的源碼的話會發現,他的<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">初始化流程很冗長</strong>,會根據配置建立不少線程池主要用來<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">發送消息</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">拉取消息</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">查詢消息</strong>、<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">客戶端管理</strong>和<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">消費者管理</strong>,也有不少<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">定時任務</strong>,同時也註冊了不少<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">請求處理器</strong>,用來發送拉取消息查詢消息的。</p> <h4 id="hconsumer-1" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Consumer</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">不說了直接懟圖吧!要死了,下次我仍是作掃盲,寫點爽文吧555</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hh97vkpuj30s50fwdin.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>是消息接受,那他怎麼接收消息的呢?</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hhknjydzj30g9073q36.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">消費端會經過<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RebalanceService</strong>線程,10秒鐘作一次基於<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Topic</strong>下的全部隊列負載。</p> <h2 id="h-12" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.4em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">面試常見問題分析</span></h2> <h3 id="h-13" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">他的優缺點是啥</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">RocketMQ優勢:</p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">單機吞吐量:十萬級</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">可用性:很是高,分佈式架構</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">消息可靠性:通過參數優化配置,消息能夠作到0丟失</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">功能支持:MQ功能較爲完善,仍是分佈式的,擴展性好</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">支持10億級別的消息堆積,不會由於堆積致使性能降低</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">源碼是java,咱們能夠本身閱讀源碼,定製本身公司的MQ,能夠掌控 </p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">天生爲金融互聯網領域而生,對於可靠性要求很高的場景,尤爲是電商裏面的訂單扣款,以及業務削峯,在大量交易涌入時,後端可能沒法及時處理的狀況</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RoketMQ</strong>在穩定性上可能更值得信賴,這些業務場景在阿里雙11已經經歷了屢次考驗,若是你的業務有上述併發場景,建議能夠選擇<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong></p></li> </ul> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">RocketMQ缺點:</p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">支持的客戶端語言很少,目前是java及c++,其中c++不成熟</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">社區活躍度不是特別活躍那種</p></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">沒有在 mq 核心中去實現<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">JMS</strong>等接口,有些系統要遷移須要修改大量代碼 </p></li> </ul> <h3 id="h-14" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">消息去重</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">去重原則:使用業務端邏輯保持冪等性</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">冪等性</strong>:就是用戶對於同一操做發起的一次請求或者屢次請求的結果是一致的,不會由於屢次點擊而產生了反作用,數據庫的結果都是惟一的,不可變的。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">只要保持冪等性,無論來多少條重複消息,最後處理的結果都同樣,須要業務端來實現。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">去重策略</strong>:保證每條消息都有惟一編號(<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">好比惟一流水號)</strong>,且保證消息處理成功與去重表的日誌同時出現。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">創建一個消息表,拿到這個消息作數據庫的insert操做。給這個消息作一個惟一主鍵(primary key)或者惟一約束,那麼就算出現重複消費的狀況,就會致使主鍵衝突,那麼就再也不處理這條消息。</p> <h3 id="h-15" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">消息重複</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">消息領域有一個對消息投遞的QoS定義,分爲:</p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">最多一次(At most once)</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">至少一次(At least once)</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">僅一次( Exactly once)</span></li> </ul> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">QoS:Quality of Service,服務質量</p> </blockquote> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">幾乎全部的MQ產品都聲稱本身作到了<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">At least once</strong>。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">既然是至少一次,那避免不了消息重複,尤爲是在分佈式網絡環境下。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">好比:網絡緣由閃斷,ACK返回失敗等等故障,確認信息沒有傳送到消息隊列,致使消息隊列不知道本身已經消費過該消息了,再次將該消息分發給其餘的消費者。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">不一樣的消息隊列發送的確認信息形式不一樣,例如<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RabbitMQ</strong>是發送一個ACK確認消息,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>是返回一個CONSUME_SUCCESS成功標誌,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Kafka</strong>實際上有個offset的概念。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>沒有內置消息去重的解決方案,最新版本是否支持還需確認。</p> <h3 id="h-16" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">消息的可用性</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">當咱們選擇好了集羣模式以後,那麼咱們須要關心的就是怎麼去存儲和複製這個數據,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>對消息的刷盤提供了同步和異步的策略來知足咱們的,當咱們選擇同步刷盤以後,若是刷盤超時會給返回FLUSH_DISK_TIMEOUT,若是是異步刷盤不會返回刷盤相關信息,選擇同步刷盤能夠盡最大程度知足咱們的消息不會丟失。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">除了存儲有選擇以後,咱們的主從同步提供了同步和異步兩種模式來進行復制,固然選擇同步能夠提高可用性,可是消息的發送RT時間會降低10%左右。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>採用的是混合型的存儲結構,即爲<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>單個實例下全部的隊列共用一個日誌數據文件(即爲CommitLog)來存儲。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">而<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Kafka</strong>採用的是獨立型的存儲結構,每一個隊列一個文件。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">這裏帥丙認爲,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>採用混合型存儲結構的缺點在於,會存在較多的隨機讀操做,所以讀的效率偏低。同時消費消息須要依賴<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">ConsumeQueue</strong>,構建該邏輯消費隊列須要必定開銷。</p> <h3 id="hrocketmq-1" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">RocketMQ 刷盤實現</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong> 在消息的存取時直接操做的是內存(內存映射文件),這能夠提供系統的吞吐量,可是沒法避免機器掉電時數據丟失,因此須要持久化到磁盤中。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">刷盤的最終實現都是使用<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">NIO</strong>中的 MappedByteBuffer.force() 將映射區的數據寫入到磁盤,若是是同步刷盤的話,在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>把消息寫到<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">CommitLog</strong>映射區後,就會等待寫入完成。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">異步而言,只是喚醒對應的線程,不保證執行的時機,流程如圖所示。</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9fzta1rbij30d30b40vj.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <h3 id="h-17" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">順序消息:</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">我簡單的說一下咱們使用的<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>裏面的一個簡單實現吧。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Tip</strong>:爲啥用<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>舉例呢,這玩意是阿里開源的,我問了下身邊的朋友不少公司都有使用,因此讀者大機率是這個的話我就用這個舉例吧,具體的細節我後面會在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>和<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Kafka</strong>各自章節說到。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">生產者消費者通常須要保證順序消息的話,可能就是一個業務場景下的,好比訂單的建立、支付、發貨、收貨。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">那這些東西是否是一個訂單號呢?一個訂單的確定是一個訂單號的說,那簡單了呀。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">一個topic下有多個隊列</strong>,爲了保證發送有序,<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>提供了<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">MessageQueueSelector</strong>隊列選擇機制,他有三種實現:</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://img-blog.csdnimg.cn/20181128124324615.png" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">咱們可以使用<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Hash取模法</strong>,讓同一個訂單發送到同一個隊列中,再使用同步發送,只有同個訂單的建立消息發送成功,再發送支付消息。這樣,咱們保證了發送有序。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>的topic內的隊列機制,能夠保證存儲知足<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">FIFO</strong>(First Input First Output 簡單說就是指先進先出),剩下的只須要消費者順序消費便可。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>僅保證順序發送,順序消費由消費者業務保證!!!</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">這裏很好理解,一個訂單你發送的時候放到一個隊列裏面去,你同一個的訂單號Hash一下是否是仍是同樣的結果,那確定是一個消費者消費,那順序是否是就保證了?</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">真正的順序消費不一樣的中間件都有本身的不一樣實現我這裏就舉個例子,你們思路理解下。</p> <h3 id="h-18" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">分佈式事務:</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <h4 id="hhalfmessage" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">Half Message(半消息)</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">是指暫不能被Consumer消費的消息</strong>。Producer 已經把消息成功發送到了 Broker 端,但此消息被標記爲<code style="font-size: inherit; line-height: inherit; overflow-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0px 2px; color: rgb(248, 35, 117); background: rgb(248, 248, 248);">暫不能投遞</code>狀態,處於該種狀態下的消息稱爲半消息。須要 Producer</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">對消息的<code style="font-size: inherit; line-height: inherit; overflow-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0px 2px; color: rgb(248, 35, 117); background: rgb(248, 248, 248);">二次確認</code>後,Consumer才能去消費它。</p> <h4 id="h-19" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.2em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">消息回查</span></h4> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">因爲網絡閃段,生產者應用重啓等緣由。致使 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong> 端一直沒有對 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Half Message(半消息)</strong> 進行 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">二次確認</strong>。這是<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Brock</strong>服務器會定時掃描<code style="font-size: inherit; line-height: inherit; overflow-wrap: break-word; padding: 2px 4px; border-radius: 4px; margin: 0px 2px; color: rgb(248, 35, 117); background: rgb(248, 248, 248);">長期處於半消息的消息</code>,會</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">主動詢問 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Producer</strong>端 該消息的最終狀態(<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Commit或者Rollback</strong>),該消息即爲 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">消息回查</strong>。</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hhqfstm1j31do0fgjy9.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <ol style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: decimal;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">A服務先發送個Half Message給Brock端,消息中攜帶 B服務 即將要+100元的信息。 </span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">當A服務知道Half Message發送成功後,那麼開始第3步執行本地事務。 </span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">執行本地事務(會有三種狀況一、執行成功。二、執行失敗。三、網絡等緣由致使沒有響應) </span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">若是本地事務成功,那麼Product像Brock服務器發送Commit,這樣B服務就能夠消費該message。 </span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">若是本地事務失敗,那麼Product像Brock服務器發送Rollback,那麼就會直接刪除上面這條半消息。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">若是由於網絡等緣由遲遲沒有返回失敗仍是成功,那麼會執行RocketMQ的回調接口,來進行事務的回查。</span></li> </ol> <h3 id="h-20" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">消息過濾</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>端消息過濾  <br>在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>中,按照<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>的要求作過濾,優勢是減小了對於<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>無用消息的網絡傳輸。缺點是增長了Broker的負擔,實現相對複雜。</li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>端消息過濾<br>這種過濾方式可由應用徹底自定義實現,可是缺點是不少無用的消息要傳輸到<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>端。</li> </ul> <h3 id="hbrokerbuffer" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">Broker的Buffer問題</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">Broker的<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>一般指的是Broker中一個隊列的內存Buffer大小,這類<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>一般大小有限。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">另外,RocketMQ沒有內存<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>概念,RocketMQ的隊列都是持久化磁盤,數據按期清除。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">RocketMQ同其餘MQ有很是顯著的區別,RocketMQ的內存<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>抽象成一個無限長度的隊列,無論有多少數據進來都能裝得下,這個無限是有前提的,Broker會按期刪除過時的數據。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">例如Broker只保存3天的消息,那麼這個<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>雖然長度無限,可是3天前的數據會被從隊尾刪除。</p> <h3 id="h-21" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">回溯消費</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">回溯消費是指Consumer已經消費成功的消息,因爲業務上的需求須要從新消費,要支持此功能,Broker在向Consumer投遞成功消息後,消息仍然須要保留。而且從新消費通常是按照時間維度。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">例如因爲Consumer系統故障,恢復後須要從新消費1小時前的數據,那麼Broker要提供一種機制,能夠按照時間維度來回退消費進度。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>支持按照時間回溯消費,時間維度精確到毫秒,能夠向前回溯,也能夠向後回溯。</p> <h3 id="h-22" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">消息堆積</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">消息中間件的主要功能是異步解耦,還有個重要功能是擋住前端的數據洪峯,保證後端系統的穩定性,這就要求消息中間件具備必定的消息堆積能力,消息堆積分如下兩種狀況:</p> <ul style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; padding-left: 32px; list-style-type: disc;"> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;">消息堆積在內存<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>,一旦超過內存<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>,能夠根據必定的丟棄策略來丟棄消息,如CORBA Notification規範中描述。適合能容忍丟棄消息的業務,這種狀況消息的堆積能力主要在於內存<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Buffer</strong>大小,並且消息堆積後,性能降低不會太大,由於內存中數據多少對於對外提供的訪問能力影響有限。</li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">消息堆積到持久化存儲系統中,例如DB,KV存儲,文件記錄形式。 當消息不能在內存Cache命中時,要不可避免的訪問磁盤,會產生大量讀IO,讀IO的吞吐量直接決定了消息堆積後的訪問能力。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">評估消息堆積能力主要有如下四點:</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">消息能堆積多少條,多少字節?即消息的堆積容量。</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">消息堆積後,發消息的吞吐量大小,是否會受堆積影響?</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">消息堆積後,正常消費的Consumer是否會受影響?</span></li> <li style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px; margin-bottom: 0.5em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">消息堆積後,訪問堆積在磁盤的消息時,吞吐量有多大?</span></li> </ul> <h3 id="h-23" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; border-bottom: 2px solid rgb(128, 128, 128); font-size: 1.3em;"><span style="font-size: inherit; line-height: inherit; margin: 0px; display: inline-block; font-weight: normal; background: rgb(128, 128, 128); color: rgb(255, 255, 255); padding: 3px 10px 1px; border-top-right-radius: 3px; border-top-left-radius: 3px; margin-right: 3px;">定時消息</span><span style="display: inline-block; vertical-align: bottom; border-right: 20px solid transparent;"> </span></h3> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">定時消息是指消息發到<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>後,不能馬上被<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Consumer</strong>消費,要到特定的時間點或者等待特定的時間後才能被消費。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">若是要支持任意的時間精度,在<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Broker</strong>層面,必需要作消息排序,若是再涉及到持久化,那麼消息排序要不可避免的產生巨大性能開銷。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">RocketMQ</strong>支持定時消息,可是不支持任意時間精度,支持特定的level,例如定時5s,10s,1m等。</p> <h2 id="h-24" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.4em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">總結</span></h2> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">寫這種單純介紹中間件的枯燥乏味,你們看起來估計也累,目前已經破一萬個字了,之後我這種類型的少寫,你們總是讓我寫點深度的,我說真的不少東西我源碼一貼,看都沒人看。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">Kafka</strong>我就不發博客了,你們能夠去<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">GItHub</strong>上第一時間閱讀,後面會出怎麼搭建項目在服務器的教程,還有一些大牛我的經歷和我的書單的東西,今年應該先這麼寫,主要是真心太忙了,望理解。</p> <h2 id="h-25" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.4em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">絮叨</span></h2> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hj3o7hm1j30nj0igq5x.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hj434k7vj30re0fxq4c.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">我也不過多描述了,反正嘛網絡上重拳出擊嘛,現實中惟惟諾諾,讓他說理由也說不出來,不回我。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">他說的是下面這個場景多線程的狀況,就是第一個線程還沒走完,第二個如今進來,也判斷沒處理過那不就兩個都繼續加了麼?</p> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006tNbRwly1g9hj5k3v0ej30910iv0wh.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;"><strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">訂單號+業務場景</strong>,組成一個惟一主鍵,你插入數據庫只能成功第一個,後續的都會報錯的,報違反惟一主鍵的錯誤。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">還有就是有人疑惑爲啥不直接就不判斷就等他插入的時候報錯,丟掉後續的就行了?</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">你要知道<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">報錯有不少種</strong>,你哪裏知道不是數據庫掛了的錯?或者別的運行時異常?</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">不過你若是能夠作到拋特定的異常也能夠,反正咱們要<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">減小數據庫的報錯</strong>,若是併發大,像我如今負責的系統都是10W+QPS,那日誌會打滿瘋狂報警的。(就是正常狀況咱們都常常報警)</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">解決問題的思路有不少,噴我能夠,講清楚問題,講清楚你的理由。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">不少你們都只是單方面的知識攝入,就這樣還要噴我,還有一上來就問我爲啥今天沒發文章,我欠你的?我工做日上班,週六週日都懟上去了,時間有限啊,哥哥。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">你們都有本身的事情,寫文章也耗時耗腦,不免出錯,還望理解。</p> <h2 id="h-26" style="color: inherit; line-height: inherit; padding: 0px; margin: 1.6em 0px; font-weight: bold; font-size: 1.4em;"><span style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;">平常求贊</span></h2> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">好了各位,以上就是這篇文章的所有內容了,能看到這裏的人呀,都是<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">人才</strong>。</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">我後面會每週都更新幾篇《吊打面試官》系列和互聯網經常使用技術棧相關的文章,很是感謝<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">人才</strong>們能看到這裏,若是這個文章寫得還不錯,以爲「敖丙」我<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">有點東西</strong>的話 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">求點贊👍</strong> <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">求關注❤️</strong> <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">求分享👥</strong> 對暖男我來講真的 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">很是有用</strong>!!!</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">創做不易,各位的支持和承認,就是我創做的最大動力,咱們下篇文章見!</p> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 1.7em 0px;">敖丙 | 文 【原創】【轉載請聯繫本人】 若是本篇博客有任何錯誤,請批評指教,不勝感激 !</p> <hr style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; height: 1px; margin: 1.5rem 0px; border-right: none; border-bottom: none; border-left: none; border-image: initial; border-top: 1px dashed rgb(165, 165, 165);"> <blockquote style="line-height: inherit; display: block; padding: 15px 15px 15px 1rem; font-size: 0.9em; margin: 1em 0px; color: rgb(0, 0, 0); border-left: 5px solid rgb(239, 112, 96); background: rgb(239, 235, 233); overflow: auto; overflow-wrap: normal; word-break: normal;"> <p style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; margin: 0px;">《吊打面試官》系列每週持續更新,能夠關注個人公衆號「 <strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">JavaFamily</strong> 」第一時間閱讀和催更(公衆號比博客早一到兩篇喲),本文<strong style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; font-weight: bold; color: rgb(233, 105, 0);">GitHub</strong><a href="https://github.com/AobingJava/JavaFamily" style="font-size: inherit; line-height: inherit; margin: 0px; padding: 0px; text-decoration: none; color: rgb(30, 107, 184); overflow-wrap: break-word;">https://github.com/JavaFamily</a> 已收錄,有一線大廠面試點思惟導圖,歡迎Star和完善,裏面也有我我的聯繫方式有什麼問題也能夠直接找我,也有技術交流羣,咱們一塊兒有點東西。</p> </blockquote> <figure style="font-size: inherit; color: inherit; line-height: inherit; margin: 0px; padding: 0px;"><img src="https://tva1.sinaimg.cn/large/006y8mN6ly1g9e2keyxw6j30m80nmjyj.jpg" alt="" title="" style="font-size: inherit; color: inherit; line-height: inherit; padding: 0px; display: block; margin: 0px auto; max-width: 100%;"><figcaption style="line-height: inherit; margin: 0px; padding: 0px; margin-top: 10px; text-align: center; color: rgb(153, 153, 153); font-size: 0.7em;"></figcaption></figure></div>前端

原文出處:https://www.cnblogs.com/aobing/p/11967997.htmljava

相關文章
相關標籤/搜索