上節咱們剩餘Compare和Fix按鈕沒有講,本節咱們就來說一下Compare按鈕的功能,年末了,要開年會了,一個個積極的跟邱少雲似得,給員工漲工資的時候能不能也積極點。不說了,說多了都是淚。html
還記得我以前寫的大數據實戰之環境搭建,咱們今天主要是拿DB的數據和solr的數據做比較,得出比較結果。首先,先啓動centOS,啓動tomcat,這樣咱們才能利用SolrNet調用solr公開的API。關於solr的啓動再次我就再也不贅述。ajax
OK,咱們將solr實例啓動起來後,用firefox瀏覽管理界面,沒問題。目前solr中並無任何數據。sql
接下來咱們就要看點擊Compare按鈕了,點擊Compare按鈕,Compare按鈕在哪裏呢,在Partial頁。數據庫
@using Bruce.GRLC.Model.ViewModel; @model UserInfoViewModel <div class="col-md-6 form-group"> <label class="control-label"> The Same: </label> <select id="selsamelist" class="form-control" multiple="multiple" style="max-height:500px;min-height:400px"> @if (Model!=null&&Model.SameWithSolrEntityList != null && Model.SameWithSolrEntityList.Count > 0) { foreach (var entity in Model.SameWithSolrEntityList) { <option value="@entity.UserNo">@entity.Name</option> } } </select> <input id="btncompare" type="button" class="btn btn-info" value="Compare" style="width: 80px;margin-top:5px" /> </div> <div class="col-md-6 form-group"> <label class="control-label"> The Difference: </label> <select id="seldifflist" class="form-control" multiple="multiple" style="max-height: 500px;min-height: 400px"> @if (Model.DifferenceWithSolrEntityList != null && Model.DifferenceWithSolrEntityList.Count > 0) { foreach (var entity in Model.DifferenceWithSolrEntityList) { <option value="@entity.UserNo">@entity.Name</option> } } </select> <input type="button" class="btn btn-info" value="Fix" style="width:80px;margin-top:5px" /> </div>
OK,咱們看到了按鈕在這裏,咱們設置按鈕的樣式爲btn btn-info,這都是bootStrap提供的樣式,咱們直接拿來用就ok了。這個頁面其實就是循環相同數據的List和不一樣數據的List,加載到一個可多選的下拉列表。你們注意到multiple="multiple",能夠多選。另外"form-control"也是bootStrap提供的樣式,用來裝飾表單元素的樣式。上面的代碼沒有什麼,咱們主要是看控制器一級一級往下是怎麼處理的,咱們先看Compare按鈕的click事件。c#
$("#btncompare").click(function () { $("#selsamelist").empty(); $("#seldifflist").empty(); $.ajax({ url: "/Home/GetCompareResult?pam=" + new Date().toTimeString(), type: "POST", datatype: "Html", beforeSend: function () { $("#divcompare").show(); }, complete: function () { $("#divcompare").hide(); }, success: function (data) { $("#divcompareresult").html(data); }, error: function () { alert("比較失敗!"); } });
首先咱們先清除兩個多選下拉列表的數據,由於咱們這裏並非返回整個Partial頁面來替換,因此咱們必須先把兩個多選下拉數據清除掉。OK,咱們看一下控制器。數組
public PartialViewResult GetCompareResult() { GRLCBiz instance = GRLCBiz.GetInstance(); instance.CompareDBAndSolr(); UserInfoViewModel userInfoViewModel = new UserInfoViewModel(); userInfoViewModel.DifferenceWithSolrEntityList = instance.differenceUserEntityList; List<string> differenceUserIDList = userInfoViewModel.DifferenceWithSolrEntityList.Select(d => d.UserNo.Trim()).ToList(); userInfoViewModel.SameWithSolrEntityList = instance.userEntityList.Where(u => !differenceUserIDList.Contains(u.UserID.Trim())) .Select((userDB, userSolr) => { return new UserSolrEntity() { UserNo = userDB.UserID.Trim(), Name = userDB.UserName == null ? string.Empty : userDB.UserName.Trim() }; }).ToList(); return PartialView("~/Views/Partial/DiffAndSameWithSolrPartial.cshtml", userInfoViewModel); }
其實很簡單,首先咱們調用CompareDBAndSolr方法,這個方法是獲取到比較結果的關鍵。那麼比較的話,咱們怎麼比較呢,你們有經驗的同窗必定想到多線程。不錯,就是多線程,但是有人要說了,多線程的話,你怎麼知道線程都執行完了,由於只有多個線程都執行完了,咱們才能拿到最終的比較結果,顯示到頁面上。說到這裏,若是你是一個.net4.0之前的用戶,你可能會想到AutoResetEvent。tomcat
說到這個AutoResetEvent,它能夠有一個初始的狀態,若是是true,則代表是一個終止狀態,反之則是非終止狀態,那麼看一下咱們的程序。多線程
public void CompareDBAndSolr() { if (userEntityList == null) { userEntityList = this.GetAllDBUserList(); } if (userEntityList == null || userEntityList.Count == 0) return; int threadCount = 0; int totalCount = userEntityList.Count; threadCount = totalCount % ConstValues.CONN_ComparePerThread == 0 ? totalCount / ConstValues.CONN_ComparePerThread : totalCount / ConstValues.CONN_ComparePerThread + 1; if (threadCount > ConstValues.CONN_CompareThreadCount) { threadCount = ConstValues.CONN_CompareThreadCount; } differenceUserEntityList = new List<UserSolrEntity>(); autoResetEvents = new AutoResetEvent[threadCount]; for (int i = 0; i < threadCount; i++) { autoResetEvents[i] = new AutoResetEvent(false); ThreadPool.QueueUserWorkItem(new WaitCallback(CompareUserInfoByThread), i); Thread.Sleep(ConstValues.CONN_ThreadCreateInterval); } WaitHandle.WaitAll(this.autoResetEvents); }
在這個方法中咱們先拿到數據庫的全部用戶的數據ide
SELECT A.UseNo, ISNULL(B.Name,'') AS Name, ISNULL(B.Age,0) AS Age, ISNULL(B.Temper,'') AS Married FROM Bonus.dbo.[User] A WITH(NOLOCK) INNER JOIN Bonus.dbo.UerInfo B WITH(NOLOCK) ON A.UseNo = B.UseNo
而後計算應該使用的線程數。CONN_ComparePerThread配置的是每一個線程比較的數據量,因此咱們先拿總數據對其進行求餘,獲得總線程數,而後再判斷若是總線程數大於配置的線程數CONN_CompareThreadCount,則取配置的線程數。由於機器的資源有限,不可能開啓成百上千個線程,那樣CPU資源佔用很大,機器肯能會卡死。因此設置最大線程數是必須的。當咱們拿到線程數之後,咱們實例化一個AutoResetEvent數組,來管理這些線程之間的通訊。接下來循環建立AutoResetEvent,設置其初始狀態爲非終止。而後將線程要執行的方法加入線程池工做隊列,並傳遞線程編號i做爲方法參數。最後這句WaitHandle.WaitAll(this.autoResetEvents);意思是等待全部的線程任務結束(狀態標記爲終止狀態)。大數據
咱們接下來看CompareUserInfoByThread方法。
private void CompareUserInfoByThread(object userState) { int threadIndex = (int)userState; try { UserDBEntity[] copyUserDBEntityList = null; while (this.movePosition < this.userEntityList.Count) { lock (this.userEntityList) { if (this.movePosition >= this.userEntityList.Count) { break; } if (this.movePosition <= this.userEntityList.Count - ConstValues.CONN_ComparePerThread) { copyUserDBEntityList = new UserDBEntity[ConstValues.CONN_ComparePerThread]; userEntityList.CopyTo(this.movePosition, copyUserDBEntityList, 0, ConstValues.CONN_ComparePerThread); } else { //最後幾個,count<CONN_ComparePerThread copyUserDBEntityList = new UserDBEntity[userEntityList.Count - this.movePosition]; userEntityList.CopyTo(this.movePosition, copyUserDBEntityList, 0, copyUserDBEntityList.Length); } this.movePosition += ConstValues.CONN_ComparePerThread; } this.CompareUserInfoStart(copyUserDBEntityList, threadIndex); } } catch (Exception ex) { LogHelper.WriteExceptionLog(MethodBase.GetCurrentMethod(), ex); } finally { this.autoResetEvents[threadIndex].Set(); } }
這個方法其實就是多線程瓜分數據了,每一個線程Copy出一批(CONN_ComparePerThread)DB的數據去和solr做對比,每一個線程拿到本身的那批數據後,將計數器movePosition增長CONN_ComparePerThread。直到這些線程將這些數據瓜分完,你們注意到在finally語句塊,咱們對每一個AutoResetEvent對象調用set方法,意思是告訴WaitHandle,我已經執行完了,即終止狀態。這樣當WaitHanlde收到每一個線程執行完畢的信號後,結束等待,不然就會一直等待下去,這就是爲何Set方法的調用必定要放到finally塊的緣由。OK,繼續看下一個方法CompareUserInfoStart
private void CompareUserInfoStart(UserDBEntity[] userDBEntityList, int threadIndex) { List<string> userIDList = userDBEntityList.Select(u => u.UserID.Trim()).ToList(); StringBuilder solrFilter = new StringBuilder(); foreach (var userID in userIDList) { solrFilter.Append("UserNo:"); solrFilter.Append(userID); solrFilter.Append(" OR "); } solrFilter.Length = solrFilter.Length - 4; List<UserSolrEntity> userSolrEntityList = SolrHelper.GetInstance().QueryByFilter<UserSolrEntity>(solrFilter.ToString()); List<UserSolrEntity> userDBConvertSolrEntityList = userDBEntityList.Select((userDB, userSolr) => { return new UserSolrEntity() { UserNo = userDB.UserID.Trim(), Age = userDB.Age, Name = userDB.UserName.Trim(), IsMarried = userDB.Married == "1" }; }).ToList(); lock (_lockObj) { differenceUserEntityList.AddRange(userDBConvertSolrEntityList.Except(userSolrEntityList, new UserSolrEntityCompare())); } } }
咱們拿到對比的DB數據實體List以後,獲得userID,而後拼成solr的查詢條件solrFilter,而後調用SolrHelper中的QueryByFilter<T>方法去查詢出一個Solr的實體List,而後咱們將DB的實體List經過Linq轉化爲Solr的實體List userDBConvertSolrEntityList。而後經過Except方法找出不一樣的實體List,放置到differenceUserEntityList。在這裏注意IEqualityCompare接口的實現。
public class UserSolrEntityCompare : IEqualityComparer<UserSolrEntity> { public bool Equals(UserSolrEntity original, UserSolrEntity destination) { original.Name = original.Name ?? string.Empty; original.UserNo = original.UserNo ?? string.Empty; destination.Name = destination.Name ?? string.Empty; destination.UserNo = destination.UserNo ?? string.Empty; return original.UserNo.Trim().Equals(destination.UserNo.Trim()) && original.Age == destination.Age && original.Name.Trim().Equals(destination.Name.Trim()) && original.IsMarried == destination.IsMarried; } public int GetHashCode(UserSolrEntity userSolrEntity) { return userSolrEntity.UserNo.GetHashCode(); } }
OK,到這裏就所有結束了,咱們在action中拿到了比較出的結果,而後組成viewModel,返回給Partial頁面去綁定。看一下效果Comparing,please wait......
下面是比較出的結果,由於solr中沒有數據,因此都是不相同的,由於取DB數據是INNER JOIN,因此只有四條數據。
OK,本節到此結束,下節咱們看一下Fix功能的實現。