針對 ElasticSearch .Net 客戶端的一些封裝

時間 2019-11-22

原文原文鏈接

ElasticSearch是一個基於Lucene的搜索服務器。它提供了一個分佈式多用戶能力的全文搜索引擎，基於RESTful web接口。Elasticsearch是用Java開發的，並做爲Apache許可條款下的開放源碼發佈，是當前流行的企業級搜索引擎。設計用於雲計算中，可以達到實時搜索，穩定，可靠，快速，安裝使用方便。git

　　ElasticSearch 爲.net提供了兩個客戶端，分別是 Elasticsearch.Net 和 NEST　github

　 Elasticsearch.net爲何會有兩個客戶端？

　　Elasticsearch.Net是一個很是底層且靈活的客戶端，它不在乎你如何的構建本身的請求和響應。它很是抽象，所以全部的Elasticsearch API被表示爲方法，沒有太多關於你想如何構建json/request/response對象的東東，而且它還內置了可配置、可重寫的集羣故障轉移機制。web

　　Elasticsearch.Net有很是大的彈性，若是你想更好的提高你的搜索服務，你徹底可使用它來作爲你的客戶端。sql

　　NEST是一個高層的客戶端，能夠映射全部請求和響應對象，擁有一個強類型查詢DSL（領域特定語言），而且可使用.net的特性好比協變、Auto Mapping Of POCOs，NEST內部使用的依然是Elasticsearch.Net客戶端。express

　　具體客戶端的用法可參考官方的文檔說明，本文主要針對 NEST 的查詢作擴展。json

　　原由：以前在學習Dapper的時候看過一個 DapperExtensions 的封裝其實Es的查詢基本就是相似Sql的查詢。所以參考DapperExtensions 進行了Es版本的遷移api

　　經過官網說明能夠看到 NEST 的對象初始化的方式進行查詢都是已下面的方式開頭：服務器

var searchRequest = new SearchRequest<XXT>(XXIndex)

　　咱們能夠經過查看源碼app

　　咱們能夠看到全部的查詢基本都是在SearchRequest上面作的擴展這樣咱們也能夠開始咱們的第一步操做：elasticsearch

　　1.關於分頁，咱們定義以下分頁對象：　　

 1 /// <summary>
 2     /// 分頁類型
 3     /// </summary>
 4     public class PageEntity
 5     {
 6         /// <summary>
 7         ///     每頁行數
 8         /// </summary>
 9         public int PageSize { get; set; }
10 
11         /// <summary>
12         ///     當前頁
13         /// </summary>
14         public int PageIndex { get; set; }
15 
16         /// <summary>
17         ///     總記錄數
18         /// </summary>
19         public int Records { get; set; }
20 
21         /// <summary>
22         ///     總頁數
23         /// </summary>
24         public int Total
25         {
26             get
27             {
28                 if (Records > 0)
29                     return Records % PageSize == 0 ? Records / PageSize : Records / PageSize + 1;
30 
31                 return 0;
32             }
33         }
34 
35 
36         /// <summary>
37         ///     排序列
38         /// </summary>
39         public string Sidx { get; set; }
40 
41         /// <summary>
42         ///     排序類型
43         /// </summary>
44         public string Sord { get; set; }
45     }

View Code

  2.定義ElasticsearchPage 分頁對象

/// <summary>
    ///     ElasticsearchPage
    /// </summary>
    public class ElasticsearchPage<T> : PageEntity
    {
        public string Index { get; set; }
        
        public ElasticsearchPage(string index)
        {
            Index = index;
        }

        /// <summary>
        /// InitSearchRequest
        /// </summary>
        /// <returns></returns>
        public SearchRequest<T> InitSearchRequest()
        {
            return new SearchRequest<T>(Index)
            {
                From = (PageIndex - 1) * PageSize,
                Size = PageSize
            };
        }
    }

View Code

  至此咱們的SearchRequest的初始化操做已經完成了咱們能夠經過以下方式進行調用

1  var elasticsearchPage = new ElasticsearchPage<Content>("content")
2             {
3                 PageIndex = pageIndex,
4                 PageSize = pageSize
5             };
6 
7 var searchRequest = elasticsearchPage.InitSearchRequest();

View Code

經過SearchRequest的源碼咱們能夠得知，全部的查詢都是基於內部屬性進行(擴展的思路來自DapperExtensions)：

　 3.QueryContainer的擴展，相似Where 語句：

　　咱們定義一個比較操做符相似 Sql中的 like != in 等等

 1 /// <summary>
 2     ///     比較操做符
 3     /// </summary>
 4     public enum ExpressOperator
 5     {
 6         /// <summary>
 7         ///     精準匹配 term（主要用於精確匹配哪些值，好比數字，日期，布爾值或 not_analyzed 的字符串(未經分析的文本數據類型)： ）
 8         /// </summary>
 9         Eq,
10 
11         /// <summary>
12         ///     大於
13         /// </summary>
14         Gt,
15 
16         /// <summary>
17         ///     大於等於
18         /// </summary>
19         Ge,
20 
21         /// <summary>
22         ///     小於
23         /// </summary>
24         Lt,
25 
26         /// <summary>
27         ///     小於等於
28         /// </summary>
29         Le,
30 
31         /// <summary>
32         ///     模糊查詢 (You can use % in the value to do wilcard searching)
33         /// </summary>
34         Like,
35 
36         /// <summary>
37         /// in 查詢
38         /// </summary>
39         In
40     }

View Code

　　接着咱們定義一個以下接口，主要包括：

　　1. 提供返回一個 QueryContainer GetQuery方法

　　2. 屬性名稱 PropertyName

　　3. 操做符 ExpressOperator

　　4. 謂詞值 Value

 1  /// <summary>
 2     ///     謂詞接口
 3     /// </summary>
 4     public interface IPredicate
 5     {
 6         QueryContainer GetQuery(QueryContainer query);
 7     }
 8 
 9     /// <summary>
10     ///     基礎謂詞接口
11     /// </summary>
12     public interface IBasePredicate : IPredicate
13     {
14         /// <summary>
15         ///     屬性名稱
16         /// </summary>
17         string PropertyName { get; set; }
18     }
19 
20     public abstract class BasePredicate : IBasePredicate
21     {
22         public string PropertyName { get; set; }
23         public abstract QueryContainer GetQuery(QueryContainer query);
24     }
25 
26     /// <summary>
27     ///     比較謂詞
28     /// </summary>
29     public interface IComparePredicate : IBasePredicate
30     {
31         /// <summary>
32         ///     操做符
33         /// </summary>
34         ExpressOperator ExpressOperator { get; set; }
35     }
36 
37     public abstract class ComparePredicate : BasePredicate
38     {
39         public ExpressOperator ExpressOperator { get; set; }
40     }
41 
42     /// <summary>
43     ///     字段謂詞
44     /// </summary>
45     public interface IFieldPredicate : IComparePredicate
46     {
47         /// <summary>
48         ///     謂詞的值
49         /// </summary>
50         object Value { get; set; }
51     }

View Code

　　具體實現定義 FieldPredicate 而且繼承如上接口，經過操做符映射爲 Nest具體查詢對象

 1 public class FieldPredicate<T> : ComparePredicate, IFieldPredicate
 2         where T : class
 3     {
 4         public object Value { get; set; }
 5 
 6         public override QueryContainer GetQuery(QueryContainer query)
 7         {
 8             switch (ExpressOperator)
 9             {
10                 case ExpressOperator.Eq:
11                     query = new TermQuery
12                     {
13                         Field = PropertyName,
14                         Value = Value
15                     };
16                     break;
17                 case ExpressOperator.Gt:
18                     query = new TermRangeQuery
19                     {
20                         Field = PropertyName,
21                         GreaterThan = Value.ToString()
22                     };
23                     break;
24                 case ExpressOperator.Ge:
25                     query = new TermRangeQuery
26                     {
27                         Field = PropertyName,
28                         GreaterThanOrEqualTo = Value.ToString()
29                     };
30                     break;
31                 case ExpressOperator.Lt:
32                     query = new TermRangeQuery
33                     {
34                         Field = PropertyName,
35                         LessThan = Value.ToString()
36                     };
37                     break;
38                 case ExpressOperator.Le:
39                     query = new TermRangeQuery
40                     {
41                         Field = PropertyName,
42                         LessThanOrEqualTo = Value.ToString()
43                     };
44                     break;
45                 case ExpressOperator.Like:
46                     query = new MatchPhraseQuery
47                     {
48                         Field = PropertyName,
49                         Query = Value.ToString()
50                     };
51                     break;
52                 case ExpressOperator.In:
53                     query = new TermsQuery
54                     {
55                         Field = PropertyName,
56                         Terms=(List<object>)Value
57                     };
58                     break;
59                 default:
60                     throw new ElasticsearchException("構建Elasticsearch查詢謂詞異常");
61             }
62             return query;
63         }
64     }

View Code

　　4.定義好這些後咱們就能夠拼接咱們的條件了，咱們定義了 PropertyName 可是咱們更傾向於一種相似EF的查詢方式能夠經過 Expression<Func<T, object>> 的方式因此咱們這邊提供一個泛型方式

，由於在建立 Elasticsearch 文檔的時候咱們已經創建了Map 文件咱們經過反射讀取 PropertySearchName屬性就能夠讀取到咱們的 PropertyName 這邊 PropertySearchName 是本身定義的屬性

爲何不反解Nest 的屬性針對不一樣類型須要反解的屬性也是不相同的因此避免麻煩直接從新定義了新的屬性。代碼以下：

1 public class PropertySearchNameAttribute: Attribute
2     {
3         public PropertySearchNameAttribute(string name)
4         {
5             Name = name;
6         }
7         public string Name { get; set; }
8     }

View Code

而後咱們就能夠來定義的們初始化IFieldPredicate 的方法了

　首先咱們解析咱們的需求：

　　1.咱們須要一個Expression<Func<T, object>>

　　2.咱們須要一個操做符

　　3.咱們須要比較什麼值

針對需求咱們能夠獲得這樣一個方法：

注：所依賴的反射方法詳解文末

 1 /// <summary>
 2         ///     工廠方法建立一個新的  IFieldPredicate 謂語: [FieldName] [Operator] [Value].
 3         /// </summary>
 4         /// <typeparam name="T">實例類型</typeparam>
 5         /// <param name="expression">返回左操做數的表達式  [FieldName].</param>
 6         /// <param name="op">比較運算符</param>
 7         /// <param name="value">謂語的值.</param>
 8         /// <returns>An instance of IFieldPredicate.</returns>
 9         public static IFieldPredicate Field<T>(Expression<Func<T, object>> expression, ExpressOperator op, object value) where T : class
10         {
11             var propertySearchName = (PropertySearchNameAttribute)
12                 LoadAttributeHelper.LoadAttributeByType<T, PropertySearchNameAttribute>(expression);
13 
14             return new FieldPredicate<T>
15             {
16                 PropertyName = propertySearchName.Name,
17                 ExpressOperator = op,
18                 Value = value
19             };
20         }

View Code

而後咱們就能夠像以前拼接sql的方式來進行拼接條件了

就以咱們項目中的業務需求作個演示

1  var predicateList = new List<IPredicate>();
2 //最大價格
3 if (requestContentDto.MaxPrice != null)
4                 predicateList.Add(Predicates.Field<Content>(x => x.UnitPrice, ExpressOperator.Le,
5                     requestContentDto.MaxPrice));
6 //最小价格
7 if (requestContentDto.MinPrice != null)
8                 predicateList.Add(Predicates.Field<Content>(x => x.UnitPrice, ExpressOperator.Ge,
9                     requestContentDto.MinPrice));

View Code

而後針對實際業務咱們在寫sql的時候就回有 (xx1 and xx2) or xx3 這樣的業務需求了

針對這種業務需求咱們須要在提供一個 IPredicateGroup 進行分組查詢謂詞

首先咱們定義一個PredicateGroup 加入謂詞時使用的操做符 GroupOperator

1     /// <summary>
2     ///     PredicateGroup 加入謂詞時使用的操做符
3     /// </summary>
4     public enum GroupOperator
5     {
6         And,
7         Or
8     }

View Code

而後咱們定義 IPredicateGroup 及實現

 1  /// <summary>
 2     ///     分組查詢謂詞
 3     /// </summary>
 4     public interface IPredicateGroup : IPredicate
 5     {
 6         /// <summary>
 7         /// </summary>
 8         GroupOperator Operator { get; set; }
 9 
10         IList<IPredicate> Predicates { get; set; }
11     }
12 
13     /// <summary>
14     ///     分組查詢謂詞
15     /// </summary>
16     public class PredicateGroup : IPredicateGroup
17     {
18         public GroupOperator Operator { get; set; }
19         public IList<IPredicate> Predicates { get; set; }
20 
21         /// <summary>
22         ///     GetQuery
23         /// </summary>
24         /// <param name="query"></param>
25         /// <returns></returns>
26         public QueryContainer GetQuery(QueryContainer query)
27         {
28             switch (Operator)
29             {
30                 case GroupOperator.And:
31                     return Predicates.Aggregate(query, (q, p) => q && p.GetQuery(query));
32                 case GroupOperator.Or:
33                     return Predicates.Aggregate(query, (q, p) => q || p.GetQuery(query));
34                 default:
35                     throw new ElasticsearchException("構建Elasticsearch查詢謂詞異常");
36             }
37         }
38     }

View Code

如今咱們能夠用 PredicateGroup來組裝咱們的謂詞

　一樣解析咱們的需求：

　　1.咱們須要一個GroupOperator

　　2.咱們須要謂詞列表 IPredicate[]

針對需求咱們能夠獲得這樣一個方法：

 1  /// <summary>
 2         ///     工廠方法建立一個新的 IPredicateGroup 謂語.
 3         ///     謂詞組與其餘謂詞能夠鏈接在一塊兒.
 4         /// </summary>
 5         /// <param name="op">分組操做時使用的鏈接謂詞 (AND / OR).</param>
 6         /// <param name="predicate">一組謂詞列表.</param>
 7         /// <returns>An instance of IPredicateGroup.</returns>
 8         public static IPredicateGroup Group(GroupOperator op, params IPredicate[] predicate)
 9         {
10             return new PredicateGroup
11             {
12                 Operator = op,
13                 Predicates = predicate
14             };
15         }

View Code

這樣咱們就能夠進行組裝了

用法:

 1 //構建或查詢
 2 
 3 var predicateList= new List<IPredicate>();
 4 
 5 //關鍵詞
 6 if (!string.IsNullOrWhiteSpace(requestContentDto.SearchKey))
 7 predicateList.Add(Predicates.Field<Content>(x => x.Title, ExpressOperator.Like,
 8 requestContentDto.SearchKey));
 9 
10 var predicate = Predicates.Group(GroupOperator.And, predicateList.ToArray());
11 //構建或查詢
12 var predicateListOr = new List<IPredicate>();
13 if (!string.IsNullOrWhiteSpace(requestContentDto.Brand))
14 {
15 var array = requestContentDto.Brand.Split(',').ToList();
16 predicateListOr
17 .AddRange(array.Select
18 (item => Predicates.Field<Content>(x => x.Brand, ExpressOperator.Like, item)));
19 }
20 
21 var predicateOr = Predicates.Group(GroupOperator.Or, predicateListOr.ToArray());
22 
23 var predicatecCombination = new List<IPredicate> {predicate, predicateOr};
24 var pgCombination = Predicates.Group(GroupOperator.And, predicatecCombination.ToArray());

View Code

而後咱們的 IPredicateGroup 優雅的和 ISearchRequest 使用呢咱們提供一個鏈式的操做方法

 1  /// <summary>
 2         /// 初始化query
 3         /// </summary>
 4         /// <param name="searchRequest"></param>
 5         /// <param name="predicate"></param>
 6         public static ISearchRequest InitQueryContainer(this ISearchRequest searchRequest, IPredicate predicate)
 7         {
 8             if (predicate != null)
 9             {
10                 searchRequest.Query = predicate.GetQuery(searchRequest.Query);
11             }
12             return searchRequest;
13 
14         }

View Code

至此咱們的基礎查詢方法已經封裝完成

而後經過 Nest 的進行查詢便可

var response = ElasticClient.Search<T>(searchRequest);

具體演示代碼（以項目的業務）

  1  var elasticsearchPage = new ElasticsearchPage<Content>("content")
  2             {
  3                 PageIndex = pageIndex,
  4                 PageSize = pageSize
  5             };
  6 
  7             #region terms 分組
  8 
  9             var terms = new List<IFieldTerms>();
 10             var classificationGroupBy = "searchKey_classification";
 11             var brandGroupBy = "searchKey_brand";
 12 
 13             #endregion
 14 
 15             var searchRequest = elasticsearchPage.InitSearchRequest();
 16             var predicateList = new List<IPredicate>();
 17             //分類ID
 18             if (requestContentDto.CategoryId != null)
 19                 predicateList.Add(Predicates.Field<Content>(x => x.ClassificationCode, ExpressOperator.Like,
 20                     requestContentDto.CategoryId));
 21             else
 22                 terms.Add(Predicates.FieldTerms<Content>(x => x.ClassificationGroupBy, classificationGroupBy, 200));
 23 
 24             //品牌
 25             if (string.IsNullOrWhiteSpace(requestContentDto.Brand))
 26                 terms.Add(Predicates.FieldTerms<Content>(x => x.BrandGroupBy, brandGroupBy, 200));
 27             //供應商名稱
 28             if (!string.IsNullOrWhiteSpace(requestContentDto.BaseType))
 29                 predicateList.Add(Predicates.Field<Content>(x => x.BaseType, ExpressOperator.Like,
 30                     requestContentDto.BaseType));
 31             //是否自營
 32             if (requestContentDto.IsSelfSupport == 1)
 33                 predicateList.Add(Predicates.Field<Content>(x => x.IsSelfSupport, ExpressOperator.Eq,
 34                     requestContentDto.IsSelfSupport));
 35             //最大價格
 36             if (requestContentDto.MaxPrice != null)
 37                 predicateList.Add(Predicates.Field<Content>(x => x.UnitPrice, ExpressOperator.Le,
 38                     requestContentDto.MaxPrice));
 39             //最小价格
 40             if (requestContentDto.MinPrice != null)
 41                 predicateList.Add(Predicates.Field<Content>(x => x.UnitPrice, ExpressOperator.Ge,
 42                     requestContentDto.MinPrice));
 43             //關鍵詞
 44             if (!string.IsNullOrWhiteSpace(requestContentDto.SearchKey))
 45                 predicateList.Add(Predicates.Field<Content>(x => x.Title, ExpressOperator.Like,
 46                     requestContentDto.SearchKey));
 47 
 48             //規整排序
 49             var sortConfig = SortOrderRule(requestContentDto.SortKey);
 50             var sorts = new List<ISort>
 51             {
 52                 Predicates.Sort<Content>(sortConfig.Key, sortConfig.SortOrder)
 53             };
 54 
 55             var predicate = Predicates.Group(GroupOperator.And, predicateList.ToArray());
 56             //構建或查詢
 57             var predicateListOr = new List<IPredicate>();
 58             if (!string.IsNullOrWhiteSpace(requestContentDto.Brand))
 59             {
 60                 var array = requestContentDto.Brand.Split(',').ToList();
 61                 predicateListOr
 62                     .AddRange(array.Select
 63                         (item => Predicates.Field<Content>(x => x.Brand, ExpressOperator.Like, item)));
 64             }
 65 
 66             var predicateOr = Predicates.Group(GroupOperator.Or, predicateListOr.ToArray());
 67 
 68             var predicatecCombination = new List<IPredicate> {predicate, predicateOr};
 69             var pgCombination = Predicates.Group(GroupOperator.And, predicatecCombination.ToArray());
 70 
 71             searchRequest.InitQueryContainer(pgCombination)
 72                 .InitSort(sorts)
 73                 .InitHighlight(requestContentDto.HighlightConfigEntity)
 74                 .InitGroupBy(terms);
 75 
 76             var data = _searchProvider.SearchPage(searchRequest);
 77 
 78             #region terms 分組賦值
 79 
 80             var classificationResponses = requestContentDto.CategoryId != null
 81                 ? null
 82                 : data.Aggregations.Terms(classificationGroupBy).Buckets
 83                     .Select(x => new ClassificationResponse
 84                     {
 85                         Key = x.Key.ToString(),
 86                         DocCount = x.DocCount
 87                     }).ToList();
 88 
 89             var brandResponses = !string.IsNullOrWhiteSpace(requestContentDto.Brand)
 90                 ? null
 91                 : data.Aggregations.Terms(brandGroupBy).Buckets
 92                     .Select(x => new BrandResponse
 93                     {
 94                         Key = x.Key.ToString(),
 95                         DocCount = x.DocCount
 96                     }).ToList();
 97 
 98             #endregion
 99 
100             //初始化
101 
102             #region 高亮
103 
104             var titlePropertySearchName = (PropertySearchNameAttribute)
105                 LoadAttributeHelper.LoadAttributeByType<Content, PropertySearchNameAttribute>(x => x.Title);
106 
107             var list = data.Hits.Select(c => new Content
108             {
109                 Key = c.Source.Key,
110                 Title = (string) c.Highlights.Highlight(c.Source.Title, titlePropertySearchName.Name),
111                 ImgUrl = c.Source.ImgUrl,
112                 BaseType = c.Source.BaseType,
113                 BelongMemberName = c.Source.BelongMemberName,
114                 Brand = c.Source.Brand,
115                 Code = c.Source.Code,
116                 BrandFirstLetters = c.Source.BrandFirstLetters,
117                 ClassificationName = c.Source.ClassificationName,
118                 ResourceStatus = c.Source.ResourceStatus,
119                 BrandGroupBy = c.Source.BrandGroupBy,
120                 ClassificationGroupBy = c.Source.ClassificationGroupBy,
121                 ClassificationCode = c.Source.ClassificationCode,
122                 IsSelfSupport = c.Source.IsSelfSupport,
123                 UnitPrice = c.Source.UnitPrice
124             }).ToList();
125 
126             #endregion
127 
128             var contentResponse = new ContentResponse
129             {
130                 Records = (int) data.Total,
131                 PageIndex = elasticsearchPage.PageIndex,
132                 PageSize = elasticsearchPage.PageSize,
133                 Contents = list,
134                 BrandResponses = brandResponses,
135                 ClassificationResponses = classificationResponses
136             };
137             return contentResponse;