Elasticsearch .Net Client NEST使用說明 2.x elasticsearch_.net_client_nest2.x_到_5.x經常使用方法屬性差別 Elasticsearch

Elasticsearch .net client NEST使用說明 2.x

    

Elasticsearch.Net與NEST是Elasticsearch爲C#提供的一套客戶端驅動,方便C#調用Elasticsearch服務接口。Elasticsearch.Net是較基層的對Elasticsearch服務接口請求響應的實現,NEST是在前者基礎之上進行的封裝。本文是針對NEST的使用的總結。html

目錄:

       Elasticsearch.Net、NEST 交流羣:523061899node

  demo源碼 https://github.com/huhangfei/NestDemosgit

引用

Install-Package NEST

包含如下dllgithub

NEST.dll
Elasticsearch.Net.dll  
Newtonsoft.Json.dll

概念

存儲結構:

Elasticsearch中,文檔(Document)歸屬於一種類型(type),而這些類型存在於索引(index).算法

類比傳統關係型數據庫:數據庫

Relational DB -> Databases -> Tables -> Rows -> Columns
Elasticsearch -> Indices   -> Types  -> Documents -> Fields

距離單位:

  •   mm (毫米)api

  •   cm (釐米)服務器

  •   m (米)app

  •   km (公里)elasticsearch

  • in (英寸)

  •   ft (英尺)

  •   yd (碼)

  •   mi (英里)

  •   nmi or NM (海里)

日期單位:

  • y (year)
  •  M (month)
  • w (week)
  • d (day)
  •  h (hour)
  • m (minute)
  • s (second)
  • ms (milliseconds)

客戶端語法

鏈式lambda 表達式( powerful query DSL

s => s
.Query(q => q
    .Term(p => p.Name, "elasticsearch")
)

 

對象初始化語法

         var searchRequest = new SearchRequest<VendorPriceInfo>
            {
                Query = new TermQuery
                {
                    Field = "name",
                    Value = "elasticsearch"
                }
            };

 

Connection連接

//單node
Var node = new Uri(「……」);
var settings = new ConnectionSettings(node);
 
//uri
Var uris = new Uri [] {
       new Uri(「……」),
         new Uri(「……」)
};
//多node
Var nodes = new Node [] {
       new Node (new Uri(「……」)),
         new Node (new Uri(「……」))
};

var pool = new StaticConnectionPool(nodes);
var pool = new StaticConnectionPool(uris);
var settings = new ConnectionSettings(pool);
var client = new ElasticClient(settings);

鏈接池類型:

//對單節點請求
IConnectionPool pool = new SingleNodeConnectionPool(urls.FirstOrDefault());

//請求時隨機請求各個正常節點,不請求異常節點,異常節點恢復後會從新被請求
IConnectionPool pool = new StaticConnectionPool(urls); 

IConnectionPool pool = new SniffingConnectionPool(urls);
//false.建立客戶端時,隨機選擇一個節點做爲客戶端的請求對象,該節點異常後不會切換其它節點
//true,請求時隨機請求各個正常節點,不請求異常節點,但異常節點恢復後不會從新被請求
pool.SniffedOnStartup = true;

//建立客戶端時,選擇第一個節點做爲請求主節點,該節點異常後會切換其它節點,待主節點恢復後會自動切換回來
IConnectionPool pool = new StickyConnectionPool(urls);

索引選擇

方式1:

var settings = new ConnectionSettings().DefaultIndex("defaultindex");

方式2:

var settings = new ConnectionSettings().MapDefaultTypeIndices(m => m.Add(typeof(Project), "projects")  );

方式3:

client.Search<VendorPriceInfo>(s => s.Index("test-index"));

client.Index(data,o=>o.Index("test-index"));

優先級:方式3  > 方式2  > 方式1

索引

索引惟一Id:

1)  默認以「Id」字段值做爲索引惟一Id值,無「Id」屬性,Es自動生成惟一Id值,添加數據時統一類型數據惟一ID已存在相等值,將只作更新處理。

注:自動生成的ID22個字符長,URL-safe, Base64-encoded string universally unique identifiers, 或者叫UUIDs

2)  標記惟一Id值

[ElasticsearchType(IdProperty = "priceID")]
    public class VendorPriceInfo
    {
        public Int64 priceID { get; set; }
        public int oldID { get; set; }
        public int source { get; set; }
}

3)  索引時指定

client.Index(data, o => o.Id(data.vendorName));

優先級: 3) > 2) > 1)

索引類型:

1)  默認類型爲索引數據的類名(自動轉換爲全小寫)

2)  標記類型

   [ElasticsearchType(Name = "datatype")]
    public class VendorPriceInfo
    {
        public Int64 priceID { get; set; }
        public int oldID { get; set; }
        public int source { get; set; }
    }

3)  索引時指定

client.Index(data, o => o.Type(new TypeName() { Name = "datatype", Type = typeof(VendorPriceInfo) }));
或
client.Index(data, o => o.Type<MyClass>());//使用 2)標記的類型

優先級:3> 2) > 1)

 

建立:

client.CreateIndex("test2");
//基本配置
IIndexState indexState=new IndexState()
{
       Settings = new IndexSettings()
       {
            NumberOfReplicas = 1,//副本數
            NumberOfShards = 5//分片數
       }
};
client.CreateIndex("test2", p => p.InitializeUsing(indexState));

//建立並Mapping
client.CreateIndex("test-index3", p => p.InitializeUsing(indexState).Mappings(m => m.Map<VendorPriceInfo>(mp => mp.AutoMap())));

注:索引名稱必須小寫

判斷:

client.IndexExists("test2");

刪除:

client.DeleteIndex("test2");

Open/Close:

client.OpenIndex("index");

client.CloseIndex("index");

 

映射

概念:

 每一個類型擁有本身的映射(mapping)或者模式定義(schema definition)。一個映射定義了字段類型,每一個字段的數據類型,以及字段被Elasticsearch處理的方式。映射還用於設置關聯到類型上的元數據。

獲取映射

 var resule = client.GetMapping<VendorPriceInfo>();

特性

    /// <summary>
    /// VendorPrice 實體
    /// </summary>
    [ElasticsearchType(IdProperty = "priceID", Name = "VendorPriceInfo")]
    public class VendorPriceInfo
    {
        [Number(NumberType.Long)]
        public Int64 priceID { get; set; }
        [Date(Format = "mmddyyyy")]
        public DateTime modifyTime { get; set; }
        /// <summary>
        /// 若是string 類型的字段不須要被分析器拆分,要做爲一個正體進行查詢,需標記此聲明,不然索引的值將被分析器拆分
        /// </summary>
        [String(Index = FieldIndexOption.NotAnalyzed)]
        public string pvc_Name { get; set; }
        /// <summary>
        /// 設置索引時字段的名稱
        /// </summary>
        [String(Name = "PvcDesc")]
        public string pvc_Desc { get; set; }
        /// <summary>
        /// 如需使用座標點類型需添加座標點特性,在maping時會自動映射類型
        /// </summary>
        [GeoPoint(Name = "ZuoBiao",LatLon = true)]
        public GeoLocation Location { get; set; }
    }

 

映射

            //根據對象類型自動映射
            var result= client.Map<VendorPriceInfo>(m => m.AutoMap());
            //手動指定
            var result1 = client.Map<VendorPriceInfo>(m => m.Properties(p => p
            .GeoPoint(gp => gp.Name(n => n.Location)// 座標點類型
                .Fielddata(fd => fd
                    .Format(GeoPointFielddataFormat.Compressed)//格式 array doc_values compressed disabled
                    .Precision(new Distance(2, DistanceUnit.Meters)) //精確度
                    ))
            .String(s => s.Name(n => n.b_id))//string 類型
            ));
            //在原有字段下新增字段(用於存儲不一樣格式的數據,查詢方法查看SearchBaseDemo)
            //eg:在 vendorName 下添加無需分析器分析的值  temp
            var result2 = client.Map<VendorPriceInfo>(
                m => m
                    .Properties(p => p.String(s => s.Name(n => n.vendorName).Fields(fd => fd.String(ss => ss.Name("temp").Index(FieldIndexOption.NotAnalyzed))))));

 

注:映射時已存在的字段將沒法從新映射,只有新加的字段能映射成功。

注:映射時同一索引中,多個類型中若是有相同字段名,那麼在索引時可能會出現問題(會使用第一個映射類型)。

數據

數據操做

提示:

  • 添加數據時,若是文檔的惟一id在索引裏已存在,那麼會替換掉原數據;
  • 添加數據時,若是索引不存在,服務會自動建立索引;
  • 若是服務自動建立索引,並索引了數據,那麼索引的映射關係就是服務器自動設置的;
  • 一般正確的使用方法是在緊接着建立索引操做以後進行映射關係的操做,以保證索引數據的映射是正確的。而後纔是索引數據;
  • 文檔在Elasticsearch中是不可變的,執行Update事實上Elasticsearch的處理過程以下:
    1.       從舊文檔中檢索JSON
    2.       修改它
    3.       刪除舊文檔
    4.       索引新文檔

因此咱們也可使用Index來更新已存在文檔,只需對應文檔的惟一id

添加索引數據:

添加單條數據
var data = new VendorPriceInfo() { vendorName = "測試"};
client.Index(data);

 

添加多條數據
var datas = new List<VendorPriceInfo> {
                new VendorPriceInfo(){priceID = 1,vendorName = "test1"},
                new VendorPriceInfo(){priceID = 2,vendorName = "test2"}};
client.IndexMany(datas);

 

 

刪除數據:

單條數據
DocumentPath<VendorPriceInfo> deletePath=new DocumentPath<VendorPriceInfo>(7);
client.Delete(deletePath);

或
IDeleteRequest request = new DeleteRequest("test3", "vendorpriceinfo", 0);
client.Delete(request);
注:刪除時根據惟一id刪除
集合數據
            Indices indices = "test-1";
            Types types = "vendorpriceinfo";
            //批量刪除 須要es安裝 delete-by-query插件
            var result = client.DeleteByQuery<VendorPriceInfo>(indices, types,
                dq =>
                    dq.Query(
                        q =>
                            q.TermRange(tr => tr.Field(fd => fd.priceID).GreaterThanOrEquals("5").LessThanOrEquals("10")))
                            );

更新數據:

更新全部字段
DocumentPath<VendorPriceInfo> deletePath=new DocumentPath<VendorPriceInfo>(2);
Var response=client.Update(deletePath,(p)=>p.Doc(new VendorPriceInfo(){vendorName = "test2update..."}));
//或
IUpdateRequest<VendorPriceInfo, VendorPriceInfo> request = new UpdateRequest<VendorPriceInfo, VendorPriceInfo>(deletePath)
            {
                Doc = new VendorPriceInfo()
                {
                    priceID = 888,
                    vendorName = "test4update........"
                }
            };
            var response = client.Update<VendorPriceInfo, VendorPriceInfo>(request);
更新部分字段
IUpdateRequest<VendorPriceInfo, VendorPriceInfoP> request = new UpdateRequest<VendorPriceInfo, VendorPriceInfoP>(deletePath)
            {
                Doc = new VendorPriceInfoP()
                {
                    priceID = 888,
                    vendorName = "test4update........"
                }

            };
            var response = client.Update(request); 
更新部分字段
IUpdateRequest<VendorPriceInfo, object> request = new UpdateRequest<VendorPriceInfo, object>(deletePath)
            {
                Doc = new
                {
                    priceID = 888,
                    vendorName = " test4update........"
                }
            };
            var response = client.Update(request);

//
client.Update<VendorPriceInfo, object>(deletePath, upt => upt.Doc(new { vendorName = "ptptptptp" }));

 

 

注:更新時根據惟一id更新

獲取數據:

var response = client.Get(new DocumentPath<VendorPriceInfo>(0));
//
var response =
 client.Get(new DocumentPath<VendorPriceInfo>(0),pd=>pd.Index("test4").Type("v2")); 

//多個
var response = client.MultiGet(m => m.GetMany<VendorPriceInfo>(new List<long> { 1, 2, 3, 4 })); 

注:獲取時根據惟一id獲取

 

搜索:

說明

基本搜索

var result = client.Search<VendorPriceInfo>(
                s => s
                    .Explain() //參數能夠提供查詢的更多詳情。
                    .FielddataFields(fs => fs //對指定字段進行分析
                        .Field(p => p.vendorFullName)
                        .Field(p => p.cbName)
                    )
                    .From(0) //跳過的數據個數
                    .Size(50) //返回數據個數
                    .Query(q =>
                        q.Term(p => p.vendorID, 100) // 主要用於精確匹配哪些值,好比數字,日期,布爾值或 not_analyzed的字符串(未經分析的文本數據類型):
                        &&
                        q.Term(p => p.vendorName.Suffix("temp"), "姓名") //用於自定義屬性的查詢 (定義方法查看MappingDemo)
                        &&
                        q.Bool( //bool 查詢
                            b => b
                                .Must(mt => mt //全部分句必須所有匹配,與 AND 相同
                                    .TermRange(p => p.Field(f => f.priceID).GreaterThan("0").LessThan("1"))) //指定範圍查找
                                .Should(sd => sd //至少有一個分句匹配,與 OR 相同
                                    .Term(p => p.priceID, 32915),
                                    sd => sd.Terms(t => t.Field(fd => fd.priceID).Terms(new[] {10, 20, 30})),//多值
                                    //||
                                    //sd.Term(p => p.priceID, 1001)
                                    //||
                                    //sd.Term(p => p.priceID, 1005)
                                    sd => sd.TermRange(tr => tr.GreaterThan("10").LessThan("12").Field(f => f.vendorPrice))
                                )
                                .MustNot(mn => mn//全部分句都必須不匹配,與 NOT 相同
                                    .Term(p => p.priceID, 1001)
                                    ,
                                    mn => mn.Bool(//bool 過濾 ,bool 查詢與 bool 過濾類似,用於合併多個查詢子句。不一樣的是,bool 過濾能夠直接給出是否匹配成功, 而bool 查詢要計算每個查詢子句的 _score (相關性分值)。
                                        bb=>bb.Must(mt=>mt
                                            .Match(mc=>mc.Field(fd=>fd.carName).Query("至尊"))
                                        ))
                                )
                            )

                    )//查詢條件
                .Sort(st => st.Ascending(asc => asc.vendorPrice))//排序
                .Source(sc => sc.Include(ic => ic
                    .Fields(
                        fd => fd.vendorName,
                        fd => fd.vendorID,
                        fd => fd.priceID,
                        fd => fd.vendorPrice))) //返回特定的字段
               );
            //TResult
            var result1 = client.Search<VendorPriceInfo, VendorPriceInfoP>(s => s.Query(
                q => q.MatchAll()
                )
                .Size(15)
                );

 

var result = client.Search<VendorPriceInfo>(new SearchRequest()
        {
            Sort =new List<ISort>
                    {
                        new SortField { Field = "vendorPrice", Order = SortOrder.Ascending }
                    },
            Size = 10,
            From = 0,
            Query = new TermQuery()
            {
                Field = "priceID",
                Value = 6
            }
            ||
            new TermQuery(
            {
                Field = "priceID",
                Value = 8
            }
        });

分頁

            //分頁最大限制(from+size<=10000)
int pageSize = 10; int pageIndex = 1; var result = client.Search<VendorPriceInfo>(s => s.Query(q => q .MatchAll()) .Size(pageSize) .From((pageIndex - 1) * pageSize) .Sort(st => st.Descending(d => d.priceID)));

 

掃描和滾屏

            string scrollid = "";
            var result = client.Search<VendorPriceInfo>(s => s.Query(q => q.MatchAll())
                .Size(100)
                .SearchType(SearchType.Scan)
                .Scroll("1m"));//scrollid過時時間
            //獲得滾動掃描的id
            scrollid = result.ScrollId;
            
            //執行滾動掃描獲得數據 返回數據量是 result.Shards.Successful*size(查詢成功的分片數*size)
            result = client.Scroll<VendorPriceInfo>("1m", scrollid);
            //獲得新的id
            scrollid = result.ScrollId;

查詢條件設置加權

// 在原分值基礎上 設置不一樣匹配的加成值 具體算法爲lucene內部算法
var
result = client.Search<VendorPriceInfo>(s => s .Query(q => q.Term(t => t .Field(f => f.cityID).Value(2108).Boost(4)) || q.Term(t => t .Field(f => f.pvcId).Value(2103).Boost(1)) ) .Size(3000) .Sort(st => st.Descending(SortSpecialField.Score)) );

 

得分控制

//使用functionscore計算得分
var
result1 = client.Search<VendorPriceInfo>(s => s .Query(q=>q.FunctionScore(f=>f
              //查詢區 .Query(qq
=> qq.Term(t => t .Field(fd => fd.cityID).Value(2108)) || qq.Term(t => t .Field(fd => fd.pvcId).Value(2103)) ) .Boost(1.0) //functionscore 對分值影響 .BoostMode(FunctionBoostMode.Replace)//計算boost 模式 ;Replace爲替換 .ScoreMode(FunctionScoreMode.Sum) //計算score 模式;Sum爲累加
              //邏輯區 .Functions(fun
=>fun .Weight(w => w.Weight(2).Filter(ft => ft .Term(t => t .Field(fd => fd.cityID).Value(2108))))//匹配cityid +2 .Weight(w => w.Weight(1).Filter(ft => ft .Term(t => t .Field(fd => fd.pvcId).Value(2103))))//匹配pvcid +1 ) ) ) .Size(3000) .Sort(st => st.Descending(SortSpecialField.Score).Descending(dsc=>dsc.priceID)) );
//結果中 cityid=2108,得分=2; pvcid=2103 ,得分=1 ,二者都知足的,得分=3
 
 


 

查詢字符串-簡單的檢索

            var result = client.Search<VendorPriceInfo>(s => s
                .Query(q => q.QueryString(m => m.Fields(fd=>fd.Field(fdd=>fdd.carName).Field(fdd=>fdd.carGearBox))
                                .Query("手自一體")
                                )
                    )
                .From(0)
                .Size(15)
                );

 

全文檢索-關鍵詞搜索

 var result=client.Search<VendorPriceInfo>(s=>s
                .Query(q=>q
                    .Match(m=>m.Field(f=>f.carName)
                                .Query("尊貴型")
                                )
                )
                .From(0)
                .Size(15)
                );
            //多字段匹配
            var result1 = client.Search<VendorPriceInfo>(s => s
                .Query(q => q
                    .MultiMatch(m => m.Fields(fd=>fd.Fields(f=>f.carName,f=>f.carGearBox))
                                .Query("尊貴型")
                                )
                )
                .From(0)
                .Size(15)
                );

 

全文搜索-短語搜索

var result = client.Search<VendorPriceInfo>(s => s
                .Query(q => q.MatchPhrase(m => m.Field(f => f.carName)
                                .Query("尊貴型")
                                )
                )
                .From(0)
                .Size(15)
                );

 

 

座標點搜索-根據座標點及距離搜索

            const double lat = 39.8694890000;
            const double lon = 116.4206470000;
            const double distance = 2000.0;          

            //1
            var result = client.Search<VendorPriceInfo>(s => s
                .Query(q => q
                .Bool(b => b.Must(m => m
                    .GeoDistance(gd => gd
                        .Location(lat, lon)
                        .Distance(distance, DistanceUnit.Meters)
                        .Field(fd => fd.Location)
                        ))
                    )
                )
                .From(0)
                .Size(15)
                );
 
            //2
            var location = new GeoLocation(lat, lon);
            var distancei = new Distance(distance, DistanceUnit.Meters);
            var result1 = client.Search<VendorPriceInfo>(s => s
                .Query(q => q
                        .Bool(b => b.Must(m => m
                            .Exists(e => e.Field(fd => fd.Location))
                            )
                        )
                        &&
                        q.GeoDistance(gd => gd
                        .Location(location)
                        .Distance(distancei)
                        .Field(fd => fd.Location)
                        )
                )
                .From(0)
                .Size(15)
                );

            //3
            var result2 = client.Search<VendorPriceInfo>(s => s
                .Query(q => q
                    .Bool(b=>b
                        .Must(m=>m.MatchAll())
                        .Filter(f=>f
                            .GeoDistance(g => g
                                .Name("named_query")
                                .Field(p => p.Location)
                                .DistanceType(GeoDistanceType.Arc)
                                .Location(lat,lon)
                                .Distance("2000.0m")
                                )
                            )
                        )
                    )
                .From(0)
                .Size(15)
                );

 

 

聚合

聚合-基本

var result = client.Search<VendorPriceInfo>(s => s
                .From(0)
                .Size(15)
                .Aggregations(ag=>ag
                        .ValueCount("Count", vc => vc.Field(fd => fd.vendorPrice))//總數
                        .Sum("vendorPrice_Sum", su => su.Field(fd => fd.vendorPrice))//求和
                        .Max("vendorPrice_Max", m => m.Field(fd => fd.vendorPrice))//最大值
                        .Min("vendorPrice_Min", m => m.Field(fd => fd.vendorPrice))//最小值
                        .Average("vendorPrice_Avg", avg => avg.Field(fd => fd.vendorPrice))//平均值
                        .Terms("vendorID_group", t => t.Field(fd => fd.vendorID).Size(100))//分組
                    )
                );

 

聚合-分組

           //每一個經銷商 的平均報價
            var result = client.Search<VendorPriceInfo>(s => s
                .Size(0)
                .Aggregations(ag => ag
                        .Terms("vendorID_group", t => t
                            .Field(fd => fd.vendorID)
                            .Size(100)
                            .Aggregations(agg => agg.Average("vendorID_Price_Avg", av => av.Field(fd => fd.vendorPrice)))
                        )//分組
                        .Cardinality("vendorID_group_count", dy => dy.Field(fd => fd.vendorID))//分組數量
                        .ValueCount("Count", c => c.Field(fd => fd.vendorID))//總記錄數
                    )
                );

 

聚合-分組-聚合..

            //每一個經銷商下 每一個品牌 的平均報價
            var result = client.Search<VendorPriceInfo>(s => s
                .Size(0)
                .Aggregations(ag => ag
                    .Terms("vendorID_group", //vendorID 分組
                        t => t.Field(fd => fd.vendorID)
                        .Size(100)
                        .Aggregations(agg => agg
                                        .Terms("vendorID_cbID_group", //cbID分組
                                            tt => tt.Field(fd => fd.cbID)
                                            .Size(50)
                                            .Aggregations(aggg => aggg
                                                .Average("vendorID_cbID_Price_Avg", av => av.Field(fd => fd.vendorPrice))//Price avg
                                                .Max("vendorID_cbID_Price_Max", m => m.Field(fd => fd.vendorPrice))//Price max
                                                .Min("vendorID_cbID_Price_Min", m => m.Field(fd => fd.vendorPrice))//Price min
                                                .ValueCount("vendorID_cbID_Count", m => m.Field(fd => fd.cbID))//該經銷商對該品牌 報價數 count
                                                )
                                            )
                                        .Cardinality("vendorID_cbID_group_count", dy => dy.Field(fd => fd.cbID))//分組數量
                                        .ValueCount("vendorID_Count", c => c.Field(fd => fd.vendorID))//該經銷商的報價數
                            )
                        )
                        .Cardinality("vendorID_group_count",dy=>dy.Field(fd=>fd.vendorID))//分組數量
                        .ValueCount("Count",c=>c.Field(fd=>fd.priceID))//總記錄數
                ) //分組
                );

 

官網文檔:

https://www.elastic.co/guide/en/elasticsearch/client/net-api/2.x/introduction.html

 

 

Elasticsearch.Net、NEST 交流羣:523061899

相關文章
相關標籤/搜索