目前對於檢索功能比較有名的服務是咱們常見的elasticsearch,因此咱們這一節的重點,也是針對elasticsearch的使用。html
應用程序常常須要添加檢索功能,開源的 ElasticSearch 是目前全文搜索引擎的首選。他能夠快速的存儲、搜索和分析海量數據。Spring Boot經過整合Spring Data ElasticSearch爲咱們提供了很是便捷的檢索功能支持;java
Elasticsearch是一個分佈式搜索服務,提供Restful API,底層基於Lucene,採用多shard(分片)的方式保證數據安全,而且提供自動resharding的功能,維基百科、github等大型的站點也是採用了ElasticSearch做爲其搜索服務。node
以員工文檔形式存儲爲例:一個文檔表明一個員工數據。存儲數據到ElasticSearch的行爲叫作索引,但在索引一個文檔以前,須要肯定將文檔存儲在哪裏。linux
一個ElasticSearch集羣能夠包含多個索引,相應的每一個索引能夠包含多個類型。這些不一樣的類型存儲着多個文檔,每一個文檔又有多個屬性。咱們能夠將其和咱們經常使用的關係數據庫概念進行類比:git
有關ES更多的信息請參考官方文檔,必定要先了解其內容,才能更好的使用ES。github
一、拉取鏡像web
docker pull registry.docker-cn.com/library/elasticsearch
二、限制內存啓動(若是你的內存不夠大的話)spring
docker run -e ES_JAVA_OPTS="-Xms256m -Xmx256m" -d -p 9200:9200 -p 9300:9300 --name es01 鏡像id
ES是使用java編寫的服務,默認啓動下會佔用2G的內存,若是你的服務器配置不太夠的話,推薦使用自定義配置:-Xms是初始的堆內存大小,-Xms是最大使用的堆內存大小。docker
-d表明後臺運行 -p表明端口映射,ES默認使用兩個端口9200和9300,鏡像id能夠經過docker images命令查看。shell
真正在運用中關於elasticsearch服務確定須要特殊的維護的,通常不會這麼隨意。固然,咱們這裏只是測試,可是若是您要將其運用到實際的生產環境中,還要多去了解一些相關的知識。
運行命令後,咱們訪問瀏覽器:服務器IP:9200,可以獲得ES的返回數聽說明啓動成功了。
{ "name" : "jAQflp4", "cluster_name" : "docker-cluster", "cluster_uuid" : "d9_vBcTfSS-2CBrL9DdCpw", "version" : { "number" : "6.5.3", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "159a78a", "build_date" : "2018-12-06T20:11:28.826501Z", "build_snapshot" : false, "lucene_version" : "7.5.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" }
一樣適用springbootinitializer建立項目,選擇web模塊以及NoSql的elasticsearch模塊,完成建立,能夠看見,springboot爲咱們引入的elasticsearch啓動器pom以下:
<dependency> <groupId>org.springframework.boot</groupId> <artifactId>spring-boot-starter-data-elasticsearch</artifactId> </dependency>
spring boot默認使用spring-data-elasticsearch來進行操做的。
經過查看源碼咱們能夠知道,springboot默認使用兩種技術來和ES交互:
package org.springframework.boot.autoconfigure.data.elasticsearch; import java.util.Properties; import org.elasticsearch.client.Client; import org.elasticsearch.client.transport.TransportClient; import org.springframework.boot.autoconfigure.condition.ConditionalOnClass; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty; import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; import org.springframework.data.elasticsearch.client.TransportClientFactoryBean; /** * {@link org.springframework.boot.autoconfigure.EnableAutoConfiguration * Auto-configuration} for Elasticsearch. * * @author Artur Konczak * @author Mohsin Husen * @author Andy Wilkinson * @since 1.1.0 */ @Configuration @ConditionalOnClass({ Client.class, TransportClientFactoryBean.class }) @ConditionalOnProperty(prefix = "spring.data.elasticsearch", name = "cluster-nodes", matchIfMissing = false) @EnableConfigurationProperties(ElasticsearchProperties.class) public class ElasticsearchAutoConfiguration { private final ElasticsearchProperties properties; public ElasticsearchAutoConfiguration(ElasticsearchProperties properties) { this.properties = properties; } @Bean @ConditionalOnMissingBean public TransportClient elasticsearchClient() throws Exception { TransportClientFactoryBean factory = new TransportClientFactoryBean(); factory.setClusterNodes(this.properties.getClusterNodes()); factory.setProperties(createProperties()); factory.afterPropertiesSet(); return factory.getObject(); } private Properties createProperties() { Properties properties = new Properties(); properties.put("cluster.name", this.properties.getClusterName()); properties.putAll(this.properties.getProperties()); return properties; } }
再來看其配置屬性類:
/* * Copyright 2012-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ package org.springframework.boot.autoconfigure.data.elasticsearch; import java.util.HashMap; import java.util.Map; import org.springframework.boot.context.properties.ConfigurationProperties; /** * Configuration properties for Elasticsearch. * * @author Artur Konczak * @author Mohsin Husen * @since 1.1.0 */ @ConfigurationProperties(prefix = "spring.data.elasticsearch") public class ElasticsearchProperties { /** * Elasticsearch cluster name. */ private String clusterName = "elasticsearch"; /** * Comma-separated list of cluster node addresses. */ private String clusterNodes; /** * Additional properties used to configure the client. */ private Map<String, String> properties = new HashMap<>(); public String getClusterName() { return this.clusterName; } public void setClusterName(String clusterName) { this.clusterName = clusterName; } public String getClusterNodes() { return this.clusterNodes; } public void setClusterNodes(String clusterNodes) { this.clusterNodes = clusterNodes; } public Map<String, String> getProperties() { return this.properties; } public void setProperties(Map<String, String> properties) { this.properties = properties; } }
咱們能夠得出:
前面提到,要操做ES的話須要預先導入JEST包,去到maven網站查看jest包有不少版本,咱們能夠經過訪問本身的ES網站查看ES的大版本號,例如個人系統安裝的ES,訪問IP:9200獲得以下的反饋:
{ "name" : "jAQflp4", "cluster_name" : "docker-cluster", "cluster_uuid" : "d9_vBcTfSS-2CBrL9DdCpw", "version" : { "number" : "6.5.3", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "159a78a", "build_date" : "2018-12-06T20:11:28.826501Z", "build_snapshot" : false, "lucene_version" : "7.5.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" }
version.number爲6.5.3,所以咱們下載6.x版本的jest,導入pom以下:
<!-- https://mvnrepository.com/artifact/io.searchbox/jest --> <dependency> <groupId>io.searchbox</groupId> <artifactId>jest</artifactId> <version>6.3.1</version> </dependency>
/* * Copyright 2012-2017 the original author or authors. * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * * http://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. * See the License for the specific language governing permissions and * limitations under the License. */ package org.springframework.boot.autoconfigure.elasticsearch.jest; import java.time.Duration; import java.util.ArrayList; import java.util.Collections; import java.util.List; import org.springframework.boot.context.properties.ConfigurationProperties; /** * Configuration properties for Jest. * * @author Stephane Nicoll * @since 1.4.0 */ @ConfigurationProperties(prefix = "spring.elasticsearch.jest") public class JestProperties { /** * Comma-separated list of the Elasticsearch instances to use. */ private List<String> uris = new ArrayList<>( Collections.singletonList("http://localhost:9200")); /** * Login username. */ private String username; /** * Login password. */ private String password; /** * Whether to enable connection requests from multiple execution threads. */ private boolean multiThreaded = true; /** * Connection timeout. */ private Duration connectionTimeout = Duration.ofSeconds(3); /** * Read timeout. */ private Duration readTimeout = Duration.ofSeconds(3); /** * Proxy settings. */ private final Proxy proxy = new Proxy(); public List<String> getUris() { return this.uris; } public void setUris(List<String> uris) { this.uris = uris; } public String getUsername() { return this.username; } public void setUsername(String username) { this.username = username; } public String getPassword() { return this.password; } public void setPassword(String password) { this.password = password; } public boolean isMultiThreaded() { return this.multiThreaded; } public void setMultiThreaded(boolean multiThreaded) { this.multiThreaded = multiThreaded; } public Duration getConnectionTimeout() { return this.connectionTimeout; } public void setConnectionTimeout(Duration connectionTimeout) { this.connectionTimeout = connectionTimeout; } public Duration getReadTimeout() { return this.readTimeout; } public void setReadTimeout(Duration readTimeout) { this.readTimeout = readTimeout; } public Proxy getProxy() { return this.proxy; } public static class Proxy { /** * Proxy host the HTTP client should use. */ private String host; /** * Proxy port the HTTP client should use. */ private Integer port; public String getHost() { return this.host; } public void setHost(String host) { this.host = host; } public Integer getPort() { return this.port; } public void setPort(Integer port) { this.port = port; } } }
能夠獲取到咱們能夠配置的內容。
spring: elasticsearch: jest: uris: http://10.21.1.47:9200
先建立一個bean來輔助測試
package com.zhaoyi.elastic.bean; import io.searchbox.annotations.JestId; public class Article { @JestId private Integer id; private String author; private String title; private String content; public Integer getId() { return id; } public void setId(Integer id) { this.id = id; } public String getAuthor() { return author; } public void setAuthor(String author) { this.author = author; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } public String getContent() { return content; } public void setContent(String content) { this.content = content; } }
一個傳統的javabean,同時別忘了爲其id字段標準@JestId註解。
接下來咱們編寫測試類,測試與ES之間的交互。
@Autowired private JestClient jestClient; @Test public void jestIndexTest() { // 在ES中保存一個文檔 Article article = new Article(1, "渡航", "個人青春戀愛物語果真有問題", "是輕小說家渡航著做..."); // 構建一個索引功能 Index build = new Index.Builder(article).index("joyblack").type("article").build(); try { // 執行操做 jestClient.execute(build); } catch (IOException e) { e.printStackTrace(); } }
咱們在postman或者瀏覽器中輸入地址: IP:9200/joyblack/article/1,就能夠查看到新插入的文檔信息:
{ "_index": "joyblack", "_type": "article", "_id": "1", "_version": 1, "found": true, "_source": { "id": 1, "author": "渡航", "title": "個人青春戀愛物語果真有問題", "content": "是輕小說家渡航著做..." } }
咱們能夠修改插入article的信息,多插入幾條article數據到ES中,方便咱們解析來測試查詢操做
如今查看咱們的索引joyblack的文檔類型article中,有以下3條數據:
{ "took": 4, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 3, "max_score": 1, "hits": [ { "_index": "joyblack", "_type": "article", "_id": "2", "_score": 1, "_source": { "id": 2, "author": "伏見司", "title": "個人妹妹哪有這麼可愛", "content": "是日本輕小說家伏見司創做..." } }, { "_index": "joyblack", "_type": "article", "_id": "1", "_score": 1, "_source": { "id": 1, "author": "渡航", "title": "個人青春戀愛物語果真有問題", "content": "是輕小說家渡航著做..." } }, { "_index": "joyblack", "_type": "article", "_id": "3", "_score": 1, "_source": { "id": 3, "author": "南懷瑾", "title": "論語別裁", "content": "是2005年復旦大學出版社出版書籍,做者南懷瑾..." } } ] } }
咱們來搜索做爲爲杜航的文檔信息:
@Test package com.zhaoyi.elastic; import com.zhaoyi.elastic.bean.Article; import io.searchbox.client.JestClient; import io.searchbox.core.Index; import io.searchbox.core.Search; import io.searchbox.core.SearchResult; import org.junit.Test; import org.junit.runner.RunWith; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.test.context.junit4.SpringRunner; import java.io.IOException; @RunWith(SpringRunner.class) @SpringBootTest public class ElasticApplicationTests { private static String index = "joyblack"; private static String type = "article"; @Autowired private JestClient jestClient; @Test public void jestIndexTest() { // 在ES中保存一個文檔 Article article = new Article(4, "hawking", "時間簡史", "是英國物理學家斯蒂芬·威廉·霍金創做的科學著做..."); // 構建一個索引功能 Index build = new Index.Builder(article).index(index).type(type).build(); try { // 執行操做 jestClient.execute(build); } catch (IOException e) { e.printStackTrace(); } } @Test public void jestSearchTest(){ String query = "{\n" + "\t\"query\":{\n" + "\t\t\"match\":{\n" + "\t\t\t\"author\": \"杜航\"\n" + "\t\t}\n" + "\t}\n" + "\t\n" + "}"; // 構建搜索功能 Search build = new Search.Builder(query).addIndex(index).addType(type).build(); try { SearchResult result = jestClient.execute(build); System.out.println(result.getJsonString()); } catch (IOException e) { e.printStackTrace(); } } }
能夠獲得輸出結果:
{"took":4,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":1,"max_score":0.2876821,"hits":[{"_index":"joyblack","_type":"article","_id":"1","_score":0.2876821,"_source":{"id":1,"author":"渡航","title":"個人青春戀愛物語果真有問題","content":"是輕小說家渡航著做..."}}]}}
若是你使用term方式查詢的話,請注意「杜航」這樣的漢語在索引內部是分爲兩部分的「杜」和「航」,因此會查詢不到。
其餘的操做方式能夠參考:GitHub Jest項目地址
先複習一下咱們之間總結的關於使用springdata方式操做ES的內容:
所以,咱們開始學習使用springdata操做ES。訪問9200獲取ES服務的信息:
{ "name" : "jAQflp4", "cluster_name" : "docker-cluster", "cluster_uuid" : "d9_vBcTfSS-2CBrL9DdCpw", "version" : { "number" : "6.5.3", "build_flavor" : "default", "build_type" : "tar", "build_hash" : "159a78a", "build_date" : "2018-12-06T20:11:28.826501Z", "build_snapshot" : false, "lucene_version" : "7.5.0", "minimum_wire_compatibility_version" : "5.6.0", "minimum_index_compatibility_version" : "5.0.0" }, "tagline" : "You Know, for Search" }
spring: data: elasticsearch: cluster-name: docker-cluster cluster-nodes: 10.21.1.47:9300
若是使用docker安裝的ES,則集羣名字默認爲docker-cluster,若是使用linux直接安裝ES,默認爲ES,這些均可以經過ES的配置文件進行修改。
若是你配置以後運行提示超時錯誤,請查看springboot爲咱們引入的elasticsearch組件的版本號和你所安裝的es服務的版本是否適配,適配表格以下(能夠從官方查找到)查詢網址 ||| |-|-| |spring data elasticsearch| elasticsearch| |3.2.x| 6.5.0| |3.1.x |6.2.2| |3.0.x| 5.5.0| |2.1.x| 2.4.0| |2.0.x| 2.2.0| |1.3.x |1.5.2| ||| 參考該版本關係修改您的適配關係。
我使用的是2.x版本的springboot版本,沒有遇到上述問題,1.x版本須要作一些調整:升級springboot版本或者安裝低版本的elasticsearch版本
注意使用spring data使用的是9300端口。9300是tcp通信端口,集羣間和TCPClient使用該端口;9200是http協議的RESTful接口,咱們前面使用的jest使用的就是9200。
package com.zhaoyi.elastic.bean; import org.springframework.data.elasticsearch.annotations.Document; @Document(indexName = "joyblack", type = "book") public class Book { private Integer id; private String author; private String title; private String content; public Book() { } public Book(Integer id, String author, String title, String content) { this.id = id; this.author = author; this.title = title; this.content = content; } public Integer getId() { return id; } public void setId(Integer id) { this.id = id; } public String getAuthor() { return author; } public void setAuthor(String author) { this.author = author; } public String getTitle() { return title; } public void setTitle(String title) { this.title = title; } public String getContent() { return content; } public void setContent(String content) { this.content = content; } @Override public String toString() { return "Book{" + "id=" + id + ", author='" + author + '\'' + ", title='" + title + '\'' + ", content='" + content + '\'' + '}'; } }
能夠看到咱們這裏用了joyblack作索引,而且使用book做爲type,若是你以前在joyblack裏作了其餘的類型,請預先刪除,所以高版本的ES只容許index type一對一存在,不然會報Rejecting mapping update to [索引] as the final mapping would have more than 1 type: [原type, 多餘的type].
要麼刪除以前的index,要麼複用以前的index.注意高版本不支持刪除type,只容許刪除index。
@Autowired ElasticsearchTemplate elasticsearchTemplate; @Test public void deleteTest(){ elasticsearchTemplate.deleteIndex("joyblack"); }
建立一個repository對象
package com.zhaoyi.elastic.repository; import com.zhaoyi.elastic.bean.Book; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; public interface BookRepository extends ElasticsearchRepository<Book, Integer> { }
其基本理念和咱們以前講springdata的時候已經說明過,如今就不在闡述,咱們直接進行測試:
@Autowired BookRepository bookRepository; @Test public void insertTest(){ Book book = new Book(4, "弗洛伊德", "夢的解析", "是弗洛伊德創做的哲學著做..."); bookRepository.index(book); }
能夠看到,ES中已經按需插入了數據
{ "took": 2, "timed_out": false, "_shards": { "total": 5, "successful": 5, "skipped": 0, "failed": 0 }, "hits": { "total": 1, "max_score": 1, "hits": [ { "_index": "joyblack", "_type": "book", "_id": "4", "_score": 1, "_source": { "id": 4, "author": "弗洛伊德", "title": "夢的解析", "content": "是弗洛伊德創做的哲學著做..." } } ] } }
@Autowired BookRepository bookRepository; @Test public void testInsertBatch(){ List<Book> books = Arrays.asList( new Book(1, "十文字青", "灰與幻想的格林姆迦爾", "爲日本輕小說做家十文字青著做..."), new Book(2, "長月達平", "Re:從零開始的異世界生活", "是弗洛伊德創做的哲學著做..."), new Book(3, "貴志祐介", "來自新世界", "根據貴志祐介原做同名小說改編的動畫做品...") ); books.parallelStream().forEach(b -> bookRepository.index(b)); }
這只是業務代碼層面的批量,更多的批量研究請慎入瞭解哦。
咱們先自定義一個查詢方法:
package com.zhaoyi.elastic.repository; import com.zhaoyi.elastic.bean.Book; import org.springframework.data.elasticsearch.repository.ElasticsearchRepository; import java.util.List; public interface BookRepository extends ElasticsearchRepository<Book, Integer> { List<Book> findBookByTitle(String name); }
接下里使用該查詢方法查詢title包含來自新世界的書籍信息:
@Test public void searchTest(){ System.out.println(bookRepository.findBookByTitle("來自新世界")); }
獲得反饋:
[Book{id=3, author='貴志祐介', title='來自新世界', content='根據貴志祐介原做同名小說改編的動畫做品...'}]
固然,咱們也可使用註解的方式來自定義查詢方法,具體的用法有些許不一樣,就須要咱們本身去查閱官方文檔慢慢理解了。