搭建spring boot+elasticsearch+activemq服務

時間  2019-12-14
標籤 搭建 spring boot+elasticsearch+activemq boot elasticsearch activemq 服務 欄目 Spring 简体版
原文   原文鏈接

目前時間是：2017-01-24html

本文不涉及activemq的安裝node

需求

activemq實時傳遞數據至服務 elasticsearch作索引 對外開放查詢接口 完成全文檢索git

環境

jdk：1.8github

spirng boot：1.4.3.RELEASEspring

elasticsearch：2.4.3express

activemq：5.13.2bootstrap

ES插件

head：版本好像無太大差異 能查數據就行vim

analysis-ik：1.10.3api

search-guard-2：2.4.3.9安全

search-guard-ssl：2.4.3.19

注意

做者遇到的最大問題就是版本兼容 網上資料少且版本較低 故列出如下版本矩陣 方便查閱

spring boot與elasticsearch：

Spring Boot Version (x) Spring Data Elasticsearch Version (y) Elasticsearch Version (z)
x <= 1.3.5 y <= 1.3.4 z <= 1.7.2*
x >= 1.4.x 2.0.0 <=y < 5.0.0** 2.0.0 <= z < 5.0.0**

 

 

 

矩陣來源以及更多版本兼容：https://github.com/spring-projects/spring-data-elasticsearch

elasticsearch與ik：個人ES版本爲2.x 對應以下

IK version ES version
master 2.4.0 -> master
1.10.3 2.4.3
1.9.5 2.3.5
1.9.4 2.3.4
1.9.3 2.3.3
1.9.0 2.3.0
1.8.1 2.2.1
1.7.0 2.1.1
1.5.0 2.0.0
1.2.6 1.0.0
1.2.5 0.90.x
1.1.3 0.20.x
1.0.0 0.16.2 -> 0.19.0

 

 

 

 

 

 

 

 

 

 

 

 

 

矩陣來源以及更多版本兼容：https://github.com/medcl/elasticsearch-analysis-ik

elasticsearch與search-guard以及search-guard-ssl：

Elasticsearch Version Latest Search Guard Version Search Guard SSL Version Commercial support available
1.x.y not available - -
2.0.x not available - -
2.1.x not available - -
2.2.0 2.2.0.7 2.2.0.16 Yes
2.3.1 available upon request - -
2.3.2 available upon request - -
2.3.3 2.3.3.10 2.3.3.19 YES
2.3.4 2.3.4.10 2.3.4.19 YES
2.3.5 2.3.5.10 2.3.5.19 YES
2.4.0 2.4.0.10 2.4.0.19 YES
2.4.1 2.4.1.10 2.4.1.19 YES
2.4.2 2.4.2.10 2.4.2.19 YES
2.4.3 2.4.3.10 2.4.3.19 YES
2.4.4 2.4.4.10 2.4.4.19 YES
5.0.0 5.0.0-10 (comes bundled since SG 5) YES
5.0.1 5.0.1-10 (comes bundled since SG 5) YES
5.0.2 5.0.2-10 (comes bundled since SG 5) YES
5.1.1 5.1.1-10 (comes bundled since SG 5) YES
5.1.2 5.1.2-10 (comes bundled since SG 5) YES

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

矩陣來源以及更多版本兼容：https://github.com/floragunncom/search-guard/wiki

開始

安裝elasticsearch

個人安裝目錄：/usr/local

注意：elasticsearch不能用root用戶運行 因此建立你的用戶組和用戶 切換到新用戶再安裝 如何建立切換 請自行搜索

wget https://download.elastic.co/elasticsearch/release/org/elasticsearch/distribution/tar/elasticsearch/2.4.3/elasticsearch-2.4.3.tar.gz

tar -zxvf elasticsearch-2.4.3.tar.gz 

cd elasticsearch-2.4.3/config/

vim elasticsearch.yml 

# ======================== Elasticsearch Configuration =========================
#
# NOTE: Elasticsearch comes with reasonable defaults for most settings.
#       Before you set out to tweak and tune the configuration, make sure you
#       understand what are you trying to accomplish and the consequences.
#
# The primary way of configuring a node is via this file. This template lists
# the most important settings you may want to configure for a production cluster.
#
# Please see the documentation for further information on configuration options:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/setup-configuration.html>
#
# ---------------------------------- Cluster -----------------------------------
#
# Use a descriptive name for your cluster:
#
  cluster.name: 你的集羣名稱
#
# ------------------------------------ Node ------------------------------------
#
# Use a descriptive name for the node:
#
# node.name: node-1
#
# Add custom attributes to the node:
#
# node.rack: r1
#
# ----------------------------------- Paths ------------------------------------
#
# Path to directory where to store the data (separate multiple locations by comma):
#
# path.data: /path/to/data
#
# Path to log files:
#
# path.logs: /path/to/logs
#
# ----------------------------------- Memory -----------------------------------
#
# Lock the memory on startup:
#
# bootstrap.memory_lock: true
#
# Make sure that the `ES_HEAP_SIZE` environment variable is set to about half the memory
# available on the system and that the owner of the process is allowed to use this limit.
#
# Elasticsearch performs poorly when the system is swapping the memory.
#
# ---------------------------------- Network -----------------------------------
#
# Set the bind address to a specific IP (IPv4 or IPv6):
#
  network.host: 0.0.0.0
#
# Set a custom port for HTTP:
#
# http.port: 9200
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-network.html>
#
# --------------------------------- Discovery ----------------------------------
#
# Pass an initial list of hosts to perform discovery when new node is started:
# The default list of hosts is ["127.0.0.1", "[::1]"]
#
# discovery.zen.ping.unicast.hosts: ["host1", "host2"]
#
# Prevent the "split brain" by configuring the majority of nodes (total number of nodes / 2 + 1):
#
# discovery.zen.minimum_master_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-discovery.html>
#
# ---------------------------------- Gateway -----------------------------------
#
# Block initial recovery after a full cluster restart until N nodes are started:
#
# gateway.recover_after_nodes: 3
#
# For more information, see the documentation at:
# <http://www.elastic.co/guide/en/elasticsearch/reference/current/modules-gateway.html>
#
# ---------------------------------- Various -----------------------------------
#
# Disable starting multiple nodes on a single system:
#
# node.max_local_storage_nodes: 1
#
# Require explicit names when deleting indices:
#
# action.destructive_requires_name: true

我修改了兩個地方 cluster.name和network.host

cd ../bin/

./elasticsearch

[2017-01-24 10:02:49,627][INFO ][node                     ] [Ariel] version[2.4.3], pid[23274], build[d38a34e/2016-12-07T16:28:56Z]
[2017-01-24 10:02:49,628][INFO ][node                     ] [Ariel] initializing ...
[2017-01-24 10:02:50,259][INFO ][plugins                  ] [Ariel] modules [reindex, lang-expression, lang-groovy], plugins [], sites []
[2017-01-24 10:02:50,279][INFO ][env                      ] [Ariel] using [1] data paths, mounts [[/ (overlay)]], net usable_space [75.3gb], net total_space [113.9gb], spins? [possibly], types [overlay]
[2017-01-24 10:02:50,279][INFO ][env                      ] [Ariel] heap size [990.7mb], compressed ordinary object pointers [true]
[2017-01-24 10:02:52,051][INFO ][node                     ] [Ariel] initialized
[2017-01-24 10:02:52,051][INFO ][node                     ] [Ariel] starting ...
[2017-01-24 10:02:52,110][INFO ][transport                ] [Ariel] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2017-01-24 10:02:52,116][INFO ][discovery                ] [Ariel] elasticsearch/MI21JVBWSbKfj9nC1V6N9w
[2017-01-24 10:02:55,166][INFO ][cluster.service          ] [Ariel] new_master {Ariel}{MI21JVBWSbKfj9nC1V6N9w}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-join(elected_as_master, [0] joins received)
[2017-01-24 10:02:55,197][INFO ][http                     ] [Ariel] publish_address {127.0.0.1:9200}, bound_addresses {[::1]:9200}, {127.0.0.1:9200}
[2017-01-24 10:02:55,197][INFO ][node                     ] [Ariel] started

出現相似這樣的信息 說明安裝成功

安裝head插件

elasticsearch根目錄執行

bin/plugin install mobz/elasticsearch-head

注意：2.0如下版本應該是 -install

訪問地址：http://ip:9200/_plugin/head/

安裝analysis-ik插件

下載地址：https://github.com/medcl/elasticsearch-analysis-ik/tree/v1.10.3

使用maven打包：mvn clean package

生成的zip包在target/releases下

在elasticsearch-2.4.3/plugins下建立ik目錄

將zip包放到該目錄下並解壓 解壓出以下文件

 編輯elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加以下內容

index:  
      analysis:                     
        analyzer:        
          ik:  
              alias: [ik_analyzer]  
              type: org.elasticsearch.index.analysis.IkAnalyzerProvider  
          ik_max_word:  
              type: ik  
              use_smart: false  
          ik_smart:  
              type: ik  
              use_smart: true

或者

index.analysis.analyzer.ik.type : 「ik」

測試：http://ip:9200/_analyze?analyzer=ik&pretty=true&text=我是中國人

安裝searchguard

elasticsearch根目錄執行

bin/plugin install -b com.floragunn/search-guard-2/2.4.3.9

bin/plugin install -b com.floragunn/search-guard-ssl/2.4.3.19

下載 searchguard-ssl 的包，裏面包含自動建立證書的腳本：

wget https://github.com/floragunncom/search-guard-ssl/archive/v2.4.3.19.zip

unzip v2.4.3.19.zip 

cd search-guard-ssl-2.4.3.19/example-pki-scripts/

有三個腳本

gen_client_node_cert.sh 建立客戶端證書
gen_node_cert.sh        建立節點證書
gen_root_ca.sh          建立根證書

編輯腳本 vim example.sh

#!/bin/bash
set -e
./clean.sh
./gen_root_ca.sh password password 
./gen_node_cert.sh 0 password  password 
./gen_node_cert.sh 1 password  password 
./gen_client_node_cert.sh admin password password 
cp truststore.jks node-0-keystore.jks /usr/local/elasticsearch-2.4.3/config/
cp truststore.jks admin-keystore.jks /usr/local/elasticsearch-2.4.3/plugins/search-guard-2/sgconfig/

./example.sh 

能夠發現 最後兩句就是將證書cp到相應目錄

編輯elasticsearch-2.4.3/config/elasticsearch.yml配置文件 添加以下內容

#############################################################################################
#                                     SEARCH GUARD                                          #
#                                     Configuration                                         #
#############################################################################################
  security.manager.enabled: false
  searchguard.authcz.admin_dn:
   - "CN=admin, OU=client, O=client, L=Test, C=DE"

#############################################################################################
#                                     SEARCH GUARD SSL                                      #
#                                       Configuration                                       #
#############################################################################################


#############################################################################################
# Transport layer SSL                                                                       #
#                                                                                           #
#############################################################################################
# Enable or disable node-to-node ssl encryption (default: true)
# searchguard.ssl.transport.enabled: true
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.keystore_type: PKCS12
# Relative path to the keystore file (mandatory, this stores the server certificates), must be placed under the config/ dir
  searchguard.ssl.transport.keystore_filepath: node-0-keystore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.keystore_alias: my_alias
# Keystore password (default: changeit)
  searchguard.ssl.transport.keystore_password: password
# JKS or PKCS12 (default: JKS)
#searchguard.ssl.transport.truststore_type: PKCS12
# Relative path to the truststore file (mandatory, this stores the client/root certificates), must be placed under the config/ dir
  searchguard.ssl.transport.truststore_filepath: truststore.jks
# Alias name (default: first alias which could be found)
#searchguard.ssl.transport.truststore_alias: my_alias
# Truststore password (default: changeit)
  searchguard.ssl.transport.truststore_password: password
# Enforce hostname verification (default: true)
# searchguard.ssl.transport.enforce_hostname_verification: true
# If hostname verification specify if hostname should be resolved (default: true)
# searchguard.ssl.transport.resolve_hostname: true
# Use native Open SSL instead of JDK SSL if available (default: true)
# searchguard.ssl.transport.enable_openssl_if_available: false

在elasticsearch根目錄 執行命令 將配置插入

./plugins/search-guard-2/tools/sgadmin.sh -cn 集羣名稱 -h hostname -cd plugins/search-guard-2/sgconfig -ks plugins/search-guard-2/sgconfig/admin-keystore.jks -kspass password -ts plugins/search-guard-2/sgconfig/truststore.jks -tspass password -nhnv

注意：elasticsearch的服務必須是運行狀態

elasticsearch-2.4.3/plugins/search-guard-2/sgconfig下的配置文件是管理用戶角色的

安裝配置成功後 任何客戶端訪問elasticsearch 需提供用戶名及密碼

至此服務端安裝結束

客戶端將以源碼方式提供 爲公司信息安全着想 僅提供關鍵性代碼供參考 沒法運行

elasticsearch download

相關文章
相關標籤/搜索
每日一句
    每一个你不满意的现在，都有一个你没有努力的曾经。
本站公眾號
   歡迎關注本站公眾號,獲取更多信息
聯繫我們 最近搜索 最新文章 沪ICP备13005482号-10 MyBatis教程 SQL 教程 MySQL教程 Java 教程 Thymeleaf 教程 Hibernate教程 Spring教程 Redis教程