安裝elasticsearch 、redis、zookeeper 、kafka

前言

[TOC]

之前的筆記，從新整理了一下，都在這裏了。
操做系統是Centos 7

安裝 elasticsearch

rpm -ivh jdk-8u171-linux-x64.rpm 
cd /mnt/data/source
wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.5.2.tar.gz
tar zxmf elasticsearch-1.5.2.tar.gz 
mv elasticsearch-1.5.2 /mnt/data/apps/

插件 analysis-ik 及 head

analysis-ik

mkdir -p /mnt/data/apps/elasticsearch-1.5.2/plugins/analysis-ik

wget https://github.com/medcl/elasticsearch-analysis-ik/archive/v1.2.6.zip -O analysis-ik.zip

unzip analysis-ik.zip 
cd elasticsearch-analysis-ik-1.2.6 
mvn clean
mvn compile
mvn package
cp target/elasticsearch-analysis-ik-1.2.6.jar /mnt/data/apps/elasticsearch-1.5.2/plugins/analysis-ik

cat >>/mnt/data/apps/elasticsearch-1.5.2/config/elasticsearch.yml <<EOF
index:
  analysis:
    analyzer:
      ik:
          alias: [ik_analyzer]
          type: org.elasticsearch.index.analysis.IkAnalyzerProvider
      ik_max_word:
          type: ik
          use_smart: false
      ik_smart:
          type: ik
          use_smart: true
EOF

注意：若是你的版本不對應，可能會出現以下錯誤：
{"error":"IndexCreationException[[index] failed to create index]; nested: ElasticsearchIllegalArgumentException[failed to find analyzer type [ik] or tokenizer for [ik_max_word]]; nested: NoClassSettingsException[Failed to load class setting [type] with value [ik]]; nested: ClassNotFoundException[org.elasticsearch.index.analysis.ik.IkAnalyzerProvider]; ","status":400}

elasticsearch-head

./plugin -install mobz/elasticsearch-head/1.x

http://localhost:9200/_plugin/head/

配置服務啓動

cat >/usr/lib/systemd/system/elasticsearch.service <<EOF
[Unit]
Description=The ElasticSearch server
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
ExecStart=/mnt/data/apps/elasticsearch-1.5.2/bin/elasticsearch -d 
ExecReload=/bin/kill -s HUP \$MAINPID
KillSignal=SIGQUIT
TimeoutStopSec=5
KillMode=process
PrivateTmp=true

[Install]
WantedBy=multi-user.target
EOF

systemctl enable elasticsearch
systemctl start elasticsearch

redis 安裝配置

yum install redis

mkdir -p /mnt/data/apps/redis/data
chown -R redis:redis /mnt/data/apps/redis

redis 配置文件(/etc/redis.conf)修改以下幾個參數

bind <hostip>   ##綁定服務的IP ，能夠被局域網中的其餘機器訪問

dir /mnt/data/apps/redis/data  ## 數據目錄，數據庫的寫入會在這個目錄
## 1G
maxmemory 1073741824   ##   （https://www.cnblogs.com/changbosha/p/5849982.html）
maxmemory-policy volatile-lru   ## 內存

systemctl enable redis
systemctl start redis

啓動文件

[Unit]
Description=Redis persistent key-value database
After=network.target

[Service]
ExecStart=/usr/bin/redis-server /etc/redis.conf --supervised systemd
ExecStop=/usr/libexec/redis-shutdown
Type=notify
User=redis
Group=redis
RuntimeDirectory=redis
RuntimeDirectoryMode=0755

[Install]
WantedBy=multi-user.target

zookeeper 1臺運行3個實例

wget http://mirror.bit.edu.cn/apache/zookeeper/zookeeper-3.4.12/zookeeper-3.4.12.tar.gz
mkdir -p /mnt/data/apps/zookeeper-data/zk1/data /mnt/data/apps/zookeeper-data/zk2/data /mnt/data/apps/zookeeper-data/zk3/data
mkdir -p /mnt/data/apps/zookeeper-data/zk1/logs /mnt/data/apps/zookeeper-data/zk2/logs /mnt/data/apps/zookeeper-data/zk3/logs
cd /mnt/data/apps/zookeeper-3.4.12/conf

cat > zoo1.conf <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/mnt/data/apps/zookeeper-data/zk1/data
dataLogDir=/mnt/data/apps/zookeeper-data/zk1/logs
clientPort=2181

server.1=localhost:2287:3387
server.2=localhost:2288:3388
server.3=localhost:2289:3389
EOF

cat > zoo2.conf <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/mnt/data/apps/zookeeper-data/zk2/data
dataLogDir=/mnt/data/apps/zookeeper-data/zk2/logs
clientPort=2182

server.1=localhost:2287:3387
server.2=localhost:2288:3388
server.3=localhost:2289:3389
EOF

cat > zoo3.conf <<EOF
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/mnt/data/apps/zookeeper-data/zk3/data
dataLogDir=/mnt/data/apps/zookeeper-data/zk3/logs
clientPort=2181

server.1=localhost:2287:3387
server.2=localhost:2288:3388
server.3=localhost:2289:3389
EOF

echo "1" >/mnt/data/apps/zookeeper-data/zk1/data/myid
echo "2" >/mnt/data/apps/zookeeper-data/zk2/data/myid
echo "3" >/mnt/data/apps/zookeeper-data/zk3/data/myid

./bin/zkServer.sh start conf/zoo1.conf 
./bin/zkServer.sh start conf/zoo2.conf 
./bin/zkServer.sh start conf/zoo3.conf

kafka

配置參數見https://blog.csdn.net/lizhitao/article/details/25667831)

wget https://archive.apache.org/dist/kafka/0.9.0.1/kafka_2.10-0.9.0.1.tgz

tar zxmf kafka_2.10-0.9.0.1.tgz -C /mnt/data/apps/

## 配置中主意把本身的IP填上 
listeners=PLAINTEXT://IP:9092

kafka-server-start.sh  -daemon  config/server.properties  &

配置服務

cat >/usr/lib/systemd/system/kafka.service <<EOF
[Unit]
Description=The Kafka server
After=network.target remote-fs.target nss-lookup.target

[Service]
Type=forking
ExecStart=/mnt/data/apps/kafka_2.10-0.9.0.1/bin/kafka-server-start.sh -daemon /mnt/data/apps/kafka_2.10-0.9.0.1/config/server.properties
ExecStop=/mnt/data/apps/kafka_2.10-0.9.0.1/bin/kafka-server-stop.sh  /mnt/data/apps/kafka_2.10-0.9.0.1/config/server.properties &

[Install]
WantedBy=multi-user.target

EOF

systemctl enable kafka
systemctl start kafka

 

問題集錦

• 現象：redis運行一段時間以後就會出現假死的現象

1. 查看日誌是否有關overcommit_memory 的報錯，如有則修改以下：
   echo 1 > /proc/sys/vm/overcommit_memory
   sysctl -p
2. 檢查redis.conf，確認以下參數是否正確 maxmemory <bytes> ## 默認是無限制，建議爲物理內存3/4maxmemory-policy allkeys-lru ## 數據清除策略maxmemory-samples 3 ##上面策略的取樣值以便檢查