一:集羣的安裝php
###【在多臺機器上執行下面的命令】### #es啓動時須要使用非root用戶,全部建立一個xiaoniu用戶: useradd xiaoniu #爲hadoop用戶添加密碼: echo 123456 | passwd --stdin xiaoniu #將bigdata添加到sudoers echo "xiaoniu ALL = (root) NOPASSWD:ALL" | tee /etc/sudoers.d/xiaoniu chmod 0440 /etc/sudoers.d/xiaoniu #解決sudo: sorry, you must have a tty to run sudo問題,在/etc/sudoer註釋掉 Default requiretty 一行 sudo sed -i 's/Defaults requiretty/Defaults:xiaoniu !requiretty/' /etc/sudoers #建立一個bigdata目錄 mkdir /{bigdata,data} #給相應的目錄添加權限 chown -R xiaoniu:xiaoniu /{bigdata,data} ------------------------------------------------------------------------------------------------- 1.安裝jdk(jdk要求1.8.20以上) 2.上傳es安裝包 3.解壓es tar -zxvf elasticsearch-5.4.3.tar.gz -C /bigdata/ 4.修改配置 vi /bigdata/elasticsearch-5.4.3/config/elasticsearch.yml #集羣名稱,經過組播的方式通訊,經過名稱判斷屬於哪一個集羣 cluster.name: bigdata #節點名稱,要惟一 node.name: es-1 #數據存放位置 path.data: /data/es/data #日誌存放位置(可選) path.logs: /data/es/logs #es綁定的ip地址 network.host: 192.168.10.16 #初始化時可進行選舉的節點 discovery.zen.ping.unicast.hosts: ["node-4", "node-5", "node-6"] /bigdata/elasticsearch-5.4.3/bin/elasticsearch -d ------------------------------------------------------------------------------------------------- #出現錯誤 [1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65536] [2]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] ---su root 切換到root用戶 #用戶最大可建立文件數過小 sudo vi /etc/security/limits.conf * soft nofile 65536 * hard nofile 65536 #查看可打開文件數量 ulimit -Hn #最大虛擬內存過小 sudo vi /etc/sysctl.conf vm.max_map_count=262144 #查看虛擬內存的大小 sudo sysctl -p 5.使用scp拷貝到其餘節點 scp -r elasticsearch-5.4.3/ node-5:$PWD scp -r elasticsearch-5.4.3/ node-6:$PWD 6.在其餘節點上修改es配置,須要修改的有node.name和network.host -----TMD 改完這些,我機器還得重啓一下才行!!!! 7.啓動es(/bigdata/elasticsearch-5.4.3/bin/elasticsearch -h查看幫助文檔) /bigdata/elasticsearch-5.4.3/bin/elasticsearch -d 8.用瀏覽器訪問es所在機器的9200端口 http://192.168.10.16:9200/ { "name" : "node-2", "cluster_name" : "bigdata", "cluster_uuid" : "v4AHbENYQ8-M3Aq8J5OZ5g", "version" : { "number" : "5.4.3", "build_hash" : "eed30a8", "build_date" : "2017-06-22T00:34:03.743Z", "build_snapshot" : false, "lucene_version" : "6.5.1" }, "tagline" : "You Know, for Search" } kill `ps -ef | grep Elasticsearch | grep -v grep | awk '{print $2}'` #查看集羣狀態 curl -XGET 'http://192.168.10.16:9200/_cluster/health?pretty' http://192.168.10.16:9200/_cluster/health?pretty curl -XGET 'http://ES-01:9200/_cluster/health?pretty' http://ES-02:9200/_cluster/health?pretty ------------------------------------------------------------------------------------------------------------------ RESTful接口URL的格式: http://192.168.10.16:9200/<index>/<type>/[<id>] 其中index、type是必須提供的。 id是可選的,不提供es會自動生成。 index、type將信息進行分層,利於管理。 index能夠理解爲數據庫;type理解爲數據表;id至關於數據庫表中記錄的主鍵,是惟一的。 #向store索引中添加一些書籍 curl -XPUT 'http://192.168.10.16:9200/store/books/1' -d '{ "title": "Elasticsearch: The Definitive Guide", "name" : { "first" : "Zachary", "last" : "Tong" }, "publish_date":"2015-02-06", "price":"49.99" }' #在linux中經過curl的方式查詢 curl -XGET 'http://192.168.10.18:9200/store/books/1' #經過瀏覽器查詢 http://192.168.10.18:9200/store/books/1 #在添加一個書的信息 curl -XPUT 'http://192.168.10.18:9200/store/books/2' -d '{ "title": "Elasticsearch Blueprints", "name" : { "first" : "Vineeth", "last" : "Mohan" }, "publish_date":"2015-06-06", "price":"35.99" }' # 經過ID得到文檔信息 curl -XGET 'http://192.168.10.18:9200/store/books/1' #在瀏覽器中查看 http://92.168.10.18:9200/store/books/1 # 經過_source獲取指定的字段 curl -XGET 'http://192.168.10.16:9200/store/books/1?_source=title' curl -XGET 'http://192.168.10.16:9200/store/books/1?_source=title,price' curl -XGET 'http://192.168.10.16:9200/store/books/1?_source' #能夠經過覆蓋的方式更新 curl -XPUT 'http://192.168.10.16:9200/store/books/1' -d '{ "title": "Elasticsearch: The Definitive Guide", "name" : { "first" : "Zachary", "last" : "Tong" }, "publish_date":"2016-02-06", "price":"99.99" }' # 或者經過 _update API的方式單獨更新你想要更新的 curl -XPOST 'http://192.168.10.16:9200/store/books/1/_update' -d '{ "doc": { "price" : 88.88 } }' curl -XGET 'http://192.168.10.16:9200/store/books/1' #刪除一個文檔 curl -XDELETE 'http://192.168.10.16:9200/store/books/1' curl -XPUT 'http://192.168.10.16:9200/store/books/4' -d '{ "title": "Elasticsearch: The Definitive Guide", "author": "Guide", "publish_date":"2016-02-06", "price":"35.99" }' #https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html # 最簡單filter查詢 # SELECT * FROM books WHERE price = 35.99 # filtered 查詢價格是35.99的 # 返回的的分是1.0 curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query": { "bool": { "must": { "match_all": {} }, "filter": { "term": { "price": 35.99 } } } } }' # 返回的的分是1.0 curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query": { "constant_score": { "filter": { "term": { "price": 35.99 } } } } }' # 返回的的分是0.0 curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query": { "bool": { "filter" : { "term" : { "price" : 35.99 } } } } }' #指定多個值 curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query" : { "bool" : { "filter" : { "terms" : { "price" : [35.99, 99.99] } } } } }' curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query" : { "bool" : { "must": { "match_all": {} }, "filter" : { "terms" : { "price" : [35.99, 99.99] } } } } }' # SELECT * FROM books WHERE publish_date = "2015-02-06" curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query" : { "bool" : { "filter" : { "term" : { "publish_date" : "2015-02-06" } } } } }' # bool過濾查詢,能夠作組合過濾查詢 # SELECT * FROM books WHERE (price = 35.99 OR price = 99.99) AND publish_date != "2016-02-06" # 相似的,Elasticsearch也有 and, or, not這樣的組合條件的查詢方式 # 格式以下: # { # "bool" : { # "must" : [], # "should" : [], # "must_not" : [], # } # } # # must: 條件必須知足,至關於 and # should: 條件能夠知足也能夠不知足,至關於 or # must_not: 條件不須要知足,至關於 not curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query" : { "bool" : { "should" : [ { "term" : {"price" : 35.99}}, { "term" : {"price" : 99.99}} ], "must_not" : { "term" : {"publish_date" : "2016-02-06"} } } } }' # 嵌套查詢 # SELECT * FROM books WHERE price = 35.99 OR ( publish_date = "2016-02-06" AND price = 99.99 ) curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query": { "bool": { "should": [ { "term": { "price": 35.99 } }, { "bool": { "must": [ { "term": { "publish_date": "2016-02-06" } }, { "term": { "price": 99.99 } } ] } } ] } } }' # range範圍過濾 # SELECT * FROM books WHERE price >= 10 AND price < 99 # gt : > 大於 # lt : < 小於 # gte : >= 大於等於 # lte : <= 小於等於 curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query": { "range" : { "price" : { "gte" : 10, "lt" : 99 } } } } #name和author都必須包含Guide,而且價錢等於33.99或者188.99 curl -XGET 'http://192.168.10.16:9200/store/books/_search' -d '{ "query": { "bool": { "must": { "multi_match": { "operator": "and", "fields": [ "name", "author" ], "query": "Guide" } }, "filter": { "terms": { "price": [ 35.99, 188.99 ] } } } } }' http://192.168.10.16:9200/store/books/_search
二:Head插件安裝與使用html
只須要在一臺機器上安裝便可node
head 插件安裝linux
http://blog.csdn.net/napoay/article/details/53896348 #更新 sudo yum update -y sudo rpm -ivh http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm sudo rpm -ivh http://dl.fedoraproject.org/pub/epel/epel-release-6-8.noarch.rpm sudo rpm -ivh https://kojipkgs.fedoraproject.org//packages/http-parser/2.7.1/3.el7/x86_64/http-parser-2.7.1-3.el7.x86_64.rpm sudo yum install npm sudo yum install -y git sudo yum install -y bzip2 git clone git://github.com/mobz/elasticsearch-head.git #將源碼包下載後剪切到/bigdata目錄,並改所屬用戶和組 sudo chown -R xiaoniu:xiaoniu /bigdata/elasticsearch-head #進入到elasticsearch-head中 cd elasticsearch-head #編譯安裝 npm install 打開elasticsearch-head-master/Gruntfile.js,找到下面connect屬性,新增hostname: '0.0.0.0', connect: { server: { options: { hostname: '0.0.0.0', port: 9100, base: '.', keepalive: true } } } 編輯elasticsearch-5.4.3/config/elasticsearch.yml,加入如下內容: http.cors.enabled: true http.cors.allow-origin: "*" -------------先啓動ES再啓動插件---------------- #運行服務 必定要在插件目錄下執行 npm run start --------------------------------------------------------------------------------------------- 關閉ES集羣 kill `ps -ef | grep Elasticsearch | grep -v grep | awk '{ print $2}'` 安裝IK分詞器 下載對應版本的插件 https://github.com/medcl/elasticsearch-analysis-ik/releases 首先下載es對應版本的ik分詞器的zip包,上傳到es服務器上,在es的安裝目錄下有一個plugins的目錄,在這個目錄下建立一個叫ik的目錄 而後將解壓好的內容,拷貝到ik目錄 將ik目錄拷貝到其餘的es節點 從新啓動全部的es #建立索引名字叫news curl -XPUT http://192.168.100.211:9200/news #建立mapping(至關於數據中的schema信息,表名和字段名以及字段的類型) curl -XPOST http://192.168.100.211:9200/news/fulltext/_mapping -d' { "properties": { "content": { "type": "text", "analyzer": "ik_max_word", "search_analyzer": "ik_max_word" } } }' curl -XPOST http://192.168.100.211:9200/news/fulltext/1 -d' {"content":"美國留給伊拉克的是個爛攤子嗎"}' curl -XPOST http://192.168.100.211:9200/news/fulltext/2 -d' {"content":"公安部:各地校車將享最高路權"}' curl -XPOST http://192.168.100.211:9200/news/fulltext/3 -d' {"content":"中韓漁警衝突調查:韓警平均天天扣1艘中國漁船"}' curl -XPOST http://192.168.100.211:9200/news/fulltext/4 -d' {"content":"中國駐洛杉磯領事館遭亞裔男子槍擊 嫌犯已自首"}' curl -XPOST http://192.168.100.211:9200/news/fulltext/_search -d' { "query" : { "match" : { "content" : "中國" }}, "highlight" : { "pre_tags" : ["<font color='red'>", "<tag2>"], "post_tags" : ["</font>", "</tag2>"], "fields" : { "content" : {} } } }' ------------------------------------------------------------------- curl -XGET 'http://192.168.100.211:9200/_analyze?pretty&analyzer=ik_max_word' -d '聯想是全球最大的筆記本廠商' curl -XGET 'https://192.168.100.211:9200/_analyze?pretty&analyzer=ik_smart' -d '聯想是全球最大的筆記本廠商' curl -XPUT 'https://192.168.100.211:9200/iktest?pretty' -d '{ "settings" : { "analysis" : { "analyzer" : { "ik" : { "tokenizer" : "ik_max_word" } } } }, "mappings" : { "article" : { "dynamic" : true, "properties" : { "subject" : { "type" : "string", "analyzer" : "ik_max_word" } } } } }' curl -XPUT 'https://192.168.100.211:9200/iktest?pretty' -d '{ "settings" : { "analysis" : { "analyzer" : { "ik" : { "tokenizer" : "ik_max_word" } } } }, "mappings" : { "article" : { "dynamic" : true, "properties" : { "subject" : { "type" : "string", "analyzer" : "ik_max_word" } } } } }' curl -XGET 'http://192.168.10.16:9200/_analyze?pretty&analyzer=ik_max_word' -d ‘中華人民共和國’ --------------------------------------------------------------------------------------------- es安裝SQL插件 ./bin/elasticsearch-plugin install https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.3.0/elasticsearch-sql-5.4.3.0.zip #而後將解壓到plugins目錄下的內容拷貝到其餘es的節點的plugins目錄 下載SQL的Server wget https://github.com/NLPchina/elasticsearch-sql/releases/download/5.4.1.0/es-sql-site-standalone.zip 用npm編譯安裝 unzip es-sql-site-standalone.zip cd site-server/ npm install express --save 修改SQL的Server的端口 vi site_configuration.json 啓動服務 node node-server.js &
四:遇到 的問題c++
1:解決CentOS缺乏共享庫:libstdc++.so.6git
當在CentOS 6.2下執行某些命令時,有缺乏共享庫的報錯: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory 解決辦法: 1、執行命令: yum whatprovides libstdc++.so.6 而後會提示哪一個安裝包有這個庫文件以下: [root@localhost ~]# yum whatprovides libstdc++.so.6 Loaded plugins: aliases, changelog, downloadonly, fastestmirror, kabi, presto, refresh-packagekit, security, tmprepo, verify, : versionlock Loading support for CentOS kernel ABI Loading mirror speeds from cached hostfile * base: centos.ustc.edu.cn * centosplus: centos.ustc.edu.cn * contrib: centos.ustc.edu.cn * extras: centos.ustc.edu.cn * updates: centos.ustc.edu.cn libstdc++-4.4.7-3.el6.i686 : GNU Standard C++ Library Repo : base Matched from: Other : libstdc++.so.6 2、而後執行: yum install libstdc++-4.4.7-3.el6.i686 本篇文章來源於 Linux公社網站(www.linuxidc.com) 原文連接:https://www.linuxidc.com/Linux/2013-04/82494.htm
2:Centos6.5 升級glibc解決「libc.so.6: version GLIBC_2.14 not found」報錯問題github
從上面報錯能夠看出,程序運行時候,沒有找到「GLIBC_2.14」這個版本庫,而默認的Centos6.5 glibc版本最高爲2.12, 因此須要更新系統glibc庫。sql
繼續完成後續的安裝: 數據庫
接下來當你創建新的軟連接時候,會發現ln命令不能用了。express
收集整理用的,用到的原文連接太多了,找不回來了