ElasticSearch 安裝 go-mysql-elasticsearch 同步mysql的數據

1、首先在Centos6.5上安裝 go 語言環境

　　下載Golang語言包：https://studygolang.com/dl

1 [hoojjack@localhost src]$ ls
2 apache-maven-3.5.0  go  go1.6.2.linux-amd64.tar.gz

2、解壓.tar.gz安裝包。

 

3、配置安裝環境。

[hoojjack@localhost etc]$ vim profile
export GOPATH=/usr/local/src/go/work
export GOROOT=/usr/local/src/go
export PATH=$GOPATH/bin:$PATH

 

GOARCH（指定系統環境，i386表示x86，amd64表示x64）：amd64

GOROOT：/usr/local/src/go（go的解壓路徑）

GOBIN：%GOROOT%/bin（exe執行文件路徑）

GOOS：（go運行的系統）

GOPATH：/usr/local/src/go/work（用於存放Go語言Package的目錄，這個目錄不能在Go的安裝目錄中，放了好像也沒什麼事，至少我目前沒事 ^_^）

咱們須要添加的環境變量分別是go的安裝路徑GOROOT：/usr/local/src/go，PATH：$GOROOT/bin:$PATH，go的工做路徑GOPATH:/usr/local/src/go/work

 

4、配置go環境後配置go-mysql-elasticsearch

首先確保環境中安裝了git

安裝依賴包：

sudo yum install gettext-devel openssl-devel perl-CPAN perl-devel zlib-devel

也能夠直接用yum命令安裝：

   yum install git

而後：

獲取安裝包：go get github.com/siddontang/go-mysql-elasticsearch

安裝：

         cd   $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
         make

1 [hoojjack@localhost etc]$ cd /usr/local/src/go/src/github.com/siddontang/go-mysql-elasticsearch
2 [hoojjack@localhost go-mysql-elasticsearch]$ ls
3 bin  cmd  elastic  etc  Godeps  LICENSE  Makefile  README.md  river  var
4 [hoojjack@localhost go-mysql-elasticsearch]$ 

 

5、修改配置文件

 1 # MySQL address, user and password
 2 # user must have replication privilege in MySQL.
 3 my_addr = "127.0.0.1:3306"     //須要同步的mysql基本設置
 4 my_user = "root"
 5 my_pass = "root"
 6 
 7 # Elasticsearch address
 8 es_addr = "127.0.0.1:9200"     //本地elasticsearch配置
 9 
10 # Path to store data, like master.info, and dump MySQL data 
11 data_dir = "./var"             //數據存儲的url
12  //如下配置保存默認不變
13 # Inner Http status address
14 stat_addr = "127.0.0.1:12800"
15 
16 # pseudo server id like a slave 
17 server_id = 1001
18 
19 # mysql or mariadb
20 flavor = "mysql"
21 # mysqldump execution path
22 mysqldump = "mysqldump"
23 
24 # MySQL data source
25 [[source]]
26 schema = "hoojjack"             //elasticsearch 與 mysql 同步時對應的數據庫名稱
27 
28 # Only below tables will be synced into Elasticsearch.
29 # "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
30 # I don't think it is necessary to sync all tables in a database.
31 tables = ["test_river", "test_river_[0-9]{4}"]    //支持通配符，能夠指定只複製hoojjack數據庫中指定的表數據 32 
33 # Below is for special rule mapping
34 [[rule]]
35 schema = "hoojjack"    //數據庫名稱
36 table = "test_river"   //表名稱
37 index = "river"        //對應的索引名稱
38 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
39   [rule.field]
40 # This will map column title to elastic search my_title
41   title="es_title"     //將能夠將mysql的某個屬性對應指向elasticsearch的某個field, 如test_river的titile屬性對應es_title
42 # This will map column tags to elastic search my_tags and use array type
43   tags="my_tags,list"
44 # This will map column keywords to elastic search keywords and use array type
45   keywords=",list"
46 
47 # wildcard table rule, the wildcard table must be in source tables 
48   [[rule]]
49   schema = "hoojjack"
50   table = "test_river_[0-9]{4}"
51   index = "river"
52   type = "river"
53 
54 # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
55   [[rule.fields]]
56   mysql = "title"
57   elastic = "es_title"

6、執行go-mysql-elasticsearch同步，首先進入 go-mysql-elasticsearch 路徑，而後執行 ： ./bin/go-mysql-elasticsearch -config=./etc/river.toml

1 cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
2 ./bin/go-mysql-elasticsearch -config=./etc/river.toml

效果以下：

mysql> select * from test_river;
+----+-----------+-----------------+--------+
| id | name      | title           | number |
+----+-----------+-----------------+--------+
|  1 | hoojjack  | wo shi yi bing  |      1 |
|  2 | xunchuan  | ni shi er ren   |      2 |

7、出現的問題：

 一、mysqldump: command not found

mysql的mysqldump命令沒有找到，解決辦法：建立軟鏈接，查看mysqldump是在哪一個路徑下：find / -name mysqldump   會得到路徑，而後建立軟鏈接：ln -s /usr/local/mysql/bin/mysqldump  /usr/bin 

這是因爲系統默認會查找/usr/bin下的命令，若是這個命令不在這個目錄下，固然會找不到命令

若是沒有安裝mysql，則能夠利用yum安裝mysql5.6

1）檢查系統是否安裝其餘版本的MYSQL數據

#yum list installed | grep mysql
#yum -y remove mysql-libs.x86_64

2）安裝及配置

# wget http://repo.mysql.com/mysql-community-release-el6-5.noarch.rpm
# rpm -ivh mysql-community-release-el6-5.noarch.rpm
# yum repolist all | grep mysql

3）安裝MYSQL數據庫

yum install mysql-community-server -y

 

 

二、make build go-mysql-elasticsearch項目的時候出現問題（can’t load package: package .: no buildable Go source files in ）

 網上有不少的解決方法，可是我都沒看懂，惟一看懂的大概就是要升級下go版本，由於以前go是1.6的，索性嘗試升級到1.10，後面問題解決了（人品大爆棚），可是問題的核心沒有解決。啊啊。。

【Reference】

[1] http://blog.csdn.net/laoyang360/article/details/51771483