laravel5.1+ElasticSearch+go-mysql-elasticsearch MySQL數據實時導入(mac)

1. ElasticSearch安裝

直接使用brew install elasticsearch 安裝最新版本的es，基本沒有障礙。

2.laravel5.1框架添加elasticsearch支持

在composer.json文件中添加elasticsearch-php依賴:

{
    "require": {
        "elasticsearch/elasticsearch": "~5.0"
    }
}

執行composer命令更新es代碼加載到當前laravel框架中

composer update

laravel控制器中使用es

use Elasticsearch\ClientBuilder;//引入
$client = ClientBuilder::create()->build();//初始化
$params = [
    'index' => 'my_index',
    'type' => 'my_type',
    'id' => 'my_id',
    'body' => ['testField' => 'abc']
];

$response = $client->index($params);//添加索引
print_r($response);

$params = [
    'index' => 'my_index',
    'type' => 'my_type',
    'id' => 'my_id'
];

$response = $client->get($params);//獲取my_id對應的數據
print_r($response);
……

實例：

<?php

namespace App\Http\Controllers;

use Illuminate\Http\Request;

use App\Http\Requests;
use App\Http\Controllers\Controller;
use Elasticsearch\ClientBuilder;



class VideoController extends Controller
{
    private $client;

    public function __construct()
    {
        $this->client = ClientBuilder::create()->build();
    }
    /**
     * Display a listing of the resource.
     *
     * @return \Illuminate\Http\Response
     */
    public function index()
    {
        for ($i=0;$i<5000;$i++) {
            $params = [
                'index' => 'my_index',
                'type' => 'my_type',
                'id' => 'my_id' . $i,
                'body' => ['testField' => $i . '、Kitty,今天氣很好!!!']
            ];
            $response = $this->client->index($params);
        }

        dd($response);
    }

    /**
     * Display the specified resource.
     *
     * @param  string  $str
     * @return \Illuminate\Http\Response
     */
    public function show($str='abc')
    {
//        $params = [
//            'index' => 'my_index',
//            'type' => 'my_type',
//            'id' => $id
//        ];
        $params = [
            'index' => 'my_index',
            'type' => 'my_type',
            //"scroll" => "3s",
            "from" => 0,
            "size" => 100,
            'body' => [
                'query' => [
                    'match' => [
                        'testField' => $str,
                    ]
                ]
            ]
        ];

        $response = $this->client->search($params);
        dd($response);
    }

}

3. 實時導入插件go-mysql-elasticsearch

安裝go環境，設置環境變量

brew install go
    vi ~/.bash_profile 添加如下PATH和GOPATH，並在命令行中執行這兩條命令
    export PATH=$PATH:/usr/local/Cellar/go/1.7.4_2/bin/
    export GOPATH=/Users/User/go

獲取go-mysql-elasticsearch

go get github.com/siddontang/go-mysql-elasticsearch

編譯go-mysql-elasticsearch

cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
make

配置elasticsearch的etc目錄下的river.toml

# MySQL address, user and password
# user must have replication privilege in MySQL.
my_addr = "192.168.0.1:3306"#mysql地址
my_user = "test"#用戶名
my_pass = "test123"#密碼

# Elasticsearch address
es_addr = "127.0.0.1:9200"#es安裝地址

# Path to store data, like master.info, and dump MySQL data
data_dir = "./var"

# Inner Http status address
stat_addr = "127.0.0.1:12800"

# pseudo server id like a slave
server_id = 1001

# mysql or mariadb
flavor = "mysql"#mysql數據庫

# mysqldump execution path
# if not set or empty, ignore mysqldump.
mysqldump = "mysqldump"

# MySQL data source
[[source]]
schema = "dev_cms"#數據庫名稱

# Only below tables will be synced into Elasticsearch.
# "test_river_[0-9]{4}" is a wildcard table format, you can use it if you have many sub tables, like table_0000 - table_1023
# I don't think it is necessary to sync all tables in a database.
tables = ["test_content"]#數據表名稱，能夠多個

# Below is for special rule mapping
#[[rule]]
#schema = "test"
#table = "test_river"
#index = "river"
#type = "river"
# only MySQL field in filter will be synced
#filter=["title","tags"]
    # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
    #[rule.field]
    # This will map column title to elastic search my_title
    #title="es_title"
    # This will map column tags to elastic search my_tags and use array type
    #tags="my_tags,list"
    # This will map column keywords to elastic search keywords and use array type
    #keywords=",list"

# wildcard table rule, the wildcard table must be in source tables
[[rule]]
schema = "dev_cms"#數據庫名稱
table = "test_content"#數據表名稱
index = "es_dev_cms"#生成es數據索引名稱，對應schema
type = "es_test_content"#生成es數據類型，對應table

    # title is MySQL test_river field name, es_title is the customized name in Elasticsearch
    [[rule.fields]]
    mysql = "title"
    elastic = "es_title"

執行命令，導入數據

cd $GOPATH/src/github.com/siddontang/go-mysql-elasticsearch 
./bin/go-mysql-elasticsearch -config=./etc/river.toml &

能夠在谷歌瀏覽器安裝sense插件查看es數據