使用Elasticsearch-jdbc爲MySQL數據庫創建索引

elasticsearch-jdbc

環境

• Ubuntu 14.04
• JDK 1.8.0_66
• Elasticsearch 2.3.1
• Elasticsearch-jdbc 2.3.1.0
• Elasticsearch單節點環境

進入es目錄~/cluster/elasticsearch-2.3.1

下載elasticsearch-jdbc包，並解壓

$ wget http://xbib.org/repository/org/xbib/elasticsearch/importer/elasticsearch-jdbc/2.3.1.0/elasticsearch-jdbc-2.3.1.0-dist.zip
$ unzip elasticsearch-jdbc-2.3.1.0-dist.zip

數據庫中的數據

數據位於10.110.1.47:3306下的ispider_data數據庫，表名爲es_test
共三條數據以下：

id   name
4   zhangsan
2   lisi
3   wangwu

編輯數據導入腳本import.sh

vi import.sh
輸入：

bin=/home/es/cluster/elasticsearch-2.3.1/elasticsearch-jdbc-2.3.1.0/bin
lib=/home/es/cluster/elasticsearch-2.3.1/elasticsearch-jdbc-2.3.1.0/lib
echo '{
"type" : "jdbc",
"jdbc": {
"url":"jdbc:mysql://10.110.1.47:3306/ispider_data",
"user":"root",
"password":"123456a?",
"sql":"select * from es_test",
"index" : "customer",
"type" : "external"
}}' | java \
    -cp "${lib}/*" \
    -Dlog4j.configurationFile=${bin}/log4j2.xml \
    org.xbib.tools.Runner \
    org.xbib.tools.JDBCImporter

上述，將MySQL中的數據創建爲es中索引爲customer，類型爲external中。

查詢索引

es@search1:~/cluster/elasticsearch-2.3.1$ curl 'localhost:9200/customer/external/_search?pretty&q=*'

結果顯示：

{
  "took" : 6,
  "timed_out" : false,
  "_shards" : {
    "total" : 5,
    "successful" : 5,
    "failed" : 0
  },
  "hits" : {
    "total" : 3,
    "max_score" : 1.0,
    "hits" : [ {
      "_index" : "customer",
      "_type" : "external",
      "_id" : "AVQ8--xAkvh0m5n1OUEo",
      "_score" : 1.0,
      "_source" : {
        "id" : "4",
        "name" : "zhangsan"
      }
    }, {
      "_index" : "customer",
      "_type" : "external",
      "_id" : "AVQ8--xBkvh0m5n1OUEq",
      "_score" : 1.0,
      "_source" : {
        "id" : "3",
        "name" : "wangwu"
      }
    }, {
      "_index" : "customer",
      "_type" : "external",
      "_id" : "AVQ8--xBkvh0m5n1OUEp",
      "_score" : 1.0,
      "_source" : {
        "id" : "2",
        "name" : "lisi"
      }
    } ]
  }
}

可見，_source中的各個字段field，與數據庫中的各列對應。
注意，_source中的id是es_test表中的id列，索引中document的id是自動生成的，兩者並不同。
至此，從數據庫MySQL導入ES創建索引的過程就完成了。

參考資料

http://www.voidcn.com/blog/kdchxue/article/p-5778237.html