4 datax mysql 和hbase的 相互導入

                                              mysql-->hbase

 

 

0 參考文檔：

https://github.com/alibaba/DataX/blob/master/hbase11xwriter/doc/hbase11xwriter.md

 

雖然我是：hbase1.3.3 可是仍是能夠。

 

1 配置文件

{

  "job": {

    "setting": {

      "speed": {

        "channel": 1

      }

    },

    "content": [

      {

        "reader": {

                    "name": "mysqlreader",

                    "parameter": {

                        "column": ["id","username"],

                        "connection": [

                            {

                                "jdbcUrl": ["jdbc:mysql://192.168.43.20:3306/test"],

                                "table": ["target_user"]

                            }

                        ],

                        "password": "111111",

                        "username": "root",

                        "where": ""

                    }

                },

        "writer": {

          "name": "hbase11xwriter",

          "parameter": {

            "hbaseConfig": {

              "hbase.zookeeper.quorum": "linux01:2181,linux02:2181,linux03:2181"

            },

            "table": "mysql2hbase",

            "mode": "normal",

            "rowkeyColumn": [

                {

                  "index":0,

                  "type":"string"

                },

        {

          "index":1,

          "type":"string"

        }

                {

                  "index":-1,

                  "type":"string",

                  "value":"_"

                }

            ],

            "column": [

              {

                "index":0,

                "name": "group1:id",

                "type": "string"

              },

              {

                "index":1,

                "name": "group1:name",

                "type": "string"

              },

              

            ],

            "versionColumn":{

              "index": -1,

              "value":"123456789"

            },

            "encoding": "utf-8"

          }

        }

      }

    ]

  }

}

 

 

 

2，查看mysql數據

 

3，建立一個hbase表

create "mysql2hbase","group1"

 

4，執行datax腳本

python datax.py mysql2hbase.json

 

5，hbase查看結果

 

 

    hbase-->mysql

0 參考文檔： https://github.com/alibaba/DataX/blob/master/hbase11xreader/doc/hbase11xreader.md

 

1 準備配置文件hbase2mysql.json (能夠看出，從hbase中拿出group1列簇中拿出2個字段，按照順序對應下面mysql字段)

{

    "job": {

        "content": [

            {

               "reader": {

                    "name": "hbase11xreader",

                    "parameter": {

                        "hbaseConfig": {

                            "hbase.zookeeper.quorum": "linux01:2181,linux02:2181,linux03:2181"

                        },

                        "table": "mysql2hbase",

                        "encoding": "utf-8",

                        "mode": "normal",

                        "column": [

                            {

                                "name": "group1: id",

                                "type": "string"

                            },

                            {

                                "name": "group1: name",

                                "type": "string"

                            }

                        ],

                        "range": {

                            "startRowkey": "",

                            "endRowkey": "",

                            "isBinaryRowkey": true

                        }

                    }

                },

                "writer": {

                    "name": "mysqlwriter",

                    "parameter": {

                        "column": ["id","username"],

                        "connection": [

                            {

                                "jdbcUrl": "jdbc:mysql://192.168.43.20:3306/test",

                                "table": ["target_user"]

                            }

                        ],

                        "password": "111111",

                        "preSql": [],

                        "session": [],

                        "username": "root",

                        "writeMode": "insert"

                    }

                }

            }

        ],

        "setting": {

            "speed": {

                "channel": "1"

            }

        }

    }

}

 

 

2 準備hbase數據

 

3 準備mysql表結構

 

4，執行文檔

    python datax.py  hbase2mysql.json

 

5 查看結果