【原創】大數據基礎之Ambari（3）經過Ambari部署Airflow

ambari2.7.3（hdp3.1） 安裝 airflow1.10

ambari的hdp中原生不支持airflow安裝，下面介紹如何經過mpack方式使ambari支持airflow安裝：

1 下載

# wget https://github.com/miho120/ambari-airflow-mpack/raw/master/airflow-service-mpack.tar.gz

2 安裝

# ambari-server install-mpack --mpack airflow-service-mpack.tar.gz --verbose

3 重啓

# ambari-server restart

4 ambari頁面沒有出現ambari，檢查mpack.json

/var/lib/ambari-server/resources/mpacks/airflow-ambari-mpack-1.5.4/mpack.json

      "service_versions_map": [

        {

          "service_name" : "AIRFLOW",

          "service_version" : "1.10.0",

          "applicable_stacks" : [

            {

              "stack_name" : "HDP",

              "stack_version" : "2.4"

            },

            {

              "stack_name" : "HDP",

              "stack_version" : "2.5"

            },

            {

              "stack_name" : "HDP",

              "stack_version" : "2.6"

            },

            {

              "stack_name" : "HDP",

              "stack_version" : "3.0"

            }

          ]

        }

      ]

須要增長HDP3.1，修改完從新打包

4 卸載

ambari-server uninstall-mpack --mpack-name=airflow-ambari-mpack

5 安裝（同上）

6 重啓（同上）

打開ambari頁面

 

啓動worker時可能失敗，查看啓動命令

2019-01-18 19:09:48,159 - Execute['service airflow-worker start'] {}

查看日誌

journalctl -xe

發現有報錯

Jan 18 19:04:05 cdp-test-server-01.bj python[15246]: detected unhandled Python exception in '/var/lib/ambari-agent/cache/common-services/AIRFLOW/1.10.0/package/scripts/airflow_worker_control.py'
Jan 18 19:04:05 cdp-test-server-01.bj abrt-server[15254]: Executable '/var/lib/ambari-agent/cache/common-services/AIRFLOW/1.10.0/package/scripts/airflow_worker_control.py' doesn't belong to any package and ProcessUnpackaged is set t
Jan 18 19:04:05 cdp-test-server-01.bj abrt-server[15254]: 'post-create' on '/var/spool/abrt/Python-2019-01-18-19:04:05-15246' exited with 1
Jan 18 19:04:05 cdp-test-server-01.bj abrt-server[15254]: Deleting problem directory '/var/spool/abrt/Python-2019-01-18-19:04:05-15246'

能夠經過修改配置解決

# grep ProcessUnpackaged /etc/abrt/abrt-action-save-package-data.conf
no

改成yes

 

 

 

ambari具體使用詳見：http://www.javashuo.com/article/p-sxfoatqc-hn.html

 

參考：

https://github.com/miho120/ambari-airflow-mpack