CDH5動靜態資源池配置與回滾

關於動態 靜態資源池的配置之前都有提過，能夠從如下幾篇瞭解：

YARN動態資源池配置案例 https://yq.aliyun.com/ziliao/346856#

Hadoop YARN配置參數剖析(4)—Fair Scheduler相關參數

Hadoop YARN配置參數剖析(5)—Capacity Scheduler相關參數

Hadoop YARN配置參數剖析(5)—Capacity Scheduler相關參數

Hadoop 2.0中Capacity Scheduler與Fair Scheduler對比

Hadoop權限管理

而若是不小配錯了想回滾怎麼辦？？？

動態池還好說重設刷新資源狀態就好， 靜態的能夠參考以下：

 



解決辦法：
英文好的就看原文吧：

1. Disabling Static Service Pools
2.  
3. To disable static service pools, disable cgroup-based resource management for all hosts in all clusters:
4.  
5. 1.In the main navigation bar, click Hosts.
6. 2.Click the Configuration tab.
7. 3.Click the Resource Management category, uncheck the Enable Cgroup-based Resource Management property, then click Save Changes.
8. 4.Restart all services. 
9. Static resource management is disabled, but the percentages you set when you configured the pools, and all the changed settings (for example, heap sizes), are retained by the services. The percentages and settings will also be used when you re-enable static service pools. If you want to revert to the settings you had before static service pools were enabled, follow the procedures in Viewing and Reverting Configuration Changes.

複製代碼

譯文：不當之處多多指正
中止靜態服務池

中止靜態服務池，中止集羣上全部主機的resource management

1.在主導航欄上，單擊「主機。
2.單擊「配置」選項卡。
3.單擊Resource Management，取消Cgroup-based資源管理屬性，而後保存改變。
4.重啓全部服務
靜態資源管理是禁用的，可是配置的靜態服務池，以及其它改變的設置（例如heap size ）,已被服務保存。當你從新啓用靜態服務池，這些百分比及設置都會生效。若是你想恢復到以前的配置，按照下面的程序，恢復改變便可

 

靜態池配置重啓後遇到的坑：

spark  yarn-client 模式報錯：

ERROR spark.SparkContext: Error initializing SparkContext.
org.apache.spark.SparkException: Yarn application has already ended! It might have been killed or unable to launch application master.

yarn log:

Application application_1547969490810_0012 failed 2 times due to AM Container for appattempt_1547969490810_0012_000002 exited with exitCode: -1000

For more detailed output, check application tracking page:http://master:8088/proxy/application_1547969490810_0012/Then, click on links to logs of each attempt.

Diagnostics: Application application_1547969490810_0012 initialization failed (exitCode=255) with output: main : command provided 0

main : run as user is nobody

main : requested yarn user is hdfs

Can't create directory /data/yarn/nm/usercache/hdfs/appcache/application_1547969490810_0012 - Permission denied

Did not create any app directories

Failing this attempt. Failing the application.

解決方案：

1.檢查 spark_home 

2.檢查hdfs下spakr相關jar

3.網上說清除yarn相關usercache:  rm -rf /data/yarn/nm/usercache/*

我查了看了下目錄權限 變了，各個節點改回來試了不行，仍是刪了

ll /data/yarn/nm/usercache/
總用量 32
drwxr-s---. 4 nobody yarn 1024 9月  29 20:10 hdfs
drwxr-x---. 4 yarn   yarn 1024 10月 24 15:45 hue_ro
drwxr-x---. 4 yarn   yarn 1024 1月  16 15:32 root
drwxr-x---. 4 yarn   yarn 1024 1月  16 14:00 test

chown yarn:yarn /data/yarn/nm/usercache/hdfs