spark跑YARN模式或Client模式提交任務不成功（application state: ACCEPTED）

時間 2019-12-20

標籤 spark yarn 模式 client 提交任務不成功 application state accepted 欄目 Spark 简体版

原文原文鏈接

　　很少說，直接上乾貨！html

　　問題詳情java

　電腦8G，目前搭建3節點的spark集羣，採用YARN模式。node

master分配2G，slave1分配1G,slave2分配1G。（在安裝虛擬機時）linux

export SPARK_WORKER_MERMORY=1g （在spark-env.sh）apache

export JAVA_HOME=/usr/local/jdk/jdk1.8.0_60         （必須寫）
export SCALA_HOME=/usr/local/scala/scala-2.10.5    （必須寫）
export HADOOP_HOME=/usr/local/hadoop/hadoop-2.6.0    （必須寫）
export HADOOP_CONF_DIR=/usr/local/hadoop/hadoop-2.6.0/etc/hadoop   （必須寫）
export SPARK_MASTER_IP=192.168.80.10　　
export SPARK_WORKER_MERMORY=1G     （官網上說，至少1g）

　其實這個問題解決辦法很簡單，就是3個節點的是，你的內存最好仍是大點。如master分配4G，slave1分配2G,slave2分配2G！！！！（儘量的大）微信

固然，不少博友們，跟個人狀況是同樣的，在學校階段，電腦只有8g已是最大電腦內存限制了。app

通常是因爲有多個用戶同時向集羣提交任務或一個用戶向集羣同時提交了多個任務致使Yarn資源的分配錯誤。解決這個問題，只須要更改Hadoop的配置文件：/etc/hadoop/conf/capacity-scheduler.xml，把選項：yarn.scheduler.capacity.maximum-am-resource-percent從0.1改爲0.5。顧名思義，這個選項是增長Yarn可調度的資源量，固然也能夠視具體狀況增長更多。也可見，默認狀況下，Yarn沒有將不少資源分配給任務的能力。機器學習

具體，見oop

Spark on YARN模式的安裝（spark-1.6.1-bin-hadoop2.6.tgz + hadoop-2.6.0.tar.gz）（master、slave1和slave2）（博主推薦）

[spark@master logs]$  $SPARK_HOME/bin/spark-submit  \
> --class org.apache.spark.examples.JavaSparkPi \
> --master yarn-cluster \
> --num-executors 1 \
> --driver-memory 512m \
> --executor-memory 512m \
> --executor-cores 1 \
>  /usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar





注意：

driver-memory不指定也能夠，默認使用512M
executor-memory不指定的化，默認是1G



17/04/09 17:03:55 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/04/09 17:03:55 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.80.10:8032
17/04/09 17:03:56 INFO yarn.Client: Requesting a new application from cluster with 2 NodeManagers
17/04/09 17:03:56 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)
17/04/09 17:03:56 INFO yarn.Client: Will allocate AM container, with 896 MB memory including 384 MB overhead
17/04/09 17:03:56 INFO yarn.Client: Setting up container launch context for our AM
17/04/09 17:03:56 INFO yarn.Client: Setting up the launch environment for our AM container
17/04/09 17:03:56 INFO yarn.Client: Preparing resources for our AM container
17/04/09 17:03:59 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491728358337_0001/spark-assembly-1.6.1-hadoop2.6.0.jar
17/04/09 17:04:19 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491728358337_0001/spark-examples-1.6.1-hadoop2.6.0.jar
17/04/09 17:04:49 INFO yarn.Client: Uploading resource file:/tmp/spark-d152ed1b-09ca-47c8-8457-58f7e52ff419/__spark_conf__6499474209714260387.zip -> hdfs://master:9000/user/spark/.sparkStaging/application_1491728358337_0001/__spark_conf__6499474209714260387.zip
17/04/09 17:04:50 INFO spark.SecurityManager: Changing view acls to: spark
17/04/09 17:04:50 INFO spark.SecurityManager: Changing modify acls to: spark
17/04/09 17:04:50 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark); users with modify permissions: Set(spark)
17/04/09 17:04:50 INFO yarn.Client: Submitting application 1 to ResourceManager
17/04/09 17:04:51 INFO impl.YarnClientImpl: Submitted application application_1491728358337_0001
17/04/09 17:04:52 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:52 INFO yarn.Client: 
     client token: N/A
     diagnostics: N/A
     ApplicationMaster host: N/A
     ApplicationMaster RPC port: -1
     queue: default
     start time: 1491728691236
     final status: UNDEFINED
     tracking URL: http://master:8088/proxy/application_1491728358337_0001/
     user: spark
17/04/09 17:04:53 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:54 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:55 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:56 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:57 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:58 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:04:59 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:00 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:01 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:02 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:03 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:04 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:05 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:06 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
17/04/09 17:05:07 INFO yarn.Client: Application report for application_1491728358337_0001 (state: ACCEPTED)
^C17/04/09 17:05:08 INFO util.ShutdownHookManager: Shutdown hook called
17/04/09 17:05:08 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-d152ed1b-09ca-47c8-8457-58f7e52ff419

.........
而後最後失敗

　問題分析post

　　　　其實，這一狀況，是我剛開始啓動了太多的Application致使累積，使得內存超出了致使！

　解決方法

　　　　按照辦法1，辦法2，辦法3...的順序來

辦法1:（通常能解決）

　　先，

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/stop-all.sh

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/stop-all.sh

　　再，

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/start-all.sh

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/start-all.sh

　　在spark-env.sh，至少得

export SPARK_WORKER_MERMORY=1G

　　改爲

[spark@master spark-1.6.1-bin-hadoop2.6]$ $SPARK_HOME/bin/spark-submit \
--class org.apache.spark.examples.JavaSparkPi \
--master yarn-cluster \
--num-executors 1 \
--driver-memory 1g \
--executor-memory 1g \
--executor-cores 1 \
$SPARK_HOME/lib/spark-examples-1.6.1-hadoop2.6.0.jar



[spark@master spark-1.6.1-bin-hadoop2.6]$ $SPARK_HOME/bin/spark-submit --class org.apache.spark.examples.JavaSparkPi --master yarn-cluster --num-executors 1 --driver-memory 1g --executor-memory 1g --executor-cores 1 $SPARK_HOME/lib/spark-examples-1.6.1-hadoop2.6.0.jar
17/04/09 20:01:26 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/04/09 20:01:27 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.80.10:8032
17/04/09 20:01:31 INFO yarn.Client: Requesting a new application from cluster with 2 NodeManagers
17/04/09 20:01:32 INFO yarn.Client: Verifying our application has not requested more than the maximum memory capability of the cluster (8192 MB per container)
17/04/09 20:01:32 INFO yarn.Client: Will allocate AM container, with 1408 MB memory including 384 MB overhead
17/04/09 20:01:32 INFO yarn.Client: Setting up container launch context for our AM
17/04/09 20:01:32 INFO yarn.Client: Setting up the launch environment for our AM container
17/04/09 20:01:32 INFO yarn.Client: Preparing resources for our AM container
17/04/09 20:01:37 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-assembly-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491735225663_0006/spark-assembly-1.6.1-hadoop2.6.0.jar
17/04/09 20:01:57 INFO yarn.Client: Uploading resource file:/usr/local/spark/spark-1.6.1-bin-hadoop2.6/lib/spark-examples-1.6.1-hadoop2.6.0.jar -> hdfs://master:9000/user/spark/.sparkStaging/application_1491735225663_0006/spark-examples-1.6.1-hadoop2.6.0.jar
17/04/09 20:02:18 INFO yarn.Client: Uploading resource file:/tmp/spark-8d8662b1-6410-4b72-9751-57bb0e2785d0/__spark_conf__5191259430263541003.zip -> hdfs://master:9000/user/spark/.sparkStaging/application_1491735225663_0006/__spark_conf__5191259430263541003.zip
17/04/09 20:02:18 INFO spark.SecurityManager: Changing view acls to: spark
17/04/09 20:02:19 INFO spark.SecurityManager: Changing modify acls to: spark
17/04/09 20:02:19 INFO spark.SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(spark); users with modify permissions: Set(spark)
17/04/09 20:02:22 INFO yarn.Client: Submitting application 6 to ResourceManager
17/04/09 20:02:23 INFO impl.YarnClientImpl: Submitted application application_1491735225663_0006
17/04/09 20:02:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:25 INFO yarn.Client: 
     client token: N/A
     diagnostics: N/A
     ApplicationMaster host: N/A
     ApplicationMaster RPC port: -1
     queue: default
     start time: 1491739342315
     final status: UNDEFINED
     tracking URL: http://master:8088/proxy/application_1491735225663_0006/
     user: spark
17/04/09 20:02:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:35 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:37 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:42 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:48 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:49 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:50 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:51 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:53 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:54 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:55 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:56 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:02:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)

17/04/09 20:03:05 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:06 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:07 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:08 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:09 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:10 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:11 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:12 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:13 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:14 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:15 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:16 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:17 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:18 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:19 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:20 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:21 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:23 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:24 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:27 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:28 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:29 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:30 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:31 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:33 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:34 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:40 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:41 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:42 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:48 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:50 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:51 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:52 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:53 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:54 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:55 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:56 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:03:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:03 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:05 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:06 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:07 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:08 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:09 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:10 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:11 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:12 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:13 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:15 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:16 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:17 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:18 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:19 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:20 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:21 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:22 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:23 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:24 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:27 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:28 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:29 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:30 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:31 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:33 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:34 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:35 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:37 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:40 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:41 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:48 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:49 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:50 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:51 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:52 INFO yarn.Client: Application report for application_1491735225663_0006 (state: ACCEPTED)
17/04/09 20:04:53 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING) 只要最後出現這個了，就說明成功了！
17/04/09 20:04:53 INFO yarn.Client: 
     client token: N/A
     diagnostics: N/A
     ApplicationMaster host: 192.168.80.11
     ApplicationMaster RPC port: 0
     queue: default
     start time: 1491739342315
     final status: UNDEFINED
     tracking URL: http://master:8088/proxy/application_1491735225663_0006/
     user: spark
17/04/09 20:04:54 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:55 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:56 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:04:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:05 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:06 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:07 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:08 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:09 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:10 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:11 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:12 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:13 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:14 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:15 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:16 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:17 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:19 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:20 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:21 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:22 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:23 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:25 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:26 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:27 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:28 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:29 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:30 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:31 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:32 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:33 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:34 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:35 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:36 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:37 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:38 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:39 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:40 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:41 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:42 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:43 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:44 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:45 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:46 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:05:47 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
....
17/04/09 20:06:57 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:06:58 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:06:59 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:00 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:01 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:02 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:03 INFO yarn.Client: Application report for application_1491735225663_0006 (state: RUNNING)
17/04/09 20:07:04 INFO yarn.Client: Application report for application_1491735225663_0006 (state: FINISHED)
17/04/09 20:07:04 INFO yarn.Client: 
     client token: N/A
     diagnostics: N/A
     ApplicationMaster host: 192.168.80.11
     ApplicationMaster RPC port: 0
     queue: default
     start time: 1491739342315
     final status: SUCCEEDED
     tracking URL: http://master:8088/proxy/application_1491735225663_0006/A
     user: spark
17/04/09 20:07:04 INFO yarn.Client: Deleting staging directory .sparkStaging/application_1491735225663_0006
17/04/09 20:07:07 INFO util.ShutdownHookManager: Shutdown hook called
17/04/09 20:07:07 INFO util.ShutdownHookManager: Deleting directory /tmp/spark-8d8662b1-6410-4b72-9751-57bb0e2785d0
[spark@master spark-1.6.1-bin-hadoop2.6]$

　　若是以後，再次去執行別的jar包，出現又是沒法運行了，則

spark on yarn模式裏須要有時手工釋放linux內存。

解決辦法

方法2：首先想到是集羣中內存資源不足，能夠檢查下每臺機器是否有足夠剩餘內存( free -g)；也多是其餘已經提交的Spark應用佔了大部分資源；

[spark@master spark-1.6.1-bin-hadoop2.6]$ free -g
             total       used       free     shared    buffers     cached
Mem:             1          1          0          0          0          0
-/+ buffers/cache:          0          0
Swap:            1          0          1
[spark@master spark-1.6.1-bin-hadoop2.6]$

　　那麼，就要去殺死以前那些還正在運行（多餘沒用）的Application。

　　怎麼作？

　　進入你的$HADOOP_HOME/bin下

　　執行命令

[spark@master bin]$ yarn application -list

　　而後能夠看到應用的信息

　　如需殺死當前某個做業，使用kill application-id的命令以下

yarn application -kill application_1437456051228_1725

方法3：若是1>正常，咱們能夠看看YARN集羣是否啓動成功。注意「坑」可能就在這裏：即便Slave上的nodemanager進程存在，要注意檢查resource manager日誌，看看各個node manager是否啓動成功，有些人的問題就出如今這裏：進程在，可是日誌顯示node manager狀態爲UNHEALTHY，因此YARN集羣能識別到的總內存資源爲0。檢查了UNHEALTHY的緣由，是由於/tmp下一個目錄被識別爲bad, 由於是臨時目錄，把每一個node manager的對應目錄刪掉，而後重啓YARN集羣，最終問題解決。

ResourceManager日誌存放位置是${HADOOP_HOME}/logs/yarn-*-resourcemanager-*.log

NodeManager日誌存放位置是各個NodeManager節點上${HADOOP_HOME}/logs/yarn-*-nodemanager-*.log

　　先，

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/stop-all.sh

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/stop-all.sh

　　再，

[spark@master spark-1.6.1-bin-hadoop2.6]$ cd $HADOOP_HOME
[spark@master hadoop-2.6.0]$ sbin/start-all.sh

[spark@master ~]$ cd $SPARK_HOME
[spark@master spark-1.6.1-bin-hadoop2.6]$ sbin/start-all.sh

方法4：若是仍是有問題的話，只能去日誌文件中查看出錯的異常，查看更多一些的報錯信息。

　　　　　　可是，方法1和方法2是足夠能夠解決該問題的!

方法5：

　　　　除非你的內存足夠，能夠擴大內存。個人只是本身電腦最大才8G

方法6：

　　在yarn-site.xml下

<property>
          <name>yarn.nodemanager.pmem-check-enabled</name>
          <value>false</value>
</property>
<property>
          <name>yarn.nodemanager.vmem-check-enabled</name>
          <value>false</value>
</property>

　　具體見

http://dongxicheng.org/mapreduce-nextgen/hadoop-yarn-memory-cpu-scheduling/

【YARN中內存資源的調度和隔離】

【YARN中CPU資源的調度和隔離】

方法7：

運行hadoop的時候提示物理內存或虛擬內存溢出的解決方案running beyond physical memory或者beyond vitual memory limits

方法8

　　　　　　最壞的打算，就是以前有快照，回到初始位置吧！從新安裝，哎

歡迎你們，加入個人微信公衆號：大數據躺過的坑人工智能躺過的坑

同時，你們能夠關注個人我的博客：

http://www.cnblogs.com/zlslch/ 和 http://www.cnblogs.com/lchzls/ http://www.cnblogs.com/sunnyDream/

詳情請見：http://www.cnblogs.com/zlslch/p/7473861.html

　　人生苦短，我願分享。本公衆號將秉持活到老學到老學習無休止的交流分享開源精神，匯聚於互聯網和我的學習工做的精華乾貨知識，一切來於互聯網，反饋回互聯網。
　　目前研究領域：大數據、機器學習、深度學習、人工智能、數據挖掘、數據分析。語言涉及：Java、Scala、Python、Shell、Linux等。同時還涉及日常所使用的手機、電腦和互聯網上的使用技巧、問題和實用軟件。只要你一直關注和呆在羣裏，天天必須有收穫

對應本平臺的討論和答疑QQ羣：大數據和人工智能躺過的坑（總羣）（161156071）