YARN命令是調用bin/yarn腳本文件,若是運行yarn腳本沒有帶任何參數,則會打印yarn全部命令的描述。html
使用: yarn [--config confdir] COMMAND [--loglevel loglevel] [GENERIC_OPTIONS] [COMMAND_OPTIONS]node
YARN有一個參數解析框架,採用解析泛型參數以及運行類。web
命令參數 | 描述 |
---|---|
--config confdir |
指定一個默認的配置文件目錄,默認值是: ${HADOOP_PREFIX}/conf . |
--loglevel loglevel |
重載Log級別。有效的日誌級別包含:FATAL, ERROR, WARN, INFO, DEBUG, and TRACE。默認是INFO。 |
GENERIC_OPTIONS | YARN支持表A的通用命令項。 |
COMMAND COMMAND_OPTIONS | YARN分爲用戶命令和管理員命令。 |
表A:apache
通用項 | Description |
---|---|
-archives <comma separated list of archives> |
用逗號分隔計算中未歸檔的文件。 僅僅針對JOB。 |
-conf <configuration file> |
制定應用程序的配置文件。 |
-D <property>=<value> |
使用給定的屬性值。 |
-files <comma separated list of files> |
用逗號分隔的文件,拷貝到Map reduce機器,僅僅針對JOB |
-jt <local> or <resourcemanager:port> |
指定一個ResourceManager. 僅僅針對JOB。 |
-libjars <comma seperated list of jars> |
將用逗號分隔的jar路徑包含到classpath中去,僅僅針對JOB。 |
對於Hadoop集羣用戶頗有用的命令:服務器
使用: yarn application [options]
app
命令選項 | 描述 |
---|---|
-appStates <States> | 使用-list命令,基於應用程序的狀態來過濾應用程序。若是應用程序的狀態有多個,用逗號分隔。 有效的應用程序狀態包含 以下: ALL, NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED |
-appTypes <Types> | 使用-list命令,基於應用程序類型來過濾應用程序。若是應用程序的類型有多個,用逗號分隔。 |
-list | 從RM返回的應用程序列表,使用-appTypes參數,支持基於應用程序類型的過濾,使用-appStates參數,支持對應用程序狀態的過濾。 |
-kill <ApplicationId> | kill掉指定的應用程序。 |
-status <ApplicationId> | 打印應用程序的狀態。 |
示例1:框架
[hduser@hadoop0 bin]$ ./yarn application -list -appStates ACCEPTED 15/08/10 11:48:43 INFO client.RMProxy: Connecting to ResourceManager at hadoop1/10.0.1.41:8032 Total number of applications (application-types: [] and states: [ACCEPTED]):1 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1438998625140_1703 MAC_STATUS MAPREDUCE hduser default ACCEPTED UNDEFINED 0% N/A
示例2:webapp
[hduser@hadoop0 bin]$ ./yarn application -list 15/08/10 11:43:01 INFO client.RMProxy: Connecting to ResourceManager at hadoop1/10.0.1.41:8032 Total number of applications (application-types: [] and states: [SUBMITTED, ACCEPTED, RUNNING]):1 Application-Id Application-Name Application-Type User Queue State Final-State Progress Tracking-URL application_1438998625140_1701 MAC_STATUS MAPREDUCE hduser default ACCEPTED UNDEFINED 0% N/A
示例3:ide
[hduser@hadoop0 bin]$ ./yarn application -kill application_1438998625140_1705 15/08/10 11:57:41 INFO client.RMProxy: Connecting to ResourceManager at hadoop1/10.0.1.41:8032 Killing application application_1438998625140_1705 15/08/10 11:57:42 INFO impl.YarnClientImpl: Killed application application_1438998625140_1705
applicationattempt
使用: yarn applicationattempt [options]
oop
命令選項 | 描述 |
---|---|
-help | 幫助 |
-list <ApplicationId> | 獲取到應用程序嘗試的列表,其返回值ApplicationAttempt-Id 等於 <Application Attempt Id> |
-status <Application Attempt Id> | 打印應用程序嘗試的狀態。 |
打印應用程序嘗試的報告。
示例1:
[hadoop@hadoopcluster78 bin]$ yarn applicationattempt -list application_1437364567082_0106 15/08/10 20:58:28 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Total number of application attempts :1 ApplicationAttempt-Id State AM-Container-Id Tracking-URL appattempt_1437364567082_0106_000001 RUNNING container_1437364567082_0106_01_000001 http://hadoopcluster79:8088/proxy/application_1437364567082_0106/
示例2:
[hadoop@hadoopcluster78 bin]$ yarn applicationattempt -status appattempt_1437364567082_0106_000001 15/08/10 21:01:41 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Application Attempt Report : ApplicationAttempt-Id : appattempt_1437364567082_0106_000001 State : FINISHED AMContainer : container_1437364567082_0106_01_000001 Tracking-URL : http://hadoopcluster79:8088/proxy/application_1437364567082_0106/jobhistory/job/job_1437364567082_0106 RPC Port : 51911 AM Host : hadoopcluster80 Diagnostics :
使用: yarn classpath
打印須要獲得Hadoop的jar和所須要的lib包路徑
[hadoop@hadoopcluster78 bin]$ yarn classpath /home/hadoop/apache/hadoop-2.4.1/etc/hadoop:/home/hadoop/apache/hadoop-2.4.1/etc/hadoop:/home/hadoop/apache/hadoop-2.4.1/etc/hadoop:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/common/lib/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/common/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/hdfs:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/hdfs/lib/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/hdfs/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/yarn/lib/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/yarn/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/mapreduce/lib/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/mapreduce/*:/home/hadoop/apache/hadoop-2.4.1/contrib/capacity-scheduler/*.jar:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/yarn/*:/home/hadoop/apache/hadoop-2.4.1/share/hadoop/yarn/lib/*
使用: yarn container [options]
命令選項 | 描述 |
---|---|
-help | 幫助 |
-list <Application Attempt Id> | 應用程序嘗試的Containers列表 |
-status <ContainerId> | 打印Container的狀態 |
打印container(s)的報告
示例1:
[hadoop@hadoopcluster78 bin]$ yarn container -list appattempt_1437364567082_0106_01 15/08/10 20:45:45 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Total number of containers :25 Container-Id Start Time Finish Time State Host LOG-URL container_1437364567082_0106_01_000028 1439210458659 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000028/hadoop container_1437364567082_0106_01_000016 1439210314436 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000016/hadoop container_1437364567082_0106_01_000019 1439210338598 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000019/hadoop container_1437364567082_0106_01_000004 1439210314130 0 RUNNING hadoopcluster82:48622 //hadoopcluster82:8042/node/containerlogs/container_1437364567082_0106_01_000004/hadoop container_1437364567082_0106_01_000008 1439210314130 0 RUNNING hadoopcluster82:48622 //hadoopcluster82:8042/node/containerlogs/container_1437364567082_0106_01_000008/hadoop container_1437364567082_0106_01_000031 1439210718604 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000031/hadoop container_1437364567082_0106_01_000020 1439210339601 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000020/hadoop container_1437364567082_0106_01_000005 1439210314130 0 RUNNING hadoopcluster82:48622 //hadoopcluster82:8042/node/containerlogs/container_1437364567082_0106_01_000005/hadoop container_1437364567082_0106_01_000013 1439210314435 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000013/hadoop container_1437364567082_0106_01_000022 1439210368679 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000022/hadoop container_1437364567082_0106_01_000021 1439210353626 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000021/hadoop container_1437364567082_0106_01_000014 1439210314435 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000014/hadoop container_1437364567082_0106_01_000029 1439210473726 0 RUNNING hadoopcluster80:42366 //hadoopcluster80:8042/node/containerlogs/container_1437364567082_0106_01_000029/hadoop container_1437364567082_0106_01_000006 1439210314130 0 RUNNING hadoopcluster82:48622 //hadoopcluster82:8042/node/containerlogs/container_1437364567082_0106_01_000006/hadoop container_1437364567082_0106_01_000003 1439210314129 0 RUNNING hadoopcluster82:48622 //hadoopcluster82:8042/node/containerlogs/container_1437364567082_0106_01_000003/hadoop container_1437364567082_0106_01_000015 1439210314436 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000015/hadoop container_1437364567082_0106_01_000009 1439210314130 0 RUNNING hadoopcluster82:48622 //hadoopcluster82:8042/node/containerlogs/container_1437364567082_0106_01_000009/hadoop container_1437364567082_0106_01_000030 1439210708467 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000030/hadoop container_1437364567082_0106_01_000012 1439210314435 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000012/hadoop container_1437364567082_0106_01_000027 1439210444354 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000027/hadoop container_1437364567082_0106_01_000026 1439210428514 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000026/hadoop container_1437364567082_0106_01_000017 1439210314436 0 RUNNING hadoopcluster84:43818 //hadoopcluster84:8042/node/containerlogs/container_1437364567082_0106_01_000017/hadoop container_1437364567082_0106_01_000001 1439210306902 0 RUNNING hadoopcluster80:42366 //hadoopcluster80:8042/node/containerlogs/container_1437364567082_0106_01_000001/hadoop container_1437364567082_0106_01_000002 1439210314129 0 RUNNING hadoopcluster82:48622 //hadoopcluster82:8042/node/containerlogs/container_1437364567082_0106_01_000002/hadoop container_1437364567082_0106_01_000025 1439210414171 0 RUNNING hadoopcluster83:37140 //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0106_01_000025/hadoop
示例2:
[hadoop@hadoopcluster78 bin]$ yarn container -status container_1437364567082_0105_01_000020 15/08/10 20:28:00 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Container Report : Container-Id : container_1437364567082_0105_01_000020 Start-Time : 1439208779842 Finish-Time : 0 State : RUNNING LOG-URL : //hadoopcluster83:8042/node/containerlogs/container_1437364567082_0105_01_000020/hadoop Host : hadoopcluster83:37140 Diagnostics : null
使用: yarn jar <jar> [mainClass] args...
運行jar文件,用戶能夠將寫好的YARN代碼打包成jar文件,用這個命令去運行它。
使用: yarn logs -applicationId <application ID> [options]
注:應用程序沒有完成,該命令是不能打印日誌的。
命令選項 | 描述 |
---|---|
-applicationId <application ID> | 指定應用程序ID,應用程序的ID能夠在yarn.resourcemanager.webapp.address配置的路徑查看(即:ID) |
-appOwner <AppOwner> | 應用的全部者(若是沒有指定就是當前用戶)應用程序的ID能夠在yarn.resourcemanager.webapp.address配置的路徑查看(即:User) |
-containerId <ContainerId> | Container Id |
-help | 幫助 |
-nodeAddress <NodeAddress> | 節點地址的格式:nodename:port (端口是配置文件中:yarn.nodemanager.webapp.address參數指定) |
轉存container的日誌。
示例:
[hadoop@hadoopcluster78 bin]$ yarn logs -applicationId application_1437364567082_0104 -appOwner hadoop 15/08/10 17:59:19 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Container: container_1437364567082_0104_01_000003 on hadoopcluster82_48622 ============================================================================ LogType: stderr LogLength: 0 Log Contents: LogType: stdout LogLength: 0 Log Contents: LogType: syslog LogLength: 3673 Log Contents: 2015-08-10 17:24:01,565 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.retry.interval; Ignoring. 2015-08-10 17:24:01,580 WARN [main] org.apache.hadoop.conf.Configuration: job.xml:an attempt to override final parameter: mapreduce.job.end-notification.max.attempts; Ignoring. 。。。。。。此處省略N萬個字符 // 下面的命令,根據APP的全部者查看LOG日誌,由於application_1437364567082_0104任務我是用hadoop用戶啓動的,因此打印的是以下信息: [hadoop@hadoopcluster78 bin]$ yarn logs -applicationId application_1437364567082_0104 -appOwner root 15/08/10 17:59:25 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Logs not available at /tmp/logs/root/logs/application_1437364567082_0104 Log aggregation has not completed or is not enabled.
使用: yarn node [options]
命令選項 | 描述 |
---|---|
-all | 全部的節點,無論是什麼狀態的。 |
-list | 列出全部RUNNING狀態的節點。支持-states選項過濾指定的狀態,節點的狀態包 含:NEW,RUNNING,UNHEALTHY,DECOMMISSIONED,LOST,REBOOTED。支持--all顯示全部的節點。 |
-states <States> | 和-list配合使用,用逗號分隔節點狀態,只顯示這些狀態的節點信息。 |
-status <NodeId> | 打印指定節點的狀態。 |
示例1:
[hadoop@hadoopcluster78 bin]$ ./yarn node -list -all 15/08/10 17:34:17 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Total Nodes:4 Node-Id Node-State Node-Http-Address Number-of-Running-Containers hadoopcluster82:48622 RUNNING hadoopcluster82:8042 0 hadoopcluster84:43818 RUNNING hadoopcluster84:8042 0 hadoopcluster83:37140 RUNNING hadoopcluster83:8042 0 hadoopcluster80:42366 RUNNING hadoopcluster80:8042 0
示例2:
[hadoop@hadoopcluster78 bin]$ ./yarn node -list -states RUNNING 15/08/10 17:39:55 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Total Nodes:4 Node-Id Node-State Node-Http-Address Number-of-Running-Containers hadoopcluster82:48622 RUNNING hadoopcluster82:8042 0 hadoopcluster84:43818 RUNNING hadoopcluster84:8042 0 hadoopcluster83:37140 RUNNING hadoopcluster83:8042 0 hadoopcluster80:42366 RUNNING hadoopcluster80:8042 0
示例3:
[hadoop@hadoopcluster78 bin]$ ./yarn node -status hadoopcluster82:48622 15/08/10 17:52:52 INFO client.RMProxy: Connecting to ResourceManager at hadoopcluster79/10.0.1.79:8032 Node Report : Node-Id : hadoopcluster82:48622 Rack : /default-rack Node-State : RUNNING Node-Http-Address : hadoopcluster82:8042 Last-Health-Update : 星期一 10/八月/15 05:52:09:601CST Health-Report : Containers : 0 Memory-Used : 0MB Memory-Capacity : 10240MB CPU-Used : 0 vcores CPU-Capacity : 8 vcores
打印節點的報告。
使用: yarn queue [options]
命令選項 | 描述 |
---|---|
-help | 幫助 |
-status <QueueName> | 打印隊列的狀態 |
打印隊列信息。
使用: yarn version
打印hadoop的版本。
下列這些命令對hadoop集羣的管理員是很是有用的。
使用:
yarn daemonlog -getlevel <host:httpport> <classname> yarn daemonlog -setlevel <host:httpport> <classname> <level>
參數選項 | 描述 |
---|---|
-getlevel <host:httpport><classname> | 打印運行在<host:port>的守護進程的日誌級別。這個命令內部會鏈接http://<host:port>/logLevel?log=<name> |
-setlevel <host:httpport> <classname> <level> | 設置運行在<host:port>的守護進程的日誌級別。這個命令內部會鏈接http://<host:port>/logLevel?log=<name> |
針對指定的守護進程,獲取/設置日誌級別.
示例1:
[root@hadoopcluster78 ~]# hadoop daemonlog -getlevel hadoopcluster82:50075 org.apache.hadoop.hdfs.server.datanode.DataNode Connecting to http://hadoopcluster82:50075/logLevel?log=org.apache.hadoop.hdfs.server.datanode.DataNode Submitted Log Name: org.apache.hadoop.hdfs.server.datanode.DataNode Log Class: org.apache.commons.logging.impl.Log4JLogger Effective level: INFO [root@hadoopcluster78 ~]# yarn daemonlog -getlevel hadoopcluster79:8088 org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Connecting to http://hadoopcluster79:8088/logLevel?log=org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Submitted Log Name: org.apache.hadoop.yarn.server.resourcemanager.rmapp.RMAppImpl Log Class: org.apache.commons.logging.impl.Log4JLogger Effective level: INFO [root@hadoopcluster78 ~]# yarn daemonlog -getlevel hadoopcluster78:19888 org.apache.hadoop.mapreduce.v2.hs.JobHistory Connecting to http://hadoopcluster78:19888/logLevel?log=org.apache.hadoop.mapreduce.v2.hs.JobHistory Submitted Log Name: org.apache.hadoop.mapreduce.v2.hs.JobHistory Log Class: org.apache.commons.logging.impl.Log4JLogger Effective level: INFO
使用: yarn nodemanager
啓動NodeManager
使用: yarn proxyserver
啓動web proxy server
使用: yarn resourcemanager [-format-state-store]
參數選項 | 描述 |
---|---|
-format-state-store | RMStateStore的格式. 若是過去的應用程序再也不須要,則清理RMStateStore, RMStateStore僅僅在ResourceManager沒有運行的時候,才運行RMStateStore |
啓動ResourceManager
使用:
yarn rmadmin [-refreshQueues] [-refreshNodes] [-refreshUserToGroupsMapping] [-refreshSuperUserGroupsConfiguration] [-refreshAdminAcls] [-refreshServiceAcl] [-getGroups [username]] [-transitionToActive [--forceactive] [--forcemanual] <serviceId>] [-transitionToStandby [--forcemanual] <serviceId>] [-failover [--forcefence] [--forceactive] <serviceId1> <serviceId2>] [-getServiceState <serviceId>] [-checkHealth <serviceId>] [-help [cmd]]
參數選項 | 描述 |
---|---|
-refreshQueues | 重載隊列的ACL,狀態和調度器特定的屬性,ResourceManager將重載mapred-queues配置文件 |
-refreshNodes | 動態刷新dfs.hosts和dfs.hosts.exclude配置,無需重啓NameNode。 dfs.hosts:列出了容許連入NameNode的datanode清單(IP或者機器名) dfs.hosts.exclude:列出了禁止連入NameNode的datanode清單(IP或者機器名) 從新讀取hosts和exclude文件,更新容許連到Namenode的或那些須要退出或入編的Datanode的集合。 |
-refreshUserToGroupsMappings | 刷新用戶到組的映射。 |
-refreshSuperUserGroupsConfiguration | 刷新用戶組的配置 |
-refreshAdminAcls | 刷新ResourceManager的ACL管理 |
-refreshServiceAcl | ResourceManager重載服務級別的受權文件。 |
-getGroups [username] | 獲取指定用戶所屬的組。 |
-transitionToActive [–forceactive] [–forcemanual] <serviceId> | 嘗試將目標服務轉爲 Active 狀態。若是使用了–forceactive選項,不須要覈對非Active節點。若是採用了自動故障轉移,這個命令不能使用。雖然你能夠重寫–forcemanual選項,你須要謹慎。 |
-transitionToStandby [–forcemanual] <serviceId> | 將服務轉爲 Standby 狀態. 若是採用了自動故障轉移,這個命令不能使用。雖然你能夠重寫–forcemanual選項,你須要謹慎。 |
-failover [–forceactive] <serviceId1> <serviceId2> | 啓動從serviceId1 到 serviceId2的故障轉移。若是使用了-forceactive選項,即便服務沒有準備,也會嘗試故障轉移到目標服務。若是採用了自動故障轉移,這個命令不能使用。 |
-getServiceState <serviceId> | 返回服務的狀態。(注:ResourceManager不是HA的時候,時不能運行該命令的) |
-checkHealth <serviceId> | 請求服務器執行健康檢查,若是檢查失敗,RMAdmin將用一個非零標示退出。(注:ResourceManager不是HA的時候,時不能運行該命令的) |
-help [cmd] | 顯示指定命令的幫助,若是沒有指定,則顯示命令的幫助。 |
使用: yarn scmadmin [options]
參數選項 | 描述 |
---|---|
-help | Help |
-runCleanerTask | Runs the cleaner task |
Runs Shared Cache Manager admin client
使用: yarn sharedcachemanager
啓動Shared Cache Manager
以前yarn運行框架只有Job history server,這是hadoop2.4版本以後加的通用Job History Server,命令爲Application Timeline Server,詳情請看:The YARN Timeline Server
使用: yarn timelineserver
啓動TimeLineServer