想了解hadoop,因此就下了最新的文件,搭建相關的環境,如下爲本身在win7上搭建hadoop的相關步驟。java
一、下載hadoop,用winRAR解壓。(路徑不能有空格)node
二、安裝java,此處就不作講解了。(java路徑不能有空格)git
三、配置hadoop的環境變量HADOOP_HOME=」HADOOP的安裝目錄」github
四、在環境變量的PATH中加入HADOOP的安裝目錄/binexpress
五、修改hadoop目錄下etc/hadoop/hadoop-env.cmd文件中的JAVA_HOME變量爲當前java的安裝路徑。apache
六、下載hadoop-commin-2.2.zip,由於在windows下運行,壓縮包裏面缺乏 winutils.exe, hadoop.dll等文件,下載完成後,將要報下面的bin目錄下的全部文件所有拷貝到hadoop目錄下的bin文件夾下。windows
七、修改配置文件,如下四個文件所有在hadoop目錄下的etc/hadoop目錄下瀏覽器
修改core-site.xml,以下:app
<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>fs.defaultFS</name> <value>hdfs://localhost:9000</value> </property> </configuration>
修改hdfs-site.xml以下:less
<?xml version="1.0" encoding="UTF-8"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>file:/hadoop/data/dfs/namenode</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>file:/hadoop/data/dfs/datanode</value> </property> </configuration>
修改yarn-site.xml以下:
<?xml version="1.0"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> </configuration>
修改mapred-site.xml以下:
<?xml version="1.0"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> <!-- Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License. See accompanying LICENSE file. --> <!-- Put site-specific property overrides in this file. --> <configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>
而後打開cmd,運行hadoop namenode -format命令,運行結果基本以下:
Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. C:\Users\abhijitg>cd c:\hadoop\bin c:\hadoop\bin>hdfs namenode -format 13/11/03 18:07:47 INFO namenode.NameNode: STARTUP_MSG: /************************************************************ STARTUP_MSG: Starting NameNode STARTUP_MSG: host = ABHIJITG/x.x.x.x STARTUP_MSG: args = [-format] STARTUP_MSG: version = 2.2.0 STARTUP_MSG: classpath = <classpath jars here> STARTUP_MSG: build = Unknown -r Unknown; compiled by ABHIJITG on 2013-11-01T13:42Z STARTUP_MSG: java = 1.7.0_03 ************************************************************/ Formatting using clusterid: CID-1af0bd9f-efee-4d4e-9f03-a0032c22e5eb 13/11/03 18:07:48 INFO namenode.HostFileManager: read includes: HostSet( ) 13/11/03 18:07:48 INFO namenode.HostFileManager: read excludes: HostSet( ) 13/11/03 18:07:48 INFO blockmanagement.DatanodeManager: dfs.block.invalidate.limit=1000 13/11/03 18:07:48 INFO util.GSet: Computing capacity for map BlocksMap 13/11/03 18:07:48 INFO util.GSet: VM type = 64-bit 13/11/03 18:07:48 INFO util.GSet: 2.0% max memory = 888.9 MB 13/11/03 18:07:48 INFO util.GSet: capacity = 2^21 = 2097152 entries 13/11/03 18:07:48 INFO blockmanagement.BlockManager: dfs.block.access.token.enable=false 13/11/03 18:07:48 INFO blockmanagement.BlockManager: defaultReplication = 1 13/11/03 18:07:48 INFO blockmanagement.BlockManager: maxReplication = 512 13/11/03 18:07:48 INFO blockmanagement.BlockManager: minReplication = 1 13/11/03 18:07:48 INFO blockmanagement.BlockManager: maxReplicationStreams = 2 13/11/03 18:07:48 INFO blockmanagement.BlockManager: shouldCheckForEnoughRacks = false 13/11/03 18:07:48 INFO blockmanagement.BlockManager: replicationRecheckInterval = 3000 13/11/03 18:07:48 INFO blockmanagement.BlockManager: encryptDataTransfer = false 13/11/03 18:07:48 INFO namenode.FSNamesystem: fsOwner = ABHIJITG (auth:SIMPLE) 13/11/03 18:07:48 INFO namenode.FSNamesystem: supergroup = supergroup 13/11/03 18:07:48 INFO namenode.FSNamesystem: isPermissionEnabled = true 13/11/03 18:07:48 INFO namenode.FSNamesystem: HA Enabled: false 13/11/03 18:07:48 INFO namenode.FSNamesystem: Append Enabled: true 13/11/03 18:07:49 INFO util.GSet: Computing capacity for map INodeMap 13/11/03 18:07:49 INFO util.GSet: VM type = 64-bit 13/11/03 18:07:49 INFO util.GSet: 1.0% max memory = 888.9 MB 13/11/03 18:07:49 INFO util.GSet: capacity = 2^20 = 1048576 entries 13/11/03 18:07:49 INFO namenode.NameNode: Caching file names occuring more than 10 times 13/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.threshold-pct = 0.9990000128746033 13/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.min.datanodes = 0 13/11/03 18:07:49 INFO namenode.FSNamesystem: dfs.namenode.safemode.extension = 30000 13/11/03 18:07:49 INFO namenode.FSNamesystem: Retry cache on namenode is enabled 13/11/03 18:07:49 INFO namenode.FSNamesystem: Retry cache will use 0.03 of total heap and retry cache entry expiry time is 600000 millis 13/11/03 18:07:49 INFO util.GSet: Computing capacity for map Namenode Retry Cache 13/11/03 18:07:49 INFO util.GSet: VM type = 64-bit 13/11/03 18:07:49 INFO util.GSet: 0.029999999329447746% max memory = 888.9 MB 13/11/03 18:07:49 INFO util.GSet: capacity = 2^15 = 32768 entries 13/11/03 18:07:49 INFO common.Storage: Storage directory \hadoop\data\dfs\namenode has been successfully formatted. 13/11/03 18:07:49 INFO namenode.FSImage: Saving image file \hadoop\data\dfs\namenode\current\fsimage.ckpt_00000000000000 00000 using no compression 13/11/03 18:07:49 INFO namenode.FSImage: Image file \hadoop\data\dfs\namenode\current\fsimage.ckpt_0000000000000000000 o f size 200 bytes saved in 0 seconds. 13/11/03 18:07:49 INFO namenode.NNStorageRetentionManager: Going to retain 1 images with txid >= 0 13/11/03 18:07:49 INFO util.ExitUtil: Exiting with status 0 13/11/03 18:07:49 INFO namenode.NameNode: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down NameNode at ABHIJITG/x.x.x.x ************************************************************/
而後在cmd下切換目錄到hadoop目錄下的sbin目錄下,運行start-all 會打開四個cmd窗口,能夠打開瀏覽器輸入 http://localhost:8042以及http://localhost:50070查看是否配置成功!
若是50070打不開,就從新格式化,而後重啓hadoop就能夠了