NO.2 安裝配置

時間 2019-11-06

標籤 no.2 安裝配置简体版

原文原文鏈接

檢測當前系統下的jdk安裝狀況：

[root@Centos 桌面]# rpm -qa | grep java

tzdata-java-2012j-1.el6.noarch

java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.x86_64

java-1.6.0-openjdk-1.6.0.0-1.50.1.11.5.el6_3.x86_64

卸載當前全部的jdk：

[root@Centos 桌面]# rpm -e --nodeps tzdata-java-2012j-1.el6.noarch

[root@Centos 桌面]# rpm -e --nodeps java-1.7.0-openjdk-1.7.0.9-2.3.4.1.el6_3.x86_64

[root@Centos 桌面]# rpm -e --nodeps java-1.6.0-openjdk-1.6.0.0-1.50.1.11.5.el6_3.x86_64

複查是否卸載徹底：無輸出則卸載乾淨：

[root@Centos 桌面]# rpm -qa | grep java

[root@Centos 桌面]#

安裝本身下載的jdk並配置環境變量，詳盡參考「linux學習/NO.7」

······················································· JDK安裝完成······························································

······················································· hadoop的安裝開始··················································

1.在hadoop的conf目錄下配置 hadoop-env.sh core-site.xml hdfs-site.xml mapred-site.xml ，

首先解壓hadoop.

1.1 在hadoop-env.sh裏的配置hadoop的JDK環境

---------------------------------------------

[root@Centos ~]# cd hadoop-1.2.1/

[root@Centos hadoop-1.2.1]# cd conf

[root@Centos conf]# vi hadoop-env.sh

---------------------------------------------

配置信息以下：

export JAVA_HOME=/root/jdk1.8.0_65

1.2 在core-site.xml裏的配置hadoop的HDFS地址及端口號

------------------------------------------------

[root@Centos conf]# vi core-site.xml

------------------------------------------------

配置信息以下：

<name>fs.default.name</name>

<value>hdfs://localhost:9000</value>

</property>

</configuration>

1.3 在hdfs-site.xml裏的配置hadoop的HDFS的配置

-------------------------------------------------

[root@Centos conf]# vi hdfs-site.xml

-------------------------------------------------

配置信息以下：

<name>dfs.replication</name>

</property>

</configuration>

1.4 在mapred-site.xml裏的配置hadoop的HDFS的配置

-------------------------------------------------

[root@Centos conf]# vi mapred-site.xml

--------------------------------------------

配置信息以下：

<name>mapred.job.tracker</name>

<value>localhost:9001</value>

</property>

</configuration>

2.ssh免密碼登陸

--------------------------------------------------------------------

[root@Centos conf]# cd /root

[root@Centos ~]# ssh-keygen -t rsa

輸出：

Generating public/private rsa key pair.

Enter file in which to save the key (/root/.ssh/id_rsa):

Enter passphrase (empty for no passphrase):

Enter same passphrase again:

Your identification has been saved in /root/.ssh/id_rsa.

Your public key has been saved in /root/.ssh/id_rsa.pub.

The key fingerprint is:

ed:48:64:29:62:37:c1:e9:3d:84:bf:ad:4e:50:5e:66 root@Centos

The key's randomart image is:

+--[ RSA 2048]----+

| ..o |

| +... |

| o.++= E |

| . o.B+= |

| . S+. |

| o.o. |

| o.. |

| .. |

+-----------------+

c[root@Centos ~]# cd .ssh

[root@Centos .ssh]# ls

id_rsa id_rsa.pub

[root@Centos .ssh]# cp id_rsa.pub authorized_keys

[root@Centos .ssh]# ls

authorized_keys id_rsa id_rsa.pub

[root@Centos .ssh]# ssh localhost

The authenticity of host 'localhost (::1)' can't be established.

RSA key fingerprint is 3f:84:db:2f:53:a9:09:a6:61:a2:3a:82:80:6c:af:1a.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'localhost' (RSA) to the list of known hosts.

-------------------------------------------------------------------------------

驗證免密碼登陸

-------------------------------------------------------------------------------

[root@Centos ~]# ssh localhost

Last login: Sun Apr 3 23:19:51 2016 from localhost

[root@Centos ~]# ssh localhost

Last login: Sun Apr 3 23:20:12 2016 from localhost

Connection to localhost closed.

[root@Centos ~]#

----------------------------SSH免密碼登陸設置成功----------------------------

格式化HDFS：

[root@Centos ~]# cd /root/hadoop-1.2.1/

[root@Centos hadoop-1.2.1]# bin/hadoop namenode -format

輸出：

16/04/03 23:24:12 INFO namenode.NameNode: STARTUP_MSG:

/************************************************************

STARTUP_MSG: Starting NameNode

STARTUP_MSG: host = java.net.UnknownHostException: Centos: Centos: unknown error

STARTUP_MSG: args = [-format]

STARTUP_MSG: version = 1.2.1

STARTUP_MSG: build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013

STARTUP_MSG: java = 1.8.0_65

************************************************************/

16/04/03 23:24:13 INFO util.GSet: Computing capacity for map BlocksMap

16/04/03 23:24:13 INFO util.GSet: VM type = 64-bit

16/04/03 23:24:13 INFO util.GSet: 2.0% max memory = 1013645312

16/04/03 23:24:13 INFO util.GSet: capacity = 2^21 = 2097152 entries

16/04/03 23:24:13 INFO util.GSet: recommended=2097152, actual=2097152

16/04/03 23:24:15 INFO namenode.FSNamesystem: fsOwner=root

16/04/03 23:24:15 INFO namenode.FSNamesystem: supergroup=supergroup

16/04/03 23:24:15 INFO namenode.FSNamesystem: isPermissionEnabled=true

16/04/03 23:24:15 INFO namenode.FSNamesystem: dfs.block.invalidate.limit=100

16/04/03 23:24:15 INFO namenode.FSNamesystem: isAccessTokenEnabled=false accessKeyUpdateInterval=0 min(s), accessTokenLifetime=0 min(s)

16/04/03 23:24:15 INFO namenode.FSEditLog: dfs.namenode.edits.toleration.length = 0

16/04/03 23:24:15 INFO namenode.NameNode: Caching file names occuring more than 10 times

16/04/03 23:24:17 INFO common.Storage: Image file /tmp/hadoop-root/dfs/name/current/fsimage of size 110 bytes saved in 0 seconds.

16/04/03 23:24:18 INFO namenode.FSEditLog: closing edit log: position=4, editlog=/tmp/hadoop-root/dfs/name/current/edits

16/04/03 23:24:18 INFO namenode.FSEditLog: close success: truncate to 4, editlog=/tmp/hadoop-root/dfs/name/current/edits

16/04/03 23:24:18 INFO common.Storage: Storage directory /tmp/hadoop-root/dfs/name has been successfully formatted.

16/04/03 23:24:18 INFO namenode.NameNode: SHUTDOWN_MSG:

/************************************************************

SHUTDOWN_MSG: Shutting down NameNode at java.net.UnknownHostException: Centos: Centos: unknown error

************************************************************/

-----------------------------------------------------------------------------

此時格式化節點報錯：Centos: unknown error--------緊接着下一步配置

--------------------------------------------------------------------------

[root@Centos hadoop-1.2.1]# vi /etc/hosts

配置信息以下：

127.0.0.1 localhost Centos

-------------------------------------------------------------------------

再一次進行格式化

--------------------------------------------------------------------------

[root@Centos hadoop-1.2.1]# vi /etc/hosts

[root@Centos hadoop-1.2.1]# bin/hadoop namenode -format

具體狀況以下圖：

---------------------------namenode格式化成功------------------------------

啓動hadoop ：

關閉防火牆命令行 # service iptables stop

啓動hadoop集羣命令行 # start-all.sh

關閉hadoop集羣命令行 # stop-all.sh

關閉防火牆： # service iptables stop

啓動hadoop集羣：

[root@Centos hadoop-1.2.1]# bin/start-all.sh

輸出：

starting namenode, logging to /root/hadoop-1.2.1/libexec/../logs/hadoop-root-namenode-Centos.out

localhost: starting datanode, logging to /root/hadoop-1.2.1/libexec/../logs/hadoop-root-datanode-Centos.out

localhost: starting secondarynamenode, logging to /root/hadoop-1.2.1/libexec/../logs/hadoop-root-secondarynamenode-Centos.out

starting jobtracker, logging to /root/hadoop-1.2.1/libexec/../logs/hadoop-root-jobtracker-Centos.out

localhost: starting tasktracker, logging to /root/hadoop-1.2.1/libexec/../logs/hadoop-root-tasktracker-Centos.out

驗證集羣是否正常啓動----5個節點在列表中則啓動成功

再次驗證啓動項目

[root@Centos hadoop-1.2.1]# cd /root/mahout-distribution-0.6/

[root@Centos mahout-distribution-0.6]# jps

30692 SecondaryNameNode

30437 NameNode

31382 Jps

30903 TaskTracker

30775 JobTracker

30553 DataNode

[root@Centos mahout-distribution-0.6]# jps

30692 SecondaryNameNode

31477 Jps

30437 NameNode

30903 TaskTracker

30775 JobTracker

30553 DataNode

[root@Centos mahout-distribution-0.6]# cd /home/hadoop-1.2.1

關閉hadoop集羣

[root@Centos hadoop-1.2.1]# bin/stop-all.sh

stopping jobtracker

localhost: stopping tasktracker

stopping namenode

localhost: stopping datanode

localhost: stopping secondarynamenode

[root@Centos hadoop-1.2.1]#

------------------------hadoop僞分佈式安裝成功------------------------

mahout的安裝

1.解壓安裝mahout ：解壓到hadoop目錄中

[root@Centos hadoop-1.2.1]# tar zxvf mahout-distribution-0.6.tar.gz

2.配置環境變量

export HADOOP_HOME=/root/hadoop-1.2.1

export HADOOP_CONF_DIR=/root/hadoop-1.2.1/conf

export MAHOUT_HOME=/root/hadoop-1.2.1/mahoutdistribution-0.6

export MAHOUT_CONF_DIR=/root/hadoop-1.2.1/mahoutdistribution-0.6/conf

export PATH=$PATH:$MAHOUT_HOME/conf:$MAHOUT_HOME/bin

3.測試mahout的啓動

配置環境變量過程以下圖：

--------------------------------------mahout安裝成------------------------------------java

相關標籤/搜索

每日一句

每一个你不满意的现在，都有一个你没有努力的曾经。