《OD大數據實戰》環境整理

1、關機後服務從新啓動

1. 啓動hadoop服務

sbin/hadoop-daemon.sh start namenode
sbin/hadoop-daemon.sh start datanode
sbin/yarn-daemon.sh start resourcemanager
sbin/yarn-daemon.sh start nodemanager
sbin/mr-jobhistory-daemon.sh start historyserver
sbin/hadoop-daemon.sh start secondarynamenode

sbin/httpfs.sh start

2. 啓動hive服務

nohup hive --service metastore > ~/hive_metastore.start.log 2>&1 &
nohup hive --service hiveserver2 > ~/hiveserver2.start.log 2>&1 &

3. 啓動zookeeper服務

bin/zkServer.sh start

4. 啓動kafka服務

nohup bin/kafka-server-start.sh config/server.properties >~/kafka-start.log 2>&1 &
nohup bin/kafka-server-start.sh config/server1.properties >~/kafka-server1-start.log 2>&1 &
nohup bin/kafka-server-start.sh config/server2.properties >~/kafka-server2-start.log 2>&1 &

5. 啓動oozie服務

bin/oozied.sh start 

6. 啓動hbase服務

bin/start-hbase.sh

bin/hbase-daemon.sh start thrift

7. 啓動hue服務

nohup build/env/bin/supervisor >~/hue-start.log 2>&1 &

8. 啓動mongodb服務

nohup bin/mongod --config bin/mongodb.conf >~/mongodb-start.log 2>&1 &

9. 啓動storm

nohup bin/storm nimbus >~/storm-nimbus-start.log 2>&1 &
nohup bin/storm supervisor >~/storm-supervisor-start.log 2>&1 &
nohup bin/storm ui >~/storm-ui-start.log 2>&1 &
nohup bin/storm logviewer >~/storm-logviewer-start.log 2>&1 &

10. 啓動spark

sbin/start-master.sh
sbin/start-slaves.sh

99. 大數據UI服務

http://beifeng-hadoop-02:50070/dfshealth.html#tab-overview
http://beifeng-hadoop-02:8088/cluster
http://beifeng-hadoop-02:19888/
http://beifeng-hadoop-02:50090/status.html
http://beifeng-hadoop-02:11000/oozie/
http://beifeng-hadoop-02:60010/master-status
http://beifeng-hadoop-02:8888/
http://beifeng-hadoop-02:8081/

 

2、大數據相關服務及端口

服務名	默認端口
namenode	9000
datanode	50010/50075/50020
secondarynamenode	50090
resourcemanager	8088
nodemanager
historyserver
zookeeper	2181
oozie	11000
mongodb	27017
mongodb http	28017
storm ui	8081
spark ui	4040

 

3、 環境檢查命令

jps

ps -ef | grep processname

netstat -tlnup | grep port

lsof –i : port

netstat –nltp | grep pid

pkill -9 java