系統:centos7.4html
安裝scrapyd:pip isntall scrapydpython
由於我騰訊雲上是python2與python3並存的 因此我執行的命令是:pip3 isntall scrapydlinux
安裝後新建一個配置文件:web
sudo mkdir /etc/scrapydjson
sudo vim /etc/scrapyd/scrapyd.confvim
寫入以下內容:(給內容在https://scrapyd.readthedocs.io/en/stable/config.html可找到)centos
[scrapyd] eggs_dir = eggs logs_dir = logs items_dir = jobs_to_keep = 5 dbs_dir = dbs max_proc = 0 max_proc_per_cpu = 10 finished_to_keep = 100 poll_interval = 5.0 bind_address = 0.0.0.0 http_port = 6800 debug = off runner = scrapyd.runner application = scrapyd.app.application launcher = scrapyd.launcher.Launcher webroot = scrapyd.website.Root [services] schedule.json = scrapyd.webservice.Schedule cancel.json = scrapyd.webservice.Cancel addversion.json = scrapyd.webservice.AddVersion listprojects.json = scrapyd.webservice.ListProjects listversions.json = scrapyd.webservice.ListVersions listspiders.json = scrapyd.webservice.ListSpiders delproject.json = scrapyd.webservice.DeleteProject delversion.json = scrapyd.webservice.DeleteVersion listjobs.json = scrapyd.webservice.ListJobs daemonstatus.json = scrapyd.webservice.DaemonStatus
主要更改bind_address=0.0.0.0app
建立文件後執行命令啓動scrapyd: (scrapyd > /dev/null &) 當想要記錄輸出日誌時: (scrapyd > /root/scrapyd.log &)scrapy
坑1:當我執行完命令後報錯,說是找不到命令:ide
那是由於我係統上python2與3並存,因此找不到,這時應該作軟鏈接:
個人python3路徑: /usr/local/python3
製做軟鏈接: ln -s /usr/local/python3/bin/scrapy /usr/bin/scrapy
昨晚軟鏈接後,執行上邊命令,又報錯:
坑2:
這個好像是那個配置文件的最後一行有問題,具體緣由不大清楚,我將最後一行刪除,再次從新執行,scrapyd就跑起來了