scrapy框架異常之no more duplicates will be shown (see DUPEFILTER_DEBUG to show all duplicates)

今天在用scrapy爬蟲時,報了下面的錯誤:python 2019-01-17 16:47:18 [scrapy.dupefilters] DEBUG: Filtered duplicate request: <GET https://newhouse.fang.com/house/s/b95/> - no more duplicates will be shown (see DUPEFILTER
相關文章
相關標籤/搜索