爬取 wallhaven圖片到本地壁紙庫

項目地址，另外知乎同名文章也是我發佈的，你們能夠多多關注

首先觀察控制檯

其次再看本地壁紙庫

如今進入正題，這個小項目用到了 Jsoup具體版本見 POM），另外還用到了 JDK中的線程池、阻塞隊列（生產-消費者模式）、NIO2（文件監聽服務 API），因此至少要求 JDK版本爲7或者以上

項目分爲5個類和一個方法入口類

生產者類（任務：從列表頁拿到詳情頁連接並放入阻塞隊列）

public class Producer implements Runnable {

    private String name;
    private BlockingQueue<String> blockingQueue;

    public Producer(String name, BlockingQueue<String> blockingQueue) {
        this.name = name;
        this.blockingQueue = blockingQueue;
    }

    @Override
    public void run() {
        Document doc = null;
        try {
            for(int i = 1; i < 12018; i ++) {
                System.out.println();
                System.out.println();
                System.out.println("current page:" + i);
                System.out.println("-----------------------------------");
                if(i == 1) {
                    doc = Jsoup.connect("https://alpha.wallhaven.cc/latest").get();
                } else {
                    doc = Jsoup.connect("https://alpha.wallhaven.cc/latest?page=" + i).get();
                }
                Element div = doc.getElementById("thumbs");
                Elements sections = div.getElementsByTag("section");
                for (Element ele : sections) {
                    Elements links = ele.getElementsByClass("preview");
                    for (Element e : links) {
                        String href = e.attr("href");
                        blockingQueue.put(href);
                        System.out.println(name + " put " + href);
                    }
                }
            }
            blockingQueue.put("");
            System.out.println(name + " is over");
        } catch (IOException | InterruptedException e) {
            e.printStackTrace();
        } 
    }
}

消費者類（任務：從隊列拿到連接並獲取圖片源地址並將下載任務交給一個緩存線程池）

public class Consumer implements Runnable {

    private String name;
    private BlockingQueue<String> blockingQueue;
    private ExecutorService taskPool;

    public Consumer(String name, BlockingQueue<String> blockingQueue, ExecutorService taskPool) {
        this.name = name;
        this.blockingQueue = blockingQueue;
        this.taskPool = taskPool;
    }

    @Override
    public void run() {
        Document doc = null;
        try {
            String href = null;
            while((href = blockingQueue.take()) != "") {
                System.out.println(name + " take " + href);
                doc = Jsoup.connect(href).get();
                Element img = doc.getElementById("wallpaper");
                String src = "https:" + img.attr("src");
                taskPool.submit(new DownloadTask(src));
            }
            System.out.println(name + " is over");
        } catch (IOException | InterruptedException e) {
            e.printStackTrace();
        } 
    }

}

下載任務執行類（任務：下載圖片到本地）

public class DownloadTask implements Runnable {

    private static String path = "C:\\Users\\baiyapeng\\Desktop\\Paper\\";
    private String src;
    private String name;

    public DownloadTask(String src) {
        this.src = src;
        int n = src.lastIndexOf("/");
        this.name = src.substring(++n);
    }

    @Override
    public void run() {
        Response res = null;
        try {
            res = Jsoup.connect(src).ignoreContentType(true).timeout(30000).execute();
            byte[] bytes = res.bodyAsBytes();
            File file = new File(path + name);
            if (!file.exists()) {
                RandomAccessFile raf = new RandomAccessFile(file, "rw");
                raf.write(bytes);
                raf.close();
            }
        } catch (IOException e1) {
            e1.printStackTrace();
        }
    }

}

監聽服務類（任務：將文件路徑註冊到監聽服務上並開始監聽）

public class ResourceListener {

    private static ExecutorService fixedThreadPool = Executors.newCachedThreadPool();

    private WatchService ws;

    private ResourceListener(String path) {
        try {
            ws = FileSystems.getDefault().newWatchService();
            start();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    private void start() {
        fixedThreadPool.execute(new Listener(ws));
    }

    public static void addListener(String path) {
        try {
            ResourceListener resourceListener = new ResourceListener(path);
            Path p = Paths.get(path);
            p.register(resourceListener.ws, StandardWatchEventKinds.ENTRY_CREATE);
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

監聽回調類（任務：執行回調任務）

public class Listener implements Runnable {

    private WatchService service;

    public Listener(WatchService service) {
        this.service = service;
    }

    @Override
    public void run() {
        try {
            while (true) {
                WatchKey watchKey = service.take();
                List<WatchEvent<?>> watchEvents = watchKey.pollEvents();
                for (WatchEvent<?> event : watchEvents) {
                    System.err.println(event.context() + "已下載");
                }
                watchKey.reset();
            }
        } catch (InterruptedException e) {
            e.printStackTrace();
        } 
    }
}

方法入口類

public class DownloadTaskExecutor {

    public static void main(String[] args) throws IOException {
        
        ResourceListener.addListener("C:\\Users\\baiyapeng\\Desktop\\Paper\\");
    
        BlockingQueue<String> blockingQueue = new SynchronousQueue<String>(true);
        ExecutorService proservice = Executors.newSingleThreadExecutor();
        ExecutorService conservice = Executors.newSingleThreadExecutor();
        ExecutorService taskPool = Executors.newCachedThreadPool();
        proservice.submit(new Producer("Producer", blockingQueue));
        conservice.submit(new Consumer("Consumer", blockingQueue, taskPool));
        proservice.shutdown();
        conservice.shutdown();
    }

}

最後就是設置壁紙庫並設定更換頻率

感謝你們，有問題能夠再評論區留言~~