利用 Laravel 花 2 小時擼一個 RSS 生成器

時間 2020-01-08

標籤利用 laravel 小時一個 rss 成器欄目 RSS 简体版

原文原文鏈接

Wait no longer! Create RSS feeds for all websites you care about and read them from the comfort of your feed reader.

如今愈來愈多的網站都不支持 RSS 訂閱了，而做爲 RSS 的忠實粉絲，仍是但願有個工具能夠將本身關注的網站內容聚合在一塊兒，而後實時推送到手機上，及時獲取最新消息和新聞動態。php

因此今天就讓咱們用 2 個小時，擼一個 RSS 生成器。html

本文的主角仍然是 Laravel。node

1. 搭建 Laravel 骨架

因爲須要有一個後臺，添加咱們關注的網站，因此咱們仍是沿用 laravel-damin 插件。laravel

// 1. 建立 Laravel 5.5版本項目

composer create-project --prefer-dist laravel/laravel:5.5 lrss

cd lrss

cp .env.example .env

php artisan key:generate

// 2. 使用 laravel-admin 插件
composer require encore/laravel-admin "1.5.*"

php artisan vendor:publish --provider="Encore\Admin\AdminServiceProvider"

php artisan admin:install

注：如出現問題：SQLSTATE[42000]: Syntax error or access violation: 1071 Specified key was too long; max key length is 767 bytes git

解決方案：在 AppServiceProvider.php 加入默認字符串長度github

use Illuminate\Support\Facades\Schema;

public function boot()
{
    Schema::defaultStringLength(191);
}

剛巧，咱們想借助 Symfonys 提供的 DomCrawler 插件，來解析網站的 xpath 信息，發現 laravel-admin插件有引入：web

2. 解析 XPath

以前有想借助 huginn 這個神器來生成咱們的 RSS Feed，主要參看文章：讓全部網頁變成RSS —— Huginn http://git.huginn.cn/docs/%E8%AE%A9%E6%89%80%E6%9C%89%E7%BD%91%E9%A1%B5%E5%8F%98%E6%88%90RSS%E2%80%94%E2%80%94Huginn.htmljson

但在實際使用中，發現一直出現 Huginn 無端宕機，或者後臺 jobs 動不動就失敗。這纔有了本身擼個工具的想法。數組

但 Huginn 給了我靈感，能夠利用解析 XPath 來生成 RSS Feed。bash

建立 Xpath 控制器

爲了驗證輸入的 XPath 信息的準確性，咱們能夠參考 Huginn，

首先在 Huginn 測試 XPath 的效果，在建立 WebsiteAgent 界面，輸入以下信息：

{
  "expected_update_period_in_days": "2",
  "url": "http://www.woshipm.com/",
  "type": "html",
  "mode": "on_change",
  "extract": {
    "title": {
      "xpath": "//div[@class=\"postlist-item u-clearfix\"]/div[2]/h2/a/text()",
      "value": "normalize-space(.)"
    },
    "desc": {
      "xpath": 
"//div[@class=\"postlist-item u-clearfix\"]/div[2]/p/text()",
      "value": "normalize-space(.)"
    },
    "url": {
      "xpath": "//div[@class=\"postlist-item u-clearfix\"]/div[2]/h2/a",
      "value": "@href"
    }
  }
}

而後點「Dry Run」便可測試：

最後根據 Huginn 填入的信息，咱們來建立 Xpath Controller

// bash
php artisan make:model Xpath -m

// migration
public function up()
{
    Schema::create('xpaths', function (Blueprint $table) {
        $table->increments('id');
        
        // url
        $table->string('url', 250);
        $table->string("urldesc", 250);
        
        // title
        $table->string('titlexpath', 250);
        $table->string('titlevalue', 100)
              ->nullable();

        // desc
        $table->string('descxpath', 250);
        $table->string('descvalue', 100)
               ->nullable();

        // url
        $table->string("preurl", 50)->nullable();
        
        $table->string('urlxpath', 250);
        $table->string('urlvalue', 100)
              ->nullable();
              
        $table->timestamps();
    });
}

// migrate
php artisan migrate

// 建立 admin/Controller
php artisan admin:make XpathController --model=App\\Xpath

// 創建 route
$router->resource('xpaths', XpathController::class);

// 加入到 admin 的 menu 中
// 略

注：能夠參考以前的文章：推薦一個 Laravel admin 後臺管理插件

CURD XPath

有了 laravel-admin 插件，操做 XPath 信息就好簡單了，直接看代碼：

<?php

namespace App\Admin\Controllers;

use App\Xpath;

use Encore\Admin\Form;
use Encore\Admin\Grid;
use Encore\Admin\Facades\Admin;
use Encore\Admin\Layout\Content;
use App\Http\Controllers\Controller;
use Encore\Admin\Controllers\ModelForm;

class XpathController extends Controller
{
    use ModelForm;

    /**
     * Index interface.
     *
     * @return Content
     */
    public function index()
    {
        return Admin::content(function (Content $content) {

            $content->header('header');
            $content->description('description');

            $content->body($this->grid());
        });
    }

    /**
     * Edit interface.
     *
     * @param $id
     * @return Content
     */
    public function edit($id)
    {
        return Admin::content(function (Content $content) use ($id) {

            $content->header('header');
            $content->description('description');

            $content->body($this->form()->edit($id));
        });
    }

    /**
     * Create interface.
     *
     * @return Content
     */
    public function create()
    {
        return Admin::content(function (Content $content) {

            $content->header('header');
            $content->description('description');

            $content->body($this->form());
        });
    }

    /**
     * Make a grid builder.
     *
     * @return Grid
     */
    protected function grid()
    {
        return Admin::grid(Xpath::class, function (Grid $grid) {

            $grid->id('ID')->sortable();

            $grid->column('url');
            $grid->column('urldesc', "描述");

            $grid->column('titlexpath');
            $grid->column('titlevalue');

            $grid->column('descxpath');
            $grid->column('descvalue');

            $grid->column('preurl');
            $grid->column('urlxpath');
            $grid->column('urlvalue');

            $grid->created_at();
            $grid->updated_at();
        });
    }

    /**
     * Make a form builder.
     *
     * @return Form
     */
    protected function form()
    {
        return Admin::form(Xpath::class, function (Form $form) {

            $form->display('id', 'ID');

            // url
            $form->text('url', '連接')
                ->placeholder('請輸入解析的網址')
                ->rules('required|min:5|max:250');

            $form->text('urldesc', '一句話描述')
                ->placeholder('一句話描述')
                ->rules('required|min:5|max:250');

            // title
            $form->divide();
            $form->text('titlexpath', 'title xpath')
                ->placeholder('請輸入標題 xpath')
                ->rules('required|min:5|max:250');

            $form->text('titlevalue', 'title value 默承認以不填')
                ->default('')
                ->rules('max:100');

            // desc
            $form->divide();
            $form->text('descxpath', 'desc xpath')
                ->placeholder('請輸入詳情 xpath')
                ->rules('required|min:5|max:250');

            $form->text('descvalue', 'desc value，默承認以不填')
                ->default('')
                ->rules('max:100');

            // url
            $form->divide();
            $form->text('preurl', 'url 前綴')
                ->placeholder('請輸入文章的url 前綴')
                ->rules('max:50');

            $form->text('urlxpath', 'url xpath')
                ->placeholder('請輸入文章的url xpath')
                ->rules('required|min:5|max:250');

            $form->text('urlvalue', 'url value 默承認以不填')
                ->default('')
                ->rules('max:100');

            $form->divide();
            $form->display('created_at', 'Created At');
            $form->display('updated_at', 'Updated At');
        });
    }
}

添加兩個網站信息試試：

XPath 轉爲 RSS Feed

1. 根據填入的 Xpath 信息，解析內容：

public static function analysis(XpathModel $model) {
    $html = file_get_contents($model->url);

    $crawler = new Crawler($html);

    $titlenodes = $crawler->filterXPath($model->titlexpath);
    $titles = self::getValueByNodes($titlenodes, $model->titlevalue);

    $descnodes = $crawler->filterXPath($model->descxpath);
    $desces = self::getValueByNodes($descnodes, $model->descvalue);

    $urlnodes = $crawler->filterXPath($model->urlxpath);
    $urls = self::getValueByNodes($urlnodes, $model->urlvalue);

    return RssFeeds::feeds($model, $titles, $desces, $urls);
}

// 經過規則獲取 nodes 的值
public static function getValueByNodes(Crawler $crawler, $key = null) {
    return $crawler->each(function (Crawler $node) use ($key) {
        if (empty($key)) {
            return trim($node->text());
        } else {
            return $node->attr($key);
        }
    });
}

2. 將得到 title、desc 和 url 數組裝入 Feed Item 中，構建 RSS。

public static function feeds(Xpath $xpath, $titles = [], $desces = [], $urls = []) {
    if (!empty($xpath->preurl)) {
        $preurl = $xpath->preurl;
        $urlss = collect($urls)->map(function ($url, $key) use ($preurl) {
            return $preurl.trim($url);
        });
    } else {
        $urlss = collect($urls);
    }
    return response()
        ->view('rss',
        [
            'xpath' => $xpath,
            'titles' => $titles,
            'desces' => $desces,
            'urls' => $urlss->toArray(),
            'pubDate' => Carbon::now()
        ])
        ->header('Content-Type', 'text/xml');
}

3. 編寫 blade 模板

<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
   <channel>
       <title>{{ $xpath->url or ' title' }}</title>
       <description>{{ $xpath->urldesc or '描述' }}</description>
       <link>{{ $xpath->url }}</link>
       <atom:link href="{{ url("/feed/$xpath->id") }}" rel="self" type="application/rss+xml"/>
       <pubDate>{{ $pubDate }}</pubDate>
       <lastBuildDate>{{ $pubDate }}</lastBuildDate>
       <generator>coding01</generator>
       @foreach ($titles as $key => $title)
           <item>
               <title>{{ $title }}</title>
               <link>{{ $urls[$key] }}</link>
               <description>{{ $desces[$key] }}</description>
               <pubDate>{{ $pubDate }}</pubDate>
               <author>coding01</author>
               <guid>{{ $urls[$key] }}</guid>
               <category>{{ $title }}</category>
           </item>
       @endforeach
   </channel>
</rss>

4. 最終來看看效果吧，爲每個網站作成一個 RSS：