做業要求 | https://edu.cnblogs.com/campus/fzzcxy/2018SE1/homework/11250 |
---|---|
做業目標 | <從雲班課上爬取經驗值數據,計算經驗平均值、最高值、最低值,並根據學號(升序)、經驗值(降序)列出學生列表> |
做業源代碼 | https://gitee.com/huang-cunhui/team-work_200928 |
黃存慧 | <211806385> |
程仕 | <211806312> |
211806385,黃存慧,喜歡象棋、書法,偶爾運動,喜歡寫前端代碼和java代碼php
211806312,程仕,喜歡國學名著,看文藝電視劇,敲代碼的一把好手
html
1.爬取數據(直接爬取網站數據,網站需登陸)
前端
2.分析數據
java
3.輸出數據
git
代碼行數 | 195行 |
---|---|
編寫時間 | 8個小時 |
分析時間 | 3.5個小時 |
>>點擊跳轉碼雲地址<<
web
1.配置config.properties文件:將URL地址和COOKIE信息寫入配置文件編程
url=https://www.mosoteach.cn/web/index.php?c=interaction&m=index&clazz_course_id=9E603F91-4AF8-11EA-9C7F-98039B1848C6 cookie=_uab_collina=158873178646017777712456;login_token=c9f6dc99114336166bd5ac5cec1ecba062ee2232fadca5bd46e1e0514e70ddee;teachweb=e85e83a51f1847bd068f22c9e437501053217534; SERVERID=75616bcd1ea8ec157e112381bf1eec35|1601810787|1601810140
網頁爬蟲
2.在Team類中引入配置文件,建立Properties類對象propcookie
// 導入配置信息:獲取properties文件中的url和cookie Properties prop = new Properties(); try { prop.load(new FileInputStream("resources/config.properties")); // 文件位置:相對路徑引入 } catch (Exception e) { e.printStackTrace(); // 出錯處理 }
3.建立列表和document對象,經過配置信息的url和cookie解析目標網頁,獲取數據eclipse
// 建立列表 ArrayList<String> baseList = new ArrayList<String>(); // 建立document文件,經過url和cookie引入html網頁的數據 Document document = (Document) Jsoup.connect(prop.getProperty("url")).header("cookie", prop.getProperty("cookie")).get(); if (document != null) { // 經過interaction-row類獲取數據 Elements activDivs = ((Element) document).getElementsByClass("interaction-row"); for (int i = 0; i < activDivs.size(); i++) { if (activDivs.get(i).toString().contains("課堂完成")) { String urlString = activDivs.get(i).attr("data-url").toString(); baseList.add(urlString); } } } else { // url和cookie正常,但未獲取到網頁 System.out.println("出錯啦!"); }
4.列表賦值,同時將數據寫入文件
// 列表賦值 ArrayList<Student> newStudentList=stuList(baseList); Collections.sort(newStudentList, new Student()); // 數據寫入文件 write(newStudentList);
5.建立stuList方法,處理學生信息
public static ArrayList<Student> stuList(ArrayList<String> baseList) throws IOException { // 獲取另外一個配置信息的對象prop1 Properties prop1 = new Properties(); try { prop1.load(new FileInputStream("resources/config.properties")); } catch (Exception e) { e.printStackTrace(); } // 列表 ArrayList<Document> baseActives = new ArrayList<Document>(baseList.size()); ArrayList<Student> studentList = new ArrayList<>(); for (int i = 0; i < baseList.size(); i++) { Document document1 = (Document) Jsoup.connect(baseList.get(i)).header("cookie", prop1.getProperty("cookie")).get(); baseActives.add(document1); } for (int j = 0; j < baseActives.size(); j++) { Elements baseStuDivs = ((Element) baseActives.get(j)).getElementsByClass("homework-item"); for (int k = 0; k < baseStuDivs.size(); k++) { try { Student stu = new Student(); stu.setId(baseStuDivs.get(k).child(0).child(1).child(1).text().toString()); stu.setName(baseStuDivs.get(k).child(0).child(1).child(0).text().toString()); String score = baseStuDivs.get(k).child(3).child(1).child(1).text(); // 未評分的學生 if (baseStuDivs.get(k).child(3).child(0).child(1).text().contains("尚無評分")) { stu.setScore(0.0); } // 未提交的學生 else if (baseStuDivs.get(k).child(1).child(0).text().contains("未提交")) { stu.setScore(0.0); } else { stu.setScore(Double.parseDouble(score.substring(0, score.length() - 2))); } studentList.add(stu); } catch (Exception e) { } } ArrayList<Student> newStudentList = new ArrayList<>(); Double totalScore; for (int i = 0; i < studentList.size(); i++) { totalScore = 0.0; Student student = new Student(); for (int j = i + 1; j < studentList.size(); j++) { if (studentList.get(i).getName().contains(studentList.get(j).getName())) { totalScore += studentList.get(j).getScore(); studentList.remove(j); } } student.setId(studentList.get(i).getId()); student.setName(studentList.get(i).getName()); student.setScore(totalScore); newStudentList.add(student); } return newStudentList; } }
6.建立write方法,按照要求輸出計算的數據
public static void write(ArrayList<Student> newStudentList) throws FileNotFoundException { File file = new File("score.txt"); PrintWriter printWriter = new PrintWriter(new FileOutputStream(file), true); double ave = 0.0; for (int j = 0; j < newStudentList.size(); j++) { ave += newStudentList.get(j).getScore(); } ave = ave / newStudentList.size(); printWriter.println("最高經驗值" + newStudentList.get(0).getScore() + ",最低經驗值" + newStudentList.get(newStudentList.size() - 1).getScore() + ",平均經驗值" + ave); for (int i = 0; i < newStudentList.size(); i++) { printWriter.println(newStudentList.get(i).toString()); } printWriter.close(); }
7.建立學生類,代碼完成
import java.util.Comparator; public class Student implements Comparator<Student>{ public String id; public String name; public double score; public Student() { super(); } public Student(String id, String name, double score) { super(); this.id = id; this.name = name; this.score = score; } public String getId() { return id; } public void setId(String id) { this.id = id; } public String getName() { return name; } public void setName(String name) { this.name = name; } public double getScore() { return score; } public void setScore(double score) { this.score = score; } @Override public String toString() { return id + "," + name + "," + score; } @Override public int compare(Student o1, Student o2) { int sort1 = Integer.parseInt(o1.getId()) - Integer.parseInt(o2.getId()); int sort2 = (int) (o2.getScore() - o1.getScore()); return sort2==0?sort1:sort2; } }