去年在接觸Java爬蟲的時候,接觸到了一個關於知乎的爬蟲。我的以爲寫的很是好,當時抓取的效率和成功率仍是特別特別高,如今可能知乎反扒作的更好,這個開源知乎爬蟲沒以前抓取的那麼順利了。我記得當時在個人i7+8g的機器上爬了將近兩天,大概爬取了60多w的數據。固然,實際抓取的用戶數據數量確定比這個多,只是持久化過程不一樣步而已,也就是抓取的好幾個用戶可能只有一個存入數據庫中。css
最後,本文提供的知乎網名數據是2017年12月份左右抓取的數據。html
60w知乎網名數據:java
連接:https://pan.baidu.com/s/1pEcHAbRyKP7KGcgTuZfPVA 密碼:l3somysql
項目源碼地址(若是以爲 有幫助的話,歡迎給個Star.): git
https://github.com/Snailclimb/J2ee-Advanced (Java Web進階學習的一些源碼加詳細講解)github
細心的同窗會發現,我其實只從數據庫抓取了9條數據出來。由於個人SQL語句寫錯了(逃....),你們能夠本身修改Mapper文件。web
聲明一下,筆主使用的是MyEclipse2016(主要是爲了暑假作的項目的編碼環境的統一,因此我選擇了MyEclipse2016)。面試
須要的jar包,都在這裏配置好。另外我配置了一個Tomcat插件,這樣就能夠經過Maven Build的方式來運行項目了。具體運行方式以下:ajax
右鍵項目->run as -> Maven buildredis
而後輸入tomcat7:run後點擊run便可
這裏提一點:@ResponseBody註解要把對象轉換成json格式,因此須要添加相關轉換依賴的jar包(jackson)
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> <modelVersion>4.0.0</modelVersion> <groupId>spring</groupId> <artifactId>ssm-echarts-demo</artifactId> <packaging>war</packaging> <version>0.0.1-SNAPSHOT</version> <name>ssm-echarts-demo Maven Webapp</name> <url>http://maven.apache.org</url> <properties> <webVersion>3.0</webVersion> <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding> <!-- spring版本號 --> <spring.version>4.0.0.RELEASE</spring.version> <!-- log4j日誌文件管理包版本 --> <!-- <slf4j.version>1.6.6</slf4j.version> <log4j.version>1.2.12</log4j.version> --> <!-- mybatis版本號 --> <mybatis.version>3.1.1</mybatis.version> <!-- aspectJ版本號 --> <aspectJ.version>1.7.4</aspectJ.version> <!-- 數據庫鏈接池版本號 --> <commons-dbcp.version>1.4</commons-dbcp.version> <commons-pool.version>1.6</commons-pool.version> <!-- jackson的版本號 json和xml解析 --> <jackson.version>2.8.9</jackson.version> <!-- mybatis/spring包的版本號 --> <mybatis-spring.version>1.1.1</mybatis-spring.version> <!-- mysql的版本號 --> <mysql.version>5.1.29</mysql.version> <!-- jstl的版本號 --> <jstl.version>1.2.1</jstl.version> <!-- 文件上傳用的包 的版本號 --> <commons-fileupload.version>1.3.1</commons-fileupload.version> <commons-io.version>2.4</commons-io.version> <!-- 數據校驗使用的包的版本號 --> <validation-api.version>2.0.0.Final</validation-api.version> <hibernate-validator.version>5.4.1.Final</hibernate-validator.version> </properties> <dependencies> <!-- 單元測試 --> <dependency> <groupId>junit</groupId> <artifactId>junit</artifactId> <version>4.12</version> <scope>test</scope> </dependency> <!-- --> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-test</artifactId> <version>4.0.0.RELEASE</version> <scope>test</scope> </dependency> <dependency> <groupId>commons-logging</groupId> <artifactId>commons-logging</artifactId> <version>1.1.3</version> </dependency> <dependency> <groupId>commons-collections</groupId> <artifactId>commons-collections</artifactId> <version>3.2.1</version> </dependency> <!-- jstl --> <dependency> <groupId>javax.servlet</groupId> <artifactId>jstl</artifactId> <version>1.2</version> </dependency> <!-- mybatis --> <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis</artifactId> <version>${mybatis.version}</version> </dependency> <dependency> <groupId>org.mybatis</groupId> <artifactId>mybatis-spring</artifactId> <version>${mybatis-spring.version}</version> </dependency> <dependency> <groupId>mysql</groupId> <artifactId>mysql-connector-java</artifactId> <version>${mysql.version}</version> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>druid</artifactId> <version>0.2.23</version> </dependency> <!-- spring --> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-core</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.aspectj</groupId> <artifactId>aspectjrt</artifactId> <version>${aspectJ.version}</version> </dependency> <dependency> <groupId>org.aspectj</groupId> <artifactId>aspectjweaver</artifactId> <version>${aspectJ.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-aop</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-tx</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-jdbc</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-web</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-webmvc</artifactId> <version>${spring.version}</version> </dependency> <dependency> <groupId>org.springframework</groupId> <artifactId>spring-context-support</artifactId> <version>${spring.version}</version> </dependency> <!-- Jackson --> <dependency> <groupId>com.google.code.gson</groupId> <artifactId>gson</artifactId> <version>2.8.5</version> </dependency> <dependency> <groupId>org.codehaus.jackson</groupId> <artifactId>jackson-mapper-asl</artifactId> <version>1.9.13</version> </dependency> <dependency> <groupId>com.alibaba</groupId> <artifactId>fastjson</artifactId> <version>1.2.3</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-annotations</artifactId> <version>${jackson.version}</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-core</artifactId> <version>${jackson.version}</version> </dependency> <dependency> <groupId>com.fasterxml.jackson.core</groupId> <artifactId>jackson-databind</artifactId> <version>${jackson.version}</version> </dependency> <!-- 解決.jsp出錯 --> <dependency> <groupId>javax.servlet</groupId> <artifactId>javax.servlet-api</artifactId> <version>3.1.0</version> <scope>provided</scope> </dependency> <!-- 鏈接池 --> <dependency> <groupId>commons-dbcp</groupId> <artifactId>commons-dbcp</artifactId> <version>${commons-dbcp.version}</version> </dependency> <dependency> <groupId>commons-pool</groupId> <artifactId>commons-pool</artifactId> <version>${commons-pool.version}</version> </dependency> <!-- 文件上傳用的包 --> <dependency> <groupId>commons-fileupload</groupId> <artifactId>commons-fileupload</artifactId> <version>${commons-fileupload.version}</version> </dependency> <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <version>${commons-io.version}</version> </dependency> <!-- 數據校驗 --> <dependency> <groupId>javax.validation</groupId> <artifactId>validation-api</artifactId> <version>${validation-api.version}</version> </dependency> <dependency> <groupId>org.hibernate</groupId> <artifactId>hibernate-validator</artifactId> <version>${hibernate-validator.version}</version> </dependency> </dependencies> <build> <finalName>ssm-echarts-demo</finalName> <plugins> <plugin> <artifactId>maven-compiler-plugin</artifactId> <version>2.3.2</version> <configuration> <source>1.8</source> <target>1.8</target> </configuration> </plugin> <plugin> <groupId>org.apache.tomcat.maven</groupId> <artifactId>tomcat7-maven-plugin</artifactId> <configuration> <path>/</path> <port>8082</port> </configuration> </plugin> <plugin> <artifactId>maven-war-plugin</artifactId> <version>2.6</version> <configuration> <failOnMissingWebXml>false</failOnMissingWebXml> </configuration> </plugin> </plugins> <!--資源文件打包 --> <resources> <resource> <directory>src/main/resources</directory> <includes> <include>**/*.properties</include> <include>**/*.xml</include> <include>**/*.tld</include> </includes> <filtering>false</filtering> </resource> <resource> <directory>src/main/java</directory> <includes> <include>**/*.properties</include> <include>**/*.xml</include> <include>**/*.tld</include> </includes> <filtering>false</filtering> </resource> </resources> </build> </project>
因爲Spring配置較多因此我這裏分了4層,分別是dao、service、transaction、web。
applicationContext-dao.xml
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:aop="http://www.springframework.org/schema/aop" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.2.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsd"> <!-- 加載數據庫屬性文件,爲配置數據源作準備 --> <context:property-placeholder location="classpath:db.properties" /> <!-- bean採用註解式注入 --> <context:annotation-config></context:annotation-config> <!-- 配置數據源 使用dbcp --> <bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close"> <property name="driverClassName" value="${jdbc.driverClassName}" /> <property name="url" value="${jdbc.url}" /> <property name="username" value="${jdbc.username}" /> <property name="password" value="${jdbc.password}" /> <property name="maxActive" value="10" /> <property name="maxIdle" value="5" /> </bean> <!-- =================================MyBatis的整合====================================== --> <!-- 配置mybatis的SqlSessionFactoryBean --> <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean"> <!-- 指定mybatis 全局配置文件的位置 --> <property name="configLocation" value="classpath:mybatis/spring-mybatis.xml"></property> <property name="dataSource" ref="dataSource"></property> </bean> <!-- mapper配置: mapper批量處理,從mapper包中掃描mapper接口,自動建立代理對象而且在spring容器中註冊 遵循規範:將mapper.java和mapper.xml映射文件名稱保持一致,且在一個目錄中自動掃描 出來的mapper的bean的id爲mapper類名(首字母小寫) --> <bean class="org.mybatis.spring.mapper.MapperScannerConfigurer"> <property name="basePackage" value="com.snailclimb.dao"></property> <property name="sqlSessionFactoryBeanName" value="sqlSessionFactory"></property> </bean> </beans>
applicationContext-service.xml
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p="http://www.springframework.org/schema/p" xmlns:context="http://www.springframework.org/schema/context" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:aop="http://www.springframework.org/schema/aop" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.2.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsd"> <!-- 用戶管理service --> <context:component-scan base-package="com.snailclimb.service.impl"></context:component-scan> </beans>
applicationContext-trans.xml:
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context" xmlns:p="http://www.springframework.org/schema/p" xmlns:aop="http://www.springframework.org/schema/aop" xmlns:tx="http://www.springframework.org/schema/tx" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-4.0.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-4.0.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-4.0.xsd"> <!--事務管理器 --> <bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager"> <property name="dataSource" ref="dataSource" /> </bean> <!--通知 --> <tx:advice id="txAdvice" transaction-manager="transactionManager"> <tx:attributes> <tx:method name="save*" propagation="REQUIRED" /> <tx:method name="insert*" propagation="REQUIRED" /> <tx:method name="add*" propagation="REQUIRED" /> <tx:method name="create*" propagation="REQUIRED" /> <tx:method name="delete*" propagation="REQUIRED" /> <tx:method name="update*" propagation="REQUIRED" /> <tx:method name="find*" propagation="SUPPORTS" read-only="true" /> <tx:method name="select*" propagation="SUPPORTS" read-only="true" /> <tx:method name="get*" propagation="SUPPORTS" read-only="true" /> </tx:attributes> </tx:advice> <aop:config> <aop:advisor advice-ref="txAdvice" pointcut="execution(* com.snailclimb.service.*.*(..))" /> </aop:config> </beans>
springmvc.xml
<?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://www.springframework.org/schema/beans" xmlns:context="http://www.springframework.org/schema/context" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:mvc="http://www.springframework.org/schema/mvc" xsi:schemaLocation="http://www.springframework.org/schema/mvc http://www.springframework.org/schema/mvc/spring-mvc-4.0.xsd http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context-4.0.xsd http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-4.0.xsd"> <!-- 配置掃描器:@Controller是@Component的一個子類。@service --> <context:component-scan base-package="com.snailclimb.controller"></context:component-scan> <!-- 配置註解器: --> <mvc:annotation-driven></mvc:annotation-driven> <!-- 配置springMVC的視圖解析器 --> <bean class="org.springframework.web.servlet.view.InternalResourceViewResolver"> <property name="viewClass" value="org.springframework.web.servlet.view.JstlView" /> <property name="prefix" value="/WEB-INF/" /> <property name="suffix" value=".jsp" /> </bean> <!-- 開啓shiro註解功能 --> <!-- <bean class="org.springframework.aop.framework.autoproxy.DefaultAdvisorAutoProxyCreator" depends-on="lifecycleBeanPostProcessor"> <property name="proxyTargetClass" value="true" /> </bean> <bean class="org.apache.shiro.spring.security.interceptor.AuthorizationAttributeSourceAdvisor"> <property name="securityManager" ref="securityManager" /> </bean> --> <!-- 全局異常配置 --> <!-- <bean class="com.chen.PLoveLibrary.exception.HandleException"></bean> --> <!-- 文件上傳 --> <bean id="multipartResolver" class="org.springframework.web.multipart.commons.CommonsMultipartResolver"> <!-- 上傳文件大小上限,單位爲字節(10MB) --> <property name="maxUploadSize"> <value>10485760</value> </property> <!-- 請求的編碼格式,必須和jSP的pageEncoding屬性一致,以便正確讀取表單的內容,默認爲ISO-8859-1 --> <property name="defaultEncoding"> <value>UTF-8</value> </property> </bean> <!-- 配置校驗器 ,使用的是hibernate框架的validate,但與hibernate無關 --> <bean id="validator" class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean"> <!-- 校驗器,使用hibernate校驗器 --> <property name="providerClass" value="org.hibernate.validator.HibernateValidator" /> <!-- 指定校驗使用的資源文件,在文件中配置校驗錯誤信息,若是不指定則默認使用classpath下面的ValidationMessages.properties文件 --> <property name="validationMessageSource" ref="messageSource" /> </bean> <!-- 校驗錯誤信息配置文件 --> <bean id="messageSource" class="org.springframework.context.support.ReloadableResourceBundleMessageSource"> <!-- 資源文件名 --> <property name="basenames"> <list> <value>classpath:validateMessage</value> </list> </property> <!-- 資源文件編碼格式 --> <property name="fileEncodings" value="utf-8" /> <!-- 對資源文件內容緩存時間,單位秒 --> <property name="cacheSeconds" value="120" /> </bean> </beans>
Log4J的配置文件(Configuration File)就是用來設置記錄器的級別、存放器和佈局的,它可接key=value格式的設置或xml格式的設置信息。經過配置,能夠建立出Log4J的運行環境。
log4j.properties
log4j.rootLogger=DEBUG, stdout log4j.appender.stdout=org.apache.log4j.ConsoleAppender log4j.appender.stdout.layout=org.apache.log4j.PatternLayout log4j.appender.stdout.layout.ConversionPattern=%d [%-5p] %c - %m%n
db.properties
jdbc.driverClassName=com.mysql.jdbc.Driver jdbc.url=jdbc:mysql://localhost:3306/spider_zhihu_crawler jdbc.username=root jdbc.password=xxxxxx
spring-mybatis.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE configuration PUBLIC "-//mybatis.org//DTD Config 3.0//EN" "http://mybatis.org/dtd/mybatis-3-config.dtd"> <configuration> <!-- 開啓mybatis的緩存機制,開啓二級緩存 --> <settings> <setting name="cacheEnabled" value="true" /> </settings> <!-- mybatis中的別名命名 --> <typeAliases> <package name="com.snailclimb.bean" /> </typeAliases> </configuration>
web.xml
<?xml version="1.0" encoding="UTF-8"?> <web-app xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://java.sun.com/xml/ns/javaee" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd" id="WebApp_ID" version="3.0"> <display-name>ssm-echarts-demo</display-name> <welcome-file-list> <welcome-file>index.html</welcome-file> <welcome-file>index.htm</welcome-file> <welcome-file>index.jsp</welcome-file> <welcome-file>default.html</welcome-file> <welcome-file>default.htm</welcome-file> <welcome-file>default.jsp</welcome-file> </welcome-file-list> <!-- springMVC 靜態資源顯示問題。使用服務器來處理靜態資源。 緣由:客戶端發送request,springMVC有DispatcherServlet轉發,而DiapatchServlet的url-pattern是「/」,表明着全部的請求都要由DispatcherServlet 轉發,所以處理靜態資源,第一種就採用服務器端處理,請求先過DefaultServlet,靜態資源過濾則剩下的請求就交給dispatcherServlet處理。 --> <servlet> <servlet-name>default</servlet-name> <servlet-class>org.apache.catalina.servlets.DefaultServlet</servlet-class> <init-param> <param-name>debug</param-name> <param-value>0</param-value> </init-param> <init-param> <param-name>listings</param-name> <param-value>false</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <!-- 加載spring容器 ,若是沒有配置這個話會出現service沒法注入到controller層的狀況 --> <context-param> <param-name>contextConfigLocation</param-name> <param-value>classpath:spring/applicationContext-*.xml</param-value> </context-param> <listener> <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class> </listener> <!-- js靜態資源 --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.js</url-pattern> </servlet-mapping> <!-- css靜態資源 --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.css</url-pattern> </servlet-mapping> <!-- gif 靜態資源 --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.gif</url-pattern> </servlet-mapping> <!-- jpg靜態資源 --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.jpg</url-pattern> </servlet-mapping> <!-- ico --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.ico</url-pattern> </servlet-mapping> <!-- png --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.png</url-pattern> </servlet-mapping> <!-- htm --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.htm</url-pattern> </servlet-mapping> <!-- eot --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.eot</url-pattern> </servlet-mapping> <!-- svg --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.svg</url-pattern> </servlet-mapping> <!-- ttf --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.ttf</url-pattern> </servlet-mapping> <!-- woff --> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.woff</url-pattern> </servlet-mapping> <servlet-mapping> <servlet-name>default</servlet-name> <url-pattern>*.json</url-pattern> </servlet-mapping> <!-- 配置springMVC的DispathServlet --> <servlet> <servlet-name>spring</servlet-name> <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class> <init-param> <param-name>contextConfigLocation</param-name> <param-value>classpath:spring/springmvc.xml</param-value> </init-param> <load-on-startup>1</load-on-startup> </servlet> <servlet-mapping> <servlet-name>spring</servlet-name> <url-pattern>/</url-pattern> </servlet-mapping> <!-- 配置過濾器:CharacterEncodingFilter --> <filter> <filter-name>Encoding</filter-name> <filter-class>org.springframework.web.filter.CharacterEncodingFilter</filter-class> <init-param> <param-name>encoding</param-name> <param-value>UTF-8</param-value> </init-param> </filter> <filter-mapping> <filter-name>Encoding</filter-name> <url-pattern>*</url-pattern> </filter-mapping> </web-app>
User.java
package com.snailclimb.bean; /** * 用戶實體類 * * @author Snailclimb * */ public class User { private Integer id; private String userToken; private String location; private String business; private String sex; private String employment; private String education; private String username; private String url; private Integer agrees; private Integer thanks; private Integer asks; private Integer answers; private Integer posts; private Integer followees; private Integer followers; private String hashid; public Integer getId() { return id; } public void setId(Integer id) { this.id = id; } public String getUserToken() { return userToken; } public void setUserToken(String userToken) { this.userToken = userToken == null ? null : userToken.trim(); } public String getLocation() { return location; } public void setLocation(String location) { this.location = location == null ? null : location.trim(); } public String getBusiness() { return business; } public void setBusiness(String business) { this.business = business == null ? null : business.trim(); } public String getSex() { return sex; } public void setSex(String sex) { this.sex = sex == null ? null : sex.trim(); } public String getEmployment() { return employment; } public void setEmployment(String employment) { this.employment = employment == null ? null : employment.trim(); } public String getEducation() { return education; } public void setEducation(String education) { this.education = education == null ? null : education.trim(); } public String getUsername() { return username; } public void setUsername(String username) { this.username = username == null ? null : username.trim(); } public String getUrl() { return url; } public void setUrl(String url) { this.url = url == null ? null : url.trim(); } public Integer getAgrees() { return agrees; } public void setAgrees(Integer agrees) { this.agrees = agrees; } public Integer getThanks() { return thanks; } public void setThanks(Integer thanks) { this.thanks = thanks; } public Integer getAsks() { return asks; } public void setAsks(Integer asks) { this.asks = asks; } public Integer getAnswers() { return answers; } public void setAnswers(Integer answers) { this.answers = answers; } public Integer getPosts() { return posts; } public void setPosts(Integer posts) { this.posts = posts; } public Integer getFollowees() { return followees; } public void setFollowees(Integer followees) { this.followees = followees; } public Integer getFollowers() { return followers; } public void setFollowers(Integer followers) { this.followers = followers; } public String getHashid() { return hashid; } public void setHashid(String hashid) { this.hashid = hashid == null ? null : hashid.trim(); } }
ScoreResult.java
package com.snailclimb.bean; /** * 圓餅圖展現的時候須要使用到的對象 * * @author Snailclimb * */ public class ScoreResult { public int value; // 點贊數 public String name;// 人名 public float getValue() { return value; } public void setValue(int value) { this.value = value; } public String getName() { return name; } public void setName(String name) { this.name = name; } public ScoreResult(int value, String name) { super(); this.value = value; this.name = name; } @Override public String toString() { return "ScoreResult [value=" + value + ", name=" + name + "]"; } }
UserMapper.java
package com.snailclimb.dao; import java.util.List; import com.snailclimb.bean.User; public interface UserMapper { public List<User> selecAgreesTop10(); }
UserMapper.xml
<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd"> <mapper namespace="com.snailclimb.dao.UserMapper"> <select id="selecAgreesTop10" parameterType="User" resultType="User"> SELECT `agrees`, `username` FROM `spider_zhihu_crawler`.`user` GROUP BY `agrees` ORDER BY `agrees` DESC LIMIT 0,9; </select> </mapper>
UserService.java
package com.snailclimb.service; import java.util.List; import com.snailclimb.bean.User; public interface UserService { public List<User> selecAgreesTop10(); }
UserServiceImpl.java
package com.snailclimb.service.impl; import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Service; import com.snailclimb.bean.User; import com.snailclimb.dao.UserMapper; import com.snailclimb.service.UserService; @Service public class UserServiceImpl implements UserService { @Autowired private UserMapper userMapper; @Override public List<User> selecAgreesTop10() { return userMapper.selecAgreesTop10(); } }
package com.snailclimb.controller; import java.util.ArrayList; import java.util.List; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.stereotype.Controller; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.ResponseBody; import com.snailclimb.bean.ScoreResult; import com.snailclimb.bean.User; import com.snailclimb.service.UserService; @Controller @RequestMapping(value = "/echarts") public class UserController { @Autowired private UserService userService; /** * 折線圖和直方圖 * * @return */ @RequestMapping(value = "/agreeLineAndBar") @ResponseBody public List<User> getAgreesTop10() { return userService.selecAgreesTop10(); } /** * 圓餅圖 * * @return */ @RequestMapping(value = "/agreePie") @ResponseBody public List<ScoreResult> getAgreesTop10T() { List<User> list = userService.selecAgreesTop10(); List<ScoreResult> results = new ArrayList<ScoreResult>(); for (User user : list) { ScoreResult scoreResult = new ScoreResult(user.getAgrees(), user.getUsername()); results.add(scoreResult); } return results; } // 跳轉到圓餅圖展現 @RequestMapping(value = "/intoagreePie") public String intoagreePie() { return "agreePie"; } // 跳轉到折線圖和直方圖 @RequestMapping(value = "/intoagreeLineAndBar") public String Index() { return "agreeLineAndBar"; } }
因爲JSP頁面代碼過多,你們能夠直接去我上傳在Github的源碼上拷貝。下面我只貼一下Ajax請求的代碼。
下面以圓餅圖爲例,看看如何經過Ajax請求獲取數據動態填充
<!-- 顯示Echarts圖表 --> <div style="height:410px;min-height:100px;margin:0 auto;" id="main"></div> <script> //初始化echarts var pieCharts = //初始化echarts var pieCharts = echarts.init(document.getElementById("main")); //設置屬性 pieCharts.setOption({ title : { text : '贊同數', subtext : '贊同數比', x : 'center' }, tooltip : { trigger : 'item', formatter : "{a} <br/>{b} : {c} ({d}%)" }, legend : { orient : 'vertical', x : 'left', data : [] }, toolbox : { show : true, feature : { mark : { show : true }, //數據視圖 dataView : { show : true, readOnly : false }, restore : { show : true }, //是否能夠保存爲圖片 saveAsImage : { show : true } } }, calculable : true, series : [ { name : '贊同數', type : 'pie', radius : '55%', center : [ '50%', '60%' ], data : [] } ] }); //顯示一段動畫 pieCharts.showLoading(); //異步請求數據 $.ajax({ type : "post", async : true, url : '${pageContext.request.contextPath}/echarts/agreePie', data : [], dataType : "json", success : function(result) { //若是請求的結果不爲空,則填充數據,不然範圍錯誤消息 if (result) { pieCharts.hideLoading(); //隱藏加載動畫 pieCharts.setOption({ series : [ { data : result } ] }); } else { //返回的數據爲空時顯示提示信息 alert("圖表請求數據爲空,可能服務器暫未錄入近五天的觀測數據,您能夠稍後再試!"); pieCharts.hideLoading(); } }, error : function(errorMsg) { //請求失敗時執行該函數 alert("圖表請求數據失敗,多是服務器開小差了"); pieCharts.hideLoading(); } }) </script>
對應的Controller層代碼:
/** * 圓餅圖 */ @RequestMapping(value = "/agreePie") @ResponseBody public List<ScoreResult> getAgreesTop10T() { //查詢數據 List<User> list = userService.selecAgreesTop10(); List<ScoreResult> results = new ArrayList<ScoreResult>(); //將數據添加到results中 for (User user : list) { ScoreResult scoreResult = new ScoreResult(user.getAgrees(), user.getUsername()); results.add(scoreResult); } //返回results return results; }
後臺返回都是JSON格式的數據,以下圖所示:
這裏只是以知乎贊同數TOP10爲例子,帶着你們學習了SSM環境的搭建以及代碼的編寫,代碼中有很詳細的註釋。
經過本例子,你們徹底能夠本身作一個知乎粉絲數TOP、知乎感謝數TOP10等等例子出來。
另外本例子知識演示了圓餅圖、折線圖、柱狀圖的使用,你們能夠本身去Echarts官網深刻學習。
最後,本項目只是一個演示,還有不少須要優化的地方。好比可使用redis來作緩存提升查詢速度、能夠建立索引提升查詢速度或者直接將查詢到的數據緩存下來等等方法來提升查詢速度。
若是想要獲取更多個人原創文章,歡迎關注個人微信公衆號:" Java面試通關手冊" 。無套路,但願能與您共同進步,互相學習。