Jsoup下載抖音文件限制1M的問題解決

1.最近有個朋友找我須要下載抖音文件,我研究了下,首先想到用的是用jsoup這個組件進行下載。

2.項目中通常都是用maven開發的 ,首先要引入jsoup的gav，還用到了commons-io組件。

<dependency>
	    <groupId>org.jsoup</groupId>
	    <artifactId>jsoup</artifactId>
	    <version>1.11.2</version>
	</dependency>
	<dependency>
		<groupId>commons-io</groupId>
		<artifactId>commons-io</artifactId>
		<version>2.6</version>
	</dependency>
複製代碼

3.代碼乾貨：

package com.vhd;
 
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
 
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringUtils;
import org.jsoup.Connection.Response;
import org.jsoup.Jsoup;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
 
public class DouYinUtil1 {
 
	private final static Logger log = LoggerFactory
			.getLogger(DouYinUtil.class);
	
 
	public static void getDouYinUrl(String douyinUrl,String fileName){
		try {
			String content = Jsoup.connect(douyinUrl).ignoreContentType(true).timeout(10000).execute().body();
			String downUrl = StringUtils.substringBetween(content, "playAddr: \"", "\",");
			if(StringUtils.isEmpty(downUrl)){
				return;
			}
			Response document = Jsoup.connect(downUrl).timeout(10000).execute();
			BufferedInputStream stream = document.bodyStream();
			File file = new File(fileName);
			FileUtils.copyInputStreamToFile(stream, file);
		} catch (IOException e) {
			e.printStackTrace();
			log.error("douyinUrl:{},error:{}",douyinUrl,e);
		}
	}
	
}
複製代碼

好比爬去這個連接下的視頻,並把他下載到本地：

www.iesdouyin.com/share/video…

結果下載以後只有1M的大小.就研究了下jsouip這個組件,測試只有下載1M的文件：

String url = "https://aweme.snssdk.com/aweme/v1/playwm/?video_id=v0300fd30000bi2c7ipdja5i7hebu8d0&line=0";
	System.out.println(Jsoup.connect(url).request().maxBodySize());
複製代碼

而後查看API文檔：

jsoup.org/apidocs/org…

Connection maxBodySize​(int bytes) Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated.

能夠設置下載文件的大下.

而後在上面代碼基礎增長 了.maxBodySize(3000000)這個方法就能夠限制1M大小的問題了。

package com.vhd;
 
import java.io.BufferedInputStream;
import java.io.File;
import java.io.IOException;
 
import org.apache.commons.io.FileUtils;
import org.apache.commons.lang3.StringUtils;
import org.jsoup.Connection.Response;
import org.jsoup.Jsoup;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
 
public class DouYinUtil {
 
	private final static Logger log = LoggerFactory
			.getLogger(DouYinUtil.class);
	
 
	public static void getDouYinUrl(String douyinUrl,String fileName){
		try {
			String content = Jsoup.connect(douyinUrl).ignoreContentType(true).timeout(10000).execute().body();
			String downUrl = StringUtils.substringBetween(content, "playAddr: \"", "\",");
			if(StringUtils.isEmpty(downUrl)){
				return;
			}
			Response document = Jsoup.connect(downUrl).maxBodySize(30000000).timeout(10000).execute();
			BufferedInputStream stream = document.bodyStream();
			File file = new File(fileName);
			FileUtils.copyInputStreamToFile(stream, file);
            //此方法建議能夠使用下面copy大文件的方法性能會更好些
            //IOUtils.copyLarge(stream, new FileOutputStream("D:\\123.mp4"));
		} catch (IOException e) {
			e.printStackTrace();
			log.error("douyinUrl:{},error:{}",douyinUrl,e);
		}
	}
	
	public static void main(String[] args) {
		String url = "https://aweme.snssdk.com/aweme/v1/playwm/?video_id=v0300fd30000bi2c7ipdja5i7hebu8d0&line=0";
		System.out.println(Jsoup.connect(url).request().maxBodySize());
	}
}
複製代碼

以上幫朋友解決了下載抖音大文件的問題,也能夠作成批量下載抖音,有興趣能夠嘗試一下！

歡迎關注個人公衆號，來一塊兒學習：