怎麼直接下載niconico的視頻文件 (僞)(利用curllib)

簡單一句話說明就是: 瀏覽器跟niconico的交互無非get、post, 中間有經過ssh登陸後cookie的添加, 瀏覽器執行nico頁面上的javascript致使cookie的增長或修改, 查詢(根據番號)和請求(GET)視頻文件時須要帶上特定的cookie才能請求成功。 javascript

那麼哪些操做更改了cookie, 自定義的http請求又要怎麼寫才能把視頻文件搞下來呢?  這些信息經過firebug清楚明瞭。 html

從結論往回推倒, 先看下載某個文件的GET請求裏帶上的cookie是這樣:

Cookie nicosid=1376560121.1985494370; WT_FPC=id=183.1.222.67-2718089024.30316956:lv=1376563737475:ss=1376563716164; user_session=user_session_5876440_15212688[人工馬賽克]029617; __utma=8292653.1210056190.1376560156.1376619137.1376619257.4; __utmz=8292653.1376560156.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); nicohistory=sm20892030%3A1376619147%3A1376619352%3Abfccbcfceb510b97%3A5%2Cnm14545546%3A1376618022%3A1376619352%3A71811917e7fba6e4%3A5%2Csm21596305%3A1376560377%3A1376560377%3A8b8813d10b48d5a0%3A1; optimizelySegments=%7B%22281621258%22%3A%22ff%22%2C%22280358707%22%3A%22false%22%2C%22281626694%22%3A%22direct%22%7D; optimizelyEndUserId=oeu1376560363326r0.5227189826598727; optimizelyBuckets=%7B%7D; __utmb=8292653; __utmc=8292653 Host smile-cln15.nicovideo.jp java

實際測試其實只須要nicosidnicohistory這兩個就足夠了。  nicosid怎麼獲取? 訪問niconico首頁就會有nicosid寫到cookie裏。 nicohistory暫時無論。 另一個假設的前提是咱們已經知道視頻文件的地址。 下面是代碼: shell

#include <functional>
#include <algorithm>
#include <curl\curl.h>

size_t writefunc(void *ptr, size_t size, size_t nmemb, std::string *s)
{
	s->append((char *)ptr, size*nmemb);

	return size*nmemb;
}

void TestDownload()
{
	CURL *curl;
	CURLcode res;
	std::string str;

	curl = curl_easy_init();
	if(!curl) {
		return ;
	}
	// 視頻文件的地址
	curl_easy_setopt(curl, CURLOPT_URL, "http://smile-cln15.nicovideo.jp/smile?m=20892030.52844");
	// swf地址, 基本固定, http://res.nimg.jp/swf/player/nicoplayer.swf?ts=xxx 裏的ts=xxx也是能夠省略的
	curl_easy_setopt(curl, CURLOPT_REFERER, "http://res.nimg.jp/swf/player/nicoplayer.swf");
	// nicohistory後面的字段也是能夠精簡的
	curl_easy_setopt(curl, CURLOPT_COOKIE,
		//"nicosid=1376560121.1985494370; nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:4,nm14545546:1376618022:1376619268:b3cef30c0550ab80:4,sm21596305:1376560377:1376560377:8b8813d10b48d5a0:1");
		"nicosid=1376620271.695811941; nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:1"); 

	curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc);
	curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str);

	res = curl_easy_perform(curl);

	AtlTrace("res:%d\n", res);
	curl_easy_cleanup(curl);
}

int _tmain(int argc, _TCHAR* argv[])
{
    TestDownload();
    return 0;
}

接下來講說怎麼獲取nicosid,  其實就是http請求nico首頁, 而後設置curlib把cookie寫到文件裏, 最後打開文件看就是了。 api

void TestGetNicoidCookie()
{
	CURL *curl;
	CURLcode res;
	std::string str;

	curl = curl_easy_init();
	if(!curl) {
		return ;
	}

	curl_easy_setopt(curl, CURLOPT_URL, "http://www.nicovideo.jp/");
	curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "c:\\nicoid.txt");
	curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "c:\\nicoid.txt");
	curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

	curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc);
	curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str);

	res = curl_easy_perform(curl);
	AtlTrace("res:%d\n", res);
	curl_easy_cleanup(curl);
}
成功後打開c盤的nicoid.ext, 內容:
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.

.nicovideo.jp	TRUE	/	FALSE	1691980271	nicosid	1376620271.695811941

至於nicohistory, 原來的
nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:4,nm14545546:1376618022:1376619268:b3cef30c0550ab80:4,sm21596305:1376560377:1376560377:8b8813d10b48d5a0:1
可裁剪爲
nicohistory=sm20892030:1376619147:1376619268:b88d468b9ae66c4a:1 瀏覽器

sm20892030一看就知道是番號了, 後面三串數字, 前兩串對比utma來看應該是時戳, 最後一個估計是前兩串時戳的加密驗證(前兩串修改了一下後對面就返回403了), nicohistory怎麼獲取我實在沒精力查找是那個頁面的哪段javascript添加/修改/計算的, 否則既然瀏覽器知道怎麼算出來你確定也能知道怎麼算出來。 cookie

再往回查看一下發現前提條件——視頻文件的地址還不知道怎麼獲取, 經過firebug就知道是帶上指定番號的POST請求http://flapi.nicovideo.jp/api/getflv頁面來獲取的, 而這個請求能成功的關鍵是cookie裏帶上user_session,假設咱們已經獲得了user_sessionsession

void TestGetFileAddressFromNicoSMNumber()
{
	CURL *curl;
	CURLcode res;
	std::string str;

	curl = curl_easy_init();
	if(!curl) {
		return ;
	}

	curl_easy_setopt(curl, CURLOPT_URL, "http://flapi.nicovideo.jp/api/getflv");  //基本固定
	curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "v=sm20892030"); //番號
	curl_easy_setopt(curl, CURLOPT_REFERER, "http://res.nimg.jp/swf/player/nicoplayer.swf");
	curl_easy_setopt(curl, CURLOPT_COOKIE,
		"user_session=user_session_5[碼]440_20[人工打碼]0019268;" );

	curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc);
	curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str);

	res = curl_easy_perform(curl);

	AtlTrace("res:%d\n", res);

	std::for_each(str.begin(), str.end(),
		[](const char& a) {
			AtlTrace("%c", a);
	});

	curl_easy_cleanup(curl);
}

成功後控制檯輸出: app

res:0
thread_id=1368834513&l=253&url=http%3A%2F%2Fsmile-cln15.nicovideo.jp%2Fsmile%3Fm%3D20892030.52844&link=http%3A%2F%2Fwww.smilevideo.jp%2Fview%2F20892030%2F5876440&ms=http%3A%2F%2Fmsg.nicovideo.jp%2F16%2Fapi%2F&ms_sub=http%3A%2F%2Fsub.msg.nicovideo.jp%2F16%2Fapi%2F&user_id=[打碼]&is_premium=0&nickname=[打碼]&time=1376621655816&done=true&ng_rv=1&hms=hiroba01.nicovideo.jp&hmsp=2529&hmst=1000000005&hmstk=1376621715.hFN9Cav8gE8GemURtHo0kuODpB0 ssh

咱們須要的只是url參數而已, 就是上面加粗那段: url=http%3A%2F%2Fsmile-cln15.nicovideo.jp%2Fsmile%3Fm%3D20892030.52844, 明顯用了urlencode, 變身一下:
%3A=':'
%2F='/'
%3F='?'
%3D='='
---------------> http://smile-cln15.nicovideo.jp/smile?m=20892030.52844
這就是咱們要下載的地址了。

如今的問題是, user_session這個cookie怎麼獲得? 這個cookie要你經過帳號密碼成功登陸nico後才能拿到。 而nico的登陸用的是https, 即SSH加密的http請求, 代碼:

void TestLoginToGetUserSession()
{
	CURL *curl;
	CURLcode res;
	std::string str;

	curl = curl_easy_init();
	if(!curl) {
		return ;
	}

	curl_easy_setopt(curl, CURLOPT_URL, "https://secure.nicovideo.jp/secure/login?site=niconico");
	curl_easy_setopt(curl, CURLOPT_USE_SSL, (long)CURLUSESSL_ALL);
	curl_easy_setopt(curl, CURLOPT_COOKIEJAR, "c:\\user_session.txt");
	curl_easy_setopt(curl, CURLOPT_COOKIEFILE, "c:\\user_session.txt");
	curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, FALSE);
	curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, FALSE);
	curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L);

	curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "next_url=&mail_tel=填入你帳號(郵箱)&password=填入你密碼");
	curl_easy_setopt(curl, CURLOPT_REFERER, "https://secure.nicovideo.jp/secure/login_form");
	curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, writefunc);
	curl_easy_setopt(curl, CURLOPT_WRITEDATA, &str);

	res = curl_easy_perform(curl);
	AtlTrace("res:%d\n", res);

	curl_easy_cleanup(curl);
}
成功後打開c盤的user_session.txt, 內容:
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.

.nicovideo.jp	TRUE	/	FALSE	1691982237	nicosid	1376622237.1622763304
secure.nicovideo.jp	FALSE	/secure/	FALSE	1	user_session	deleted
secure.nicovideo.jp	FALSE	/	FALSE	1	user_session	deleted
.nicovideo.jp	TRUE	/	FALSE	1379214237	user_session	user_session_5876440_13[人工打碼]165630
.nicovideo.jp	TRUE	/	FALSE	1	repair_history	deleted
ok, user_session也拿到了。 那還有什麼問題? 對, nicohistory, 這個就交給大家了(拇指)。
相關文章
相關標籤/搜索