[LTR] RankLib.jar 包介紹
1、介紹
RankLib.jar 是一個學習排名（Learning to rank）算法的庫，目前已經實現了以下幾種算法：java
- MART
- RankNet
- RankBoost
- AdaRank
- Coordinate Ascent
- LambdaMART
- ListNet
- Random Forests
- Linear regression
2、jar 包
Usage: java -jar RankLib.jar <Params>
Params:
[+] Training (+ tuning and evaluation)
# 訓練數據
-train <file> Training data
# 指定排名算法
-ranker <type> Specify which ranking algorithm to use
0: MART (gradient boosted regression tree)
1: RankNet
2: RankBoost
3: AdaRank
4: Coordinate Ascent
6: LambdaMART
7: ListNet
8: Random Forests
9: Linear regression (L2 regularization)
# 特徵描述文件，列出要學習的特徵，每行一個特徵，默認使用全部特徵
[ -feature <file> ] Feature description file: list features to be considered by the learner, each on a separate line
If not specified, all features will be used.
#
[ -metric2t <metric> ] Metric to optimize on the training data. Supported: MAP, NDCG@k, DCG@k, P@k, RR@k, ERR@k (default=ERR@10)
[ -gmax <label> ] Highest judged relevance label. It affects the calculation of ERR (default=4, i.e. 5-point scale {0,1,2,3,4})
[ -silent ] Do not print progress messages (which are printed by default)
# 是否在驗證數據集上調整模型
[ -validate <file> ] Specify if you want to tune your system on the validation data (default=unspecified)
If specified, the final model will be the one that performs best on the validation data
# 訓練-驗證數據集的分割比例
[ -tvs <x \in [0..1]> ] If you don't have separate validation data, use this to set train-validation split to be (x)(1.0-x)
# 學習模型保存到指定文件
[ -save <model> ] Save the model learned (default=not-save)
# 是否要在數據上測試訓練的模型
[ -test <file> ] Specify if you want to evaluate the trained model on this data (default=unspecified)
# 訓練-測試數據集的分割比例
[ -tts <x \in [0..1]> ] Set train-test split to be (x)(1.0-x). -tts will override -tvs
# 默認與 metric2t 一致
[ -metric2T <metric> ] Metric to evaluate on the test data (default to the same as specified for -metric2t)
# 歸一化特徵向量，方法包括求和歸一化，均值/標準差歸一化，最大值/最小值歸一化
[ -norm <method>] Normalize all feature vectors (default=no-normalization). Method can be:
sum: normalize each feature by the sum of all its values
zscore: normalize each feature by its mean/standard deviation
linear: normalize each feature by its min/max values
# 在訓練數據集上執行交叉驗證
[ -kcv <k> ] Specify if you want to perform k-fold cross validation using the specified training data (default=NoCV)
-tvs can be used to further reserve a portion of the training data in each fold for validation
# 交叉驗證訓練庫模型的目錄
[ -kcvmd <dir> ] Directory for models trained via cross-validation (default=not-save)
[ -kcvmn <model> ] Name for model learned in each fold. It will be prefix-ed with the fold-number (default=empty)
[-] RankNet-specific parameters # 特定參數
# 訓練迭代次數
[ -epoch <T> ] The number of epochs to train (default=100)
# 隱含層個數
[ -layer <layer> ] The number of hidden layers (default=1)
# 每層隱含節點個數
[ -node <node> ] The number of hidden nodes per layer (default=10)
# 學習率
[ -lr <rate> ] Learning rate (default=0.00005)
[-] RankBoost-specific parameters # 特定參數
# 訓練迭代次數
[ -round <T> ] The number of rounds to train (default=300)
# 搜索的閾值候選個數
[ -tc <k> ] Number of threshold candidates to search. -1 to use all feature values (default=10)
[-] AdaRank-specific parameters # 特定參數
# 訓練迭代次數
[ -round <T> ] The number of rounds to train (default=500)
#
[ -noeq ] Train without enqueuing too-strong features (default=unspecified)
# 連續兩輪學習之間的偏差
[ -tolerance <t> ] Tolerance between two consecutive rounds of learning (default=0.002)
# 一個特徵能夠被連續選擇而不改變性能的最大次數
[ -max <times> ] The maximum number of times can a feature be consecutively selected without changing performance (default=5)
[-] Coordinate Ascent-specific parameters # 特定參數
[ -r <k> ] The number of random restarts (default=5)
[ -i <iteration> ] The number of iterations to search in each dimension (default=25)
[ -tolerance <t> ] Performance tolerance between two solutions (default=0.001)
[ -reg <slack> ] Regularization parameter (default=no-regularization)
[-] {MART, LambdaMART}-specific parameters # 特定參數
# 樹的個數
[ -tree <t> ] Number of trees (default=1000)
# 一個葉子的樣本個數
[ -leaf <l> ] Number of leaves for each tree (default=10)
# 學習率
[ -shrinkage <factor> ] Shrinkage, or learning rate (default=0.1)
# 樹分割時的候選特徵個數
[ -tc <k> ] Number of threshold candidates for tree spliting. -1 to use all feature values (default=256)
# 一個葉子最少的樣本個數
[ -mls <n> ] Min leaf support -- minimum #samples each leaf has to contain (default=1)
[ -estop <e> ] Stop early when no improvement is observed on validaton data in e consecutive rounds (default=100)
[-] ListNet-specific parameters
[ -epoch <T> ] The number of epochs to train (default=1500)
[ -lr <rate> ] Learning rate (default=0.00001)
[-] Random Forests-specific parameters # 隨機森林特定參數
[ -bag <r> ] Number of bags (default=300)
# 子集採樣率
[ -srate <r> ] Sub-sampling rate (default=1.0)
# 特徵採樣率
[ -frate <r> ] Feature sampling rate (default=0.3)
[ -rtype <type> ] Ranker to bag (default=0, i.e. MART)
# 樹個數
[ -tree <t> ] Number of trees in each bag (default=1)
# 每棵樹的葉節點個數
[ -leaf <l> ] Number of leaves for each tree (default=100)
# 學習率
[ -shrinkage <factor> ] Shrinkage, or learning rate (default=0.1)
# 樹分割時使用的候選特徵閾值個數
[ -tc <k> ] Number of threshold candidates for tree spliting. -1 to use all feature values (default=256)
[ -mls <n> ] Min leaf support -- minimum #samples each leaf has to contain (default=1)
[-] Linear Regression-specific parameters
[ -L2 <reg> ] L2 regularization parameter (default=1.0E-10)
[+] Testing previously saved models # 測試已保存的模型
# 加載模型
-load <model> The model to load
Multiple -load can be used to specify models from multiple folds (in increasing order),
in which case the test/rank data will be partitioned accordingly.
# 測試數據
-test <file> Test data to evaluate the model(s) (specify either this or -rank but not both)
# 對指定文件中的樣本排序，與 -test 不能同時使用
-rank <file> Rank the samples in the specified file (specify either this or -test but not both)
[ -metric2T <metric> ] Metric to evaluate on the test data (default=ERR@10)
[ -gmax <label> ] Highest judged relevance label. It affects the calculation of ERR (default=4, i.e. 5-point scale {0,1,2,3,4})
[ -score <file>] Store ranker's score for each object being ranked (has to be used with -rank)
# 打印單個排名列表上的性能（必須與 -test 一塊兒使用）
[ -idv <file> ] Save model performance (in test metric) on individual ranked lists (has to be used with -test)
# 特徵歸一化
[ -norm ] Normalize feature vectors (similar to -norm for training/tuning)
1. -train <file>
指定訓練數據的文件，訓練數據格式：node
label qid:$id $featureid:$featurevalue $featureid:$featurevalue ... # description
每行表明一個樣本，相同查詢請求的樣本的 qid 相同，label 表示該樣本和該查詢請求的相關程度，description 描述信息，不參與訓練計算。算法
二、-ranker <type>
指定排名算法網絡
- MART（Multiple Additive Regression Tree）多重增量回歸樹
- GBDT（Gradient Boosting Decision Tree）梯度漸進決策樹
- GBRT（Gradient Boosting Regression Tree）梯度漸進迴歸樹
- TreeNet 決策樹網絡
- RankNet
- RankBoost
- AdaRank
- Coordinate Ascent
- LambdaMART
- ListNet
- Random Forests
- Linear regression
三、-feature <file>
指定樣本的特徵定義文件，格式以下：dom
feature1 feature2 ... # featureK(該特徵不參與分析)
四、-metric2t <metric>
指定信息檢索中的評價指標，包括：
MAP, NDCG@k, DCG@k, P@k, RR@k, ERR@kide
五、Example
java -jar bin/RankLib.jar -train MQ2008/Fold1/train.txt -test MQ2008/Fold1/test.txt -validate MQ2008/Fold1/vali.txt -ranker 6 -metric2t NDCG@10 -metric2T ERR@10 -save mymodel.txt
命令解釋 >>>
訓練數據：MQ2008/Fold1/train.txt
測試數據：MQ2008/Fold1/test.txt
驗證數據：MQ2008/Fold1/vali.txt
排名算法：6，LambdaMART
評估指標：NDCG，取排名前 10 個數據進行計算
測試數據評估指標：ERR，取排名前 10 個數據進行計算
保存模型：mymodel.txt性能
- 參數 -validate 是可選的，但能夠更好的模型結果，對於 RankNet/MART/LambdaMART 很是重要。
- -metric2t 僅應用於 list-wise 算法（AdaRank、Coordinate Ascent 和 LambdaMART）；point-wise 和 Pair-wise 算法（MART、RankNet、RankBoost）是使用本身內部的 RMSE/pair-wise loss 做爲評價指標。ListNet 雖然是 list-wise 算法，可是也不用 metric2t 指定評價指標。
六、k-fold cross validation
- 順序分區
java -jar bin/RankLib.jar -train MQ2008/Fold1/train.txt -ranker 4 -kcv 5 -kcvmd models/ -kcvmn ca -metric2t NDCG@10 -metric2T ERR@10
按順序將訓練數據拆分5等份，第 i 份數據做爲第 i 摺疊的測試數據，第 i 摺疊的訓練數據則是由其餘摺疊的數據組成。學習
- 隨機分區
java -cp bin/RankLib.jar ciir.umass.edu.features.FeatureManager -input MQ2008/Fold1/train.txt -output mydata/ -shuffle
將訓練數據 train.txt 從新洗牌存儲在 mydata/ 目錄下 train.txt.shuffled測試
- 獲取每一個摺疊中的數據
java -cp bin/RankLib.jar ciir.umass.edu.features.FeatureManager -input MQ2008/Fold1/train.txt.shuffled -output mydata/ -k 5
七、評估已訓練的模型
java -jar bin/RankLib.jar -load mymodel.txt -test MQ2008/Fold1/test.txt -metric2T ERR@10
八、模型對比
java -jar bin/RankLib.jar -test MQ2008/Fold1/test.txt -metric2T NDCG@10 -idv output/baseline.ndcg.txt java -jar bin/RankLib.jar -load ca.model.txt -test MQ2008/Fold1/test.txt -metric2T NDCG@10 -idv output/ca.ndcg.txt java -jar bin/RankLib.jar -load lm.model.txt -test MQ2008/Fold1/test.txt -metric2T NDCG@10 -idv output/lm.ndcg.txt
輸出文件中包含了每條查詢的 NDCG@10 指標值，以及全部查詢的綜合指標，例如：ui
NDCG@10 170 0.0 NDCG@10 176 0.6722390270733757 NDCG@10 177 0.4772656487866462 NDCG@10 178 0.539003131276382 NDCG@10 185 0.6131471927654585 NDCG@10 189 1.0 NDCG@10 191 0.6309297535714574 NDCG@10 192 1.0 NDCG@10 194 0.2532778777010656 NDCG@10 197 1.0 NDCG@10 200 0.6131471927654585 NDCG@10 204 0.4772656487866462 NDCG@10 207 0.0 NDCG@10 209 0.123151194370365 NDCG@10 221 0.39038004999210174 NDCG@10 all 0.5193204478059303
而後再進行對比：
java -cp RankLib.jar ciir.umass.edu.eval.Analyzer -all output/ -base baseline.ndcg.txt > analysis.txt
對比結果 analysis.txt 以下：
Overall comparison
------------------------------------------------------------------------
System Performance Improvement Win Loss p-value
baseline_ndcg.txt [baseline] 0.093
LM_ndcg.txt 0.2863 +0.1933 (+207.8%) 9 1 0.03
CA_ndcg.txt 0.5193 +0.4263 (+458.26%) 12 0 0.0
Detailed break down
------------------------------------------------------------------------
[ < -100%) [-100%,-75%) [-75%,-50%) [-50%,-25%) [-25%,0%) (0%,+25%] (+25%,+50%] (+50%,+75%] (+75%,+100%] ( > +100%]
LM_ndcg.txt 0 0 1 0 0 4 2 2 1 0
CA_ndcg.txt 0 0 0 0 0 1 6 2 3 0
九、利用訓練模型重排名
java -jar RankLib.jar -load mymodel.txt -rank myResultLists.txt -score myScoreFile.txt
myScoreFile.txt 文件中只是增長了一列，表示從新計算的排名評分，須要本身另外根據該評分排序獲取新的排名順序。
1 0 -7.528650760650635 1 1 2.9022061824798584 1 2 -0.700125515460968 1 3 2.376657485961914 1 4 -0.29666265845298767 1 5 -2.038628101348877 1 6 -5.267711162567139 1 7 -2.022146463394165 1 8 0.6741248369216919 ...
參考
相關文章
- 1. Java.lang.reflect包介紹
- 2. R包介紹
- 3. java juc包介紹
- 4. JS 閉包介紹
- 5. javascript閉包介紹
- 6. Go net包介紹
- 7. LTR 之RankNet、LambdaRank、LambdaMART
- 8. math.h 包含函數介紹
- 9. JBPM4.4-依賴包介紹
- 10. extjs4下載包介紹
- 更多相關文章...
- • 網站主機 介紹 - 網站主機教程
- • Spring目錄結構和基礎JAR包介紹 - Spring教程
- • Java Agent入門實戰（一）-Instrumentation介紹與使用
- • 三篇文章瞭解 TiDB 技術內幕 —— 說計算
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
歡迎關注本站公眾號,獲取更多信息
相關文章
- 1. Java.lang.reflect包介紹
- 2. R包介紹
- 3. java juc包介紹
- 4. JS 閉包介紹
- 5. javascript閉包介紹
- 6. Go net包介紹
- 7. LTR 之RankNet、LambdaRank、LambdaMART
- 8. math.h 包含函數介紹
- 9. JBPM4.4-依賴包介紹
- 10. extjs4下載包介紹