驗證集，測試集，訓練集

 這三個名詞在機器學習領域的文章中極其常見，但不少人對他們的概念並非特別清楚，尤爲是後兩個常常被人混用。 Ripley, B.D（1996）在他的經典專著Pattern Recognition and Neural Networks中給出了這三個詞的定義。 Training set: A set of examples used for learning, which is to fit the parameters [i.e., weights] of the classifier. Validation set: A set of examples used to tune the parameters [i.e., architecture, not weights] of a classifier, for example to choose the number of hidden units in a neural network. Test set: A set of examples used only to assess the performance [generalization] of a fully specified classifier. 顯然，training set是用來訓練模型或肯定模型參數的，如ANN中權值等； validation set是用來作模型選擇（model selection），即作模型的最終優化及肯定的，如ANN的結構；而 test set則純粹是爲了測試已經訓練好的模型的推廣能力。固然，test set這並不能保證模型的正確性，他只是說類似的數據用此模型會得出類似的結果。但實際應用中，通常只將數據集分紅兩類，即training set 和test set，大多數文章並不涉及validation set。 Ripley還談到了Why separate test and validation sets? 1. The error rate estimate of the final model on validation data will be biased (smaller than the true error rate) since the validation set is used to select the final model. 2. After assessing the final model with the test set, YOU MUST NOT tune the model any further.