08-08 細分構建機器學習應用程序的流程-模型優化

目錄

• 細分構建機器學習應用程序的流程-模型優化
• 1、1.1 網格搜索法
• 2、1.2 隨機搜索法
  • 2.1 1.2.1 隨機採樣
  • 2.2 1.2.2 隨機搜索法

更新、更全的《機器學習》的更新網站，更有python、go、數據結構與算法、爬蟲、人工智能教學等着你：http://www.javashuo.com/article/p-vozphyqp-cm.html

細分構建機器學習應用程序的流程-模型優化

經過數據收集、數據預處理、訓練模型、測試模型上述四個步驟，通常能夠獲得一個不錯的模型，可是通常獲得的都是一個參數收斂的模型，然而咱們模型還有超參數或不一樣的核函數等，如r的非線性支持向量機的bf核或linear核；rbf核的非線性支持向量機超參數\(C、\gamma\)，正則化中的\(\alpha\)。咱們模型優化一塊主要是對模型超參數的優化，簡而言之就是輸入一組超參數，對每一個超參數對應的模型進行測試，選擇這一組超參數中最優的模型。

1、1.1 網格搜索法

網格搜索法至關於對你輸入的每個參數都進行驗證，而且能夠設置多個超參數。

from sklearn import datasets
from sklearn.svm import SVC
from sklearn.model_selection import ShuffleSplit
from sklearn.model_selection import GridSearchCV

iris = datasets.load_iris()
X = iris.data
y = iris.target

# 總共有2*4=8種選擇
parameters = {'kernel': ('linear', 'rbf'), 'C': [0.1, 1, 10, 100]}

svc = SVC(gamma="scale")

cv = ShuffleSplit(n_splits=10, test_size=0.3, random_state=1)

scoring = 'accuracy'

clf = GridSearchCV(svc, parameters, cv=cv, scoring=scoring)
clf.fit(X, y)

GridSearchCV(cv=ShuffleSplit(n_splits=10, random_state=1, test_size=0.3, train_size=None),
       error_score='raise-deprecating',
       estimator=SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape='ovr', degree=3, gamma='scale', kernel='rbf',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False),
       fit_params=None, iid='warn', n_jobs=None,
       param_grid={'kernel': ('linear', 'rbf'), 'C': [0.1, 1, 10, 100]},
       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',
       scoring='accuracy', verbose=0)

clf.cv_results_.keys()

dict_keys(['mean_fit_time', 'std_fit_time', 'mean_score_time', 'std_score_time', 'param_kernel', 'param_gamma', 'param_C', 'params', 'split0_test_score', 'split1_test_score', 'split2_test_score', 'split3_test_score', 'split4_test_score', 'mean_test_score', 'std_test_score', 'rank_test_score', 'split0_train_score', 'split1_train_score', 'split2_train_score', 'split3_train_score', 'split4_train_score', 'mean_train_score', 'std_train_score'])

clf.best_params_

{'C': 1, 'kernel': 'linear'}

clf.best_score_

0.9844444444444445

2、1.2 隨機搜索法

隨機搜索法通常用於超參數過多的時候，即當一組超參數有上千上萬個的時候，咱們會經過隨機搜索法隨機選擇一部分超參數，對模型調優。通常隨機搜索法會經過sklearn.model_selection.ParameterSampler方法進行採樣。

2.1 1.2.1 隨機採樣

from sklearn.model_selection import ParameterSampler
from scipy.stats.distributions import expon
import numpy as np

np.random.seed(1)
param_grid = {'a': [1, 2], 'b': expon()}
# expon爲指數分佈，該分佈取值爲無數個，即param_grid有無數個參數
param_grid

{'a': [1, 2],
 'b': <scipy.stats._distn_infrastructure.rv_frozen at 0x1a1772a630>}

# n_iter=4指定採樣次數爲4次
param_list = list(ParameterSampler(param_grid, n_iter=5))

rounded_list = [dict((k, round(v, 2)) for (k, v) in d.items())
                for d in param_list]
rounded_list

[{'a': 2, 'b': 5.87},
 {'a': 1, 'b': 0.0},
 {'a': 2, 'b': 6.95},
 {'a': 1, 'b': 0.1},
 {'a': 1, 'b': 0.49}]

2.2 1.2.2 隨機搜索法

from sklearn import datasets
from sklearn.svm import SVC
from sklearn.model_selection import ShuffleSplit
from sklearn.model_selection import RandomizedSearchCV

iris = datasets.load_iris()
X = iris.data
y = iris.target

# 總共有2*5*5=50種選擇
parameters = {'kernel': ('linear', 'rbf'), 'C': [
    0.1, 1, 10, 100, 1000], 'gamma': [0.1, 1, 10, 100, 1000]}
svc = SVC()

cv = ShuffleSplit(n_splits=5, test_size=0.3, random_state=1)

scoring = 'accuracy'

clf = RandomizedSearchCV(svc, parameters, cv=cv, scoring=scoring, n_iter=15)
clf.fit(X, y)

RandomizedSearchCV(cv=ShuffleSplit(n_splits=5, random_state=1, test_size=0.3, train_size=None),
          error_score='raise-deprecating',
          estimator=SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',
  kernel='rbf', max_iter=-1, probability=False, random_state=None,
  shrinking=True, tol=0.001, verbose=False),
          fit_params=None, iid='warn', n_iter=15, n_jobs=None,
          param_distributions={'kernel': ('linear', 'rbf'), 'C': [0.1, 1, 10, 100, 1000], 'gamma': [0.1, 1, 10, 100, 1000]},
          pre_dispatch='2*n_jobs', random_state=None, refit=True,
          return_train_score='warn', scoring='accuracy', verbose=0)

clf.cv_results_.keys()

dict_keys(['mean_fit_time', 'std_fit_time', 'mean_score_time', 'std_score_time', 'param_kernel', 'param_gamma', 'param_C', 'params', 'split0_test_score', 'split1_test_score', 'split2_test_score', 'split3_test_score', 'split4_test_score', 'mean_test_score', 'std_test_score', 'rank_test_score', 'split0_train_score', 'split1_train_score', 'split2_train_score', 'split3_train_score', 'split4_train_score', 'mean_train_score', 'std_train_score'])

clf.best_params_

{'kernel': 'linear', 'gamma': 1000, 'C': 1}

clf.best_score_

0.9822222222222222