機器學習之數據預處理，Pandas讀取excel數據

Python讀寫excel的工具庫不少，好比最耳熟能詳的xlrd、xlwt，xlutils，openpyxl等。其中xlrd和xlwt庫一般配合使用，一個用於讀，一個用於寫excel。xlutils結合xlrd能夠達到修改excel文件目的。openpyxl能夠對excel文件同時進行讀寫操做。

而說到數據預處理，pandas就體現除了它的強大之處，而且它還支持可讀寫多種文檔格式，其中就包括對excel的讀寫。本文重點就是介紹pandas對excel數據集的預處理。

機器學習經常使用的模型對數據輸入都是有要求的，多數機器學習算法最基本的要求是訓練數據要轉換成數值格式。固然，也有像決策樹算法這種不須要轉換爲數值的算法，這裏不作特例討論。

pandas讀取excel文件的函數是pandas.read_excel()，主要參數包括：

 

io : 讀取的excel文檔地址，

        string, path object (pathlib.Path or py._path.local.LocalPath),

file-like object, pandas ExcelFile, or xlrd workbook. The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. For instance, a local file could be file://localhost/path/to/workbook.xlsx

sheet_name : 讀取的excel指定的sheet頁

        string, int, mixed list of strings/ints, or None, default 0

Strings are used for sheet names, Integers are used in zero-indexed sheet positions.

Lists of strings/integers are used to request multiple sheets.

Specify None to get all sheets.

str|int -> DataFrame is returned. list|None -> Dict of DataFrames is returned, with keys representing sheets.

Available Cases

• Defaults to 0 -> 1st sheet as a DataFrame
• 1 -> 2nd sheet as a DataFrame
• 「Sheet1」 -> 1st sheet as a DataFrame
• [0,1,」Sheet5」] -> 1st, 2nd & 5th sheet as a dictionary of DataFrames
• None -> All sheets as a dictionary of DataFrames

header : 設置讀取的excel第一行是否做爲列名稱

        int, list of ints, default 0

Row (0-indexed) to use for the column labels of the parsed DataFrame. If a list of integers is passed those row positions will be combined into a MultiIndex. Use None if there is no header.

names :設置每列的名稱，數組形式參數

　　　array-like, default None

List of column names to use. If file contains no header row, then you should explicitly pass header=None

index_col :設置讀取的excel第一列是否做爲行名稱

　　　int, list of ints, default None

Column (0-indexed) to use as the row labels of the DataFrame. Pass None if there is no such column. If a list is passed, those columns will be combined into a MultiIndex. If a subset of data is selected with usecols, index_col is based on the subset.

usecols :執行須要讀取的數據列，一般載入的excel包含不須要的列

　　　　int or list, default None

• If None then parse all columns,
• If int then indicates last column to be parsed
• If list of ints then indicates list of column numbers to be parsed
• If string then indicates comma separated list of Excel column letters and column ranges (e.g. 「A:E」 or 「A,C,E:F」). Ranges are inclusive of both sides.

下盡是一些pandas讀取excel數據的示例：

將數據集寫入excel文件：

>>> df_out = pd.DataFrame([('string1', 1), ... ('string2', 2), ... ('string3', 3)], ... columns=['Name', 'Value']) >>> df_out  Name Value 0 string1 1 1 string2 2 2 string3 3 >>> df_out.to_excel('tmp.xlsx') 

讀取excel文件：

>>> pd.read_excel('tmp.xlsx')  Name Value 0 string1 1 1 string2 2 2 string3 3

參數index_col and header 都設置爲None表示不讀取excel的第一行和第一列做爲標題和默認索引：

>>> pd.read_excel('tmp.xlsx', index_col=None, header=None)  0 1 2 0 NaN Name Value 1 0.0 string1 1 2 1.0 string2 2 3 2.0 string3 3 

甚至能夠專門制定列的格式：

>>> pd.read_excel('tmp.xlsx', dtype={'Name':str, 'Value':float})  Name Value 0 string1 1.0 1 string2 2.0 2 string3 3.0 

下面是綜合示例：讀取text.xlsx文件的sheet1頁，僅載入D:F列的數據。這裏F列是類別標籤，須要類別1和類別2轉換爲數字，應用於機器學習的輸入建模。

 

import pandas as pd

def reader(path,sheet):
    return pd.read_excel(path, sheet_name=sheet, usecols='D:F')
    
trainrd = reader('text.xlsx','sheet1')
trainrd.head(5)  #查看前5行數據
trainrd['x']=0  #新建一列x
trainrd.loc[trainrd['類別']=='類別1','x']=0 #將類別列的文字轉換爲數字
trainrd.loc[trainrd['類別']=='類別2','x']=1