深度學習之卷積神經網絡(CNN)詳解與代碼實現（二）

時間 2020-07-08

標籤深度學習神經網絡 cnn 詳解代碼實現简体版

原文原文鏈接

用Tensorflow實現卷積神經網絡(CNN)

本文系做者原創，轉載請註明出處:https://www.cnblogs.com/further-further-further/p/10737065.html html

1.踩過的坑(tensorflow)

2.tensorboard

3.代碼實現（python3.5）

4.運行結果以及分析

1.踩過的坑(tensorflow)

上一章CNN中各個算法都是純手工實現的，可能存在一些難以發現的問題，這也是準確率不高的一個緣由，這章主要利用tensorflow框架來實現卷積神經網絡，數據源仍是cifar（具體下載見上一章）python

在利用tensorflow框架實現CNN時，須要注意如下幾點：git

1.輸入數據定義時，x只是起到佔位符的做用（看不到真實值，只是爲了可以運行代碼，獲取相應的tensor節點，這一點跟咱們以前代碼流程徹底相反，真正數據流的執行在session會話裏）算法

x：輸入數據，y_：標籤數據，keep_prob: 機率因子，防止過擬合。瀏覽器

定義，且是全局變量。網絡

x = tf.placeholder(tf.float32, [None, 3072], name='x')

y_ = tf.placeholder(tf.float32, [None, 10], name='y_')

keep_prob = tf.placeholder(tf.float32)

後面在session裏必需要初始化session

sess.run(tf.global_variables_initializer())

在session run時必需要傳獲得該tensor節點含有參數值（x, y_, keep_prob）數據結構

train_accuracy = accuracy.eval(feed_dict={ x: batch[0], y_: batch[1], keep_prob: 1.0})

2.原始數據集標籤要向量化；app

例如cifar有10個類別，若是類別標籤是 6 對應向量[0,0,0,0,0,1,0,0,0,0]框架

3.知道每一步操做的數據大小的變化，否則，報錯的時候很難定位（我的認爲這也是tensorflow的弊端，沒法實時追蹤定位）；

注意padding = 'SAME'和'VALID'的區別

padding = 'SAME' => Height_後 = Height_前/Strides 跟padding無關向上取整

padding = 'VALID'=> Height_後 = (Height_前 - Filter + 1)/Strides 向上取整

4.打印tensorboard流程圖，能夠直觀看到每步操做數據大小的變化；

2. tensorboard

tensorboard就是一個數據結構流程圖的可視化工具，經過tensorboard流程圖，能夠直觀看到神經網絡的每一步操做以及數據流的變化。

操做步驟：

1. 在session會話里加入以下代碼，打印結果會在當前代碼文件相同路徑的tensorboard文件下，默認是

tf.summary.FileWriter("tensorboard/", sess.graph)

2. 在運行裏輸入cmd，而後輸入（前提是安裝好了tensorboard => pip install tensorboard）

tensorboard --logdir=D:\Project\python\myProject\CNN\tensorflow\captchaIdentify\tensorboard --host=127.0.0.1

'D:\Project\python\myProject\CNN\tensorflow\captchaIdentify\tensorboard' 是我生成的tensorboard文件的絕對路徑，你替換成你本身的就能夠了。

正確運行後會顯示 ‘Tensorboard at http://127.0.0.1:6006’，說明tensorboard服務已經起來了，在瀏覽器頁面輸入

http://127.0.0.1:6006便可顯示流程圖。

3.代碼實現（python3.6）

代碼邏輯實現相對比較簡單，在一些重要邏輯實現上，我已作了註釋，若是你們有什麼疑義，能夠留言給我，咱們一塊兒交流。

由於原始圖片數據集太大，很差上傳，你們能夠直接在http://www.cs.toronto.edu/~kriz/cifar.html下載CIFAR-10 python version，

有163M，放在代碼文件同路徑下便可。

cifar放置路徑

start.py

 1 # coding=utf-8
 2 # Disable linter warnings to maintain consistency with tutorial.
 3 # pylint: disable=invalid-name
 4 # pylint: disable=g-bad-import-order
 5 from __future__ import absolute_import  6 from __future__ import division  7 from __future__ import print_function  8 import argparse  9 import sys  10 import tempfile  11 #from tensorflow.examples.tutorials.mnist import input_data
 12 import tensorflow as tf  13 '''
 14  卷積神經網絡實現10類(airplane, automobile, bird, cat, deer, dog, frog, horse, ship, truck)  15  60000張圖片的識別  16  5000次，準確率有 58%；  17  20000次，準確率有 68.89%；  18  相比mnist數字圖片識別準確度低，緣由有：  19  mnist訓練圖片是灰度圖片，紋理簡單，數字的可變性小，而cifar是彩色圖片，紋理複雜，動物可變性大；  20 '''
 21 try:  22     from . import datesets  23 except Exception:  24     import datesets  25 
 26 FLAGS = None  27 
 28 def deepnn(x):  29     with tf.name_scope('reshape'):  30         x_image = tf.reshape(x, [-1, 32, 32, 3])  31     ## 第一層卷積操做 ##
 32     with tf.name_scope('conv1'):  33         W_conv1 = weight_variable([5, 5, 3, 32])  34         b_conv1 = bias_variable([32])  35         h_conv1 = tf.nn.relu(conv2d(x_image, W_conv1) + b_conv1)  36 
 37     with tf.name_scope('pool1'):  38         h_pool1 = max_pool_2x2(h_conv1)  39 
 40     # Second convolutional layer -- maps 32 feature maps to 64.
 41     ## 第二層卷積操做 ##
 42     with tf.name_scope('conv2'):  43         W_conv2 = weight_variable([5, 5, 32, 64])  44         b_conv2 = bias_variable([64])  45         h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)  46 
 47     with tf.name_scope('pool2'):  48         h_pool2 = max_pool_2x2(h_conv2)  49 
 50     ## 第三層全鏈接操做 ##
 51     with tf.name_scope('fc1'):  52         W_fc1 = weight_variable([8 * 8 * 64, 1024])  53         b_fc1 = bias_variable([1024])  54         h_pool2_flat = tf.reshape(h_pool2, [-1, 8 * 8 * 64])  55         h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)  56 
 57     with tf.name_scope('dropout'):  58         keep_prob = tf.placeholder(tf.float32)  59         h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)  60 
 61     ## 第四層輸出操做 ##
 62     with tf.name_scope('fc2'):  63         W_fc2 = weight_variable([1024, 10])  64         b_fc2 = bias_variable([10])  65         y_conv = tf.matmul(h_fc1_drop, W_fc2) + b_fc2  66     return y_conv, keep_prob  67 
 68 def conv2d(x, W):  69     return tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')  70 
 71 def max_pool_2x2(x):  72     return tf.nn.max_pool(x, ksize=[1, 2, 2, 1],  73                           strides=[1, 2, 2, 1], padding='SAME')  74 
 75 def weight_variable(shape):  76     initial = tf.truncated_normal(shape, stddev=0.1)  77     return tf.Variable(initial)  78 
 79 def bias_variable(shape):  80     initial = tf.constant(0.1, shape=shape)  81     return tf.Variable(initial)  82 
 83 def main(_):  84     # Import data
 85     mnist = datesets.read_data_sets(train_dir = '.\\cifar-10-batches-py\\', one_hot=True)  86 
 87     # Create the model
 88     # 聲明一個佔位符，None表示輸入圖片的數量不定，28*28圖片分辨率
 89     x = tf.placeholder(tf.float32, [None, 3072], name='x')  90 
 91     # 類別是0-9總共10個類別，對應輸出分類結果
 92     y_ = tf.placeholder(tf.float32, [None, 10], name='y_')  93     y_conv, keep_prob = deepnn(x)  94     # 經過softmax-loss求交叉熵
 95     with tf.name_scope('loss'):  96         cross_entropy = tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y_conv)  97     # 求均值
 98     cross_entropy = tf.reduce_mean(cross_entropy)  99     # 計算梯度，更新參數值
100     with tf.name_scope('adam_optimizer'): 101         train_step = tf.train.AdamOptimizer(1e-4).minimize(cross_entropy) 102 
103     with tf.name_scope('accuracy'): 104         correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1)) 105         correct_prediction = tf.cast(correct_prediction, tf.float32) 106     accuracy = tf.reduce_mean(correct_prediction) 107 
108    # graph_location = tempfile.mkdtemp()
109    # print('Saving graph to: %s' % graph_location)
110    # train_writer.add_graph(tf.get_default_graph())
111 
112  with tf.Session() as sess: 113         # 打印流程圖
114         writer = tf.summary.FileWriter("tensorboard/", sess.graph) 115  sess.run(tf.global_variables_initializer()) 116         for i in range(20000): 117             batch = mnist.train.next_batch(50) 118             if i % 1000 == 0: 119                 train_accuracy = accuracy.eval(feed_dict={ 120                     x: batch[0], y_: batch[1], keep_prob: 1.0}) 121                 print('step %d, training accuracy %g' % (i, train_accuracy)) 122             train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5}) 123 
124         print('test accuracy %g' % accuracy.eval(feed_dict={ 125             x: mnist.test.images, y_: mnist.test.labels, keep_prob: 1.0})) 126 
127 if __name__ == '__main__': 128     parser = argparse.ArgumentParser() 129     parser.add_argument('--data_dir', type=str, 130                         default='/tmp/tensorflow/mnist/input_data', 131                         help='Directory for storing input data') 132     FLAGS, unparsed = parser.parse_known_args() 133     tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)

View Code

datasets.py

 1 import numpy  2 from tensorflow.python.framework import dtypes  3 from tensorflow.python.framework import random_seed  4 from six.moves import xrange  5 from tensorflow.contrib.learn.python.learn.datasets import base  6 import pickle  7 import os  8 
 9 class DataSet(object):  10     """Container class for a dataset (deprecated).  11 
 12  THIS CLASS IS DEPRECATED. See  13  [contrib/learn/README.md](https://www.tensorflow.org/code/tensorflow/contrib/learn/README.md)  14  for general migration instructions.  15     """
 16     def __init__(self,  17  images,  18  labels,  19                  fake_data=False,  20                  one_hot=False,  21                  dtype=dtypes.float32,  22                  reshape=True,  23                  seed=None):  24         """Construct a DataSet.  25  one_hot arg is used only if fake_data is true. `dtype` can be either  26  `uint8` to leave the input as `[0, 255]`, or `float32` to rescale into  27  `[0, 1]`. Seed arg provides for convenient deterministic testing.  28         """
 29         seed1, seed2 = random_seed.get_seed(seed)  30         # If op level seed is not set, use whatever graph level seed is returned
 31         numpy.random.seed(seed1 if seed is None else seed2)  32         dtype = dtypes.as_dtype(dtype).base_dtype  33         if dtype not in (dtypes.uint8, dtypes.float32):  34             raise TypeError(  35                 'Invalid image dtype %r, expected uint8 or float32' % dtype)  36         if fake_data:  37             self._num_examples = 10000
 38             self.one_hot = one_hot  39         else:  40             assert images.shape[0] == labels.shape[0], (  41                 'images.shape: %s labels.shape: %s' % (images.shape, labels.shape))  42             self._num_examples = images.shape[0]  43 
 44             # Convert shape from [num examples, rows, columns, depth]
 45             # to [num examples, rows*columns] (assuming depth == 1)
 46             if reshape:  47                 assert images.shape[3] == 3
 48                 images = images.reshape(images.shape[0],  49                                         images.shape[1] * images.shape[2] * images.shape[3])  50             if dtype == dtypes.float32:  51                 # Convert from [0, 255] -> [0.0, 1.0].
 52                 images = images.astype(numpy.float32)  53                 images = numpy.multiply(images, 1.0 / 255.0)  54         self._images = images  55         self._labels = labels  56         self._epochs_completed = 0  57         self._index_in_epoch = 0  58 
 59  @property  60     def images(self):  61         return self._images  62 
 63  @property  64     def labels(self):  65         return self._labels  66 
 67  @property  68     def num_examples(self):  69         return self._num_examples  70 
 71  @property  72     def epochs_completed(self):  73         return self._epochs_completed  74 
 75     def next_batch(self, batch_size, fake_data=False, shuffle=True):  76         """Return the next `batch_size` examples from this data set."""
 77         if fake_data:  78             fake_image = [1] * 784
 79             if self.one_hot:  80                 fake_label = [1] + [0] * 9
 81             else:  82                 fake_label = 0  83             return [fake_image for _ in xrange(batch_size)], [  84                 fake_label for _ in xrange(batch_size)  85  ]  86         start = self._index_in_epoch  87         # Shuffle for the first epoch
 88         if self._epochs_completed == 0 and start == 0 and shuffle:  89             perm0 = numpy.arange(self._num_examples)  90  numpy.random.shuffle(perm0)  91             self._images = self.images[perm0]  92             self._labels = self.labels[perm0]  93         # Go to the next epoch
 94         if start + batch_size > self._num_examples:  95             # Finished epoch
 96             self._epochs_completed += 1
 97             # Get the rest examples in this epoch
 98             rest_num_examples = self._num_examples - start  99             images_rest_part = self._images[start:self._num_examples] 100             labels_rest_part = self._labels[start:self._num_examples] 101             # Shuffle the data
102             if shuffle: 103                 perm = numpy.arange(self._num_examples) 104  numpy.random.shuffle(perm) 105                 self._images = self.images[perm] 106                 self._labels = self.labels[perm] 107             # Start next epoch
108             start = 0 109             self._index_in_epoch = batch_size - rest_num_examples 110             end = self._index_in_epoch 111             images_new_part = self._images[start:end] 112             labels_new_part = self._labels[start:end] 113             return numpy.concatenate( 114                 (images_rest_part, images_new_part), axis=0), numpy.concatenate( 115                 (labels_rest_part, labels_new_part), axis=0) 116         else: 117             self._index_in_epoch += batch_size 118             end = self._index_in_epoch 119             return self._images[start:end], self._labels[start:end] 120 
121 def read_data_sets(train_dir, 122                    one_hot=False, 123                    dtype=dtypes.float32, 124                    reshape=True, 125                    validation_size=5000, 126                    seed=None): 127 
128 
129 
130 
131     train_images,train_labels,test_images,test_labels = load_CIFAR10(train_dir) 132     if not 0 <= validation_size <= len(train_images): 133         raise ValueError('Validation size should be between 0 and {}. Received: {}.'
134  .format(len(train_images), validation_size)) 135 
136     validation_images = train_images[:validation_size] 137     validation_labels = train_labels[:validation_size] 138     validation_labels = dense_to_one_hot(validation_labels, 10) 139     train_images = train_images[validation_size:] 140     train_labels = train_labels[validation_size:] 141     train_labels = dense_to_one_hot(train_labels, 10) 142 
143     test_labels = dense_to_one_hot(test_labels, 10) 144 
145     options = dict(dtype=dtype, reshape=reshape, seed=seed) 146     train = DataSet(train_images, train_labels, **options) 147     validation = DataSet(validation_images, validation_labels, **options) 148     test = DataSet(test_images, test_labels, **options) 149 
150     return base.Datasets(train=train, validation=validation, test=test) 151 
152 
153 def load_CIFAR_batch(filename): 154     """ load single batch of cifar """
155     with open(filename, 'rb') as f: 156         datadict = pickle.load(f, encoding='bytes') 157         X = datadict[b'data'] 158         Y = datadict[b'labels'] 159         X = X.reshape(10000, 3, 32, 32).transpose(0,2,3,1).astype("float") 160         Y = numpy.array(Y) 161         return X, Y 162 
163 def load_CIFAR10(ROOT): 164     """ load all of cifar """
165     xs = [] 166     ys = [] 167     for b in range(1,6): 168         f = os.path.join(ROOT, 'data_batch_%d' % (b, )) 169         X, Y = load_CIFAR_batch(f) 170  xs.append(X) 171  ys.append(Y) 172     Xtr = numpy.concatenate(xs) 173     Ytr = numpy.concatenate(ys) 174     del X, Y 175     Xte, Yte = load_CIFAR_batch(os.path.join(ROOT, 'test_batch')) 176     return Xtr, Ytr, Xte, Yte 177 
178 def dense_to_one_hot(labels_dense, num_classes): 179     """Convert class labels from scalars to one-hot vectors."""
180     num_labels = labels_dense.shape[0] 181     index_offset = numpy.arange(num_labels) * num_classes 182     labels_one_hot = numpy.zeros((num_labels, num_classes)) 183     labels_one_hot.flat[index_offset + labels_dense.ravel()] = 1
184     return labels_one_hot