基於MNIST數據集實現2層CNN神經網絡案例實戰-大數據ML樣本集案例實戰

版權聲明:本套技術專欄是做者(秦凱新)平時工做的總結和昇華,經過從真實商業環境抽取案例進行總結和分享,並給出商業應用的調優建議和集羣環境容量規劃等內容,請持續關注本套博客。QQ郵箱地址:1120746959@qq.com,若有任何學術交流,可隨時聯繫。git

1 神經網絡基本結構定義

  • 28*28=784個像素點,第一層神經元256,第二層神經元128

2 基本神經網絡構建

  • 變量初始化網絡

    import numpy as np
      import tensorflow as tf
      import matplotlib.pyplot as plt
      import input_data
      mnist = input_data.read_data_sets('data/', one_hot=True)
      Extracting data/train-images-idx3-ubyte.gz
      Extracting data/train-labels-idx1-ubyte.gz
      Extracting data/t10k-images-idx3-ubyte.gz
      Extracting data/t10k-labels-idx1-ubyte.gz
    
      # NETWORK TOPOLOGIES
      #第一層神經元
      n_hidden_1 = 256 
      #第二層神經元
      n_hidden_2 = 128
      #28*28 784像素點
      n_input    = 784 
      # 類別10
      n_classes  = 10  
      
      # INPUTS AND OUTPUTS
      x = tf.placeholder("float", [None, n_input])
      y = tf.placeholder("float", [None, n_classes])
          
      # NETWORK PARAMETERS
      stddev = 0.1
      #初始化
      weights = {
          'w1': tf.Variable(tf.random_normal([n_input, n_hidden_1], stddev=stddev)),
          'w2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2], stddev=stddev)),
          'out': tf.Variable(tf.random_normal([n_hidden_2, n_classes], stddev=stddev))
      }
      #初始化
      biases = {
          'b1': tf.Variable(tf.random_normal([n_hidden_1])),
          'b2': tf.Variable(tf.random_normal([n_hidden_2])),
          'out': tf.Variable(tf.random_normal([n_classes]))
      }
      print ("NETWORK READY")
    複製代碼
  • 前向傳播(每一層增長激活函數sigmoid,最後一層不加sigmoid)dom

    def multilayer_perceptron(_X, _weights, _biases):
          layer_1 = tf.nn.sigmoid(tf.add(tf.matmul(_X, _weights['w1']), _biases['b1'])) 
          layer_2 = tf.nn.sigmoid(tf.add(tf.matmul(layer_1, _weights['w2']), _biases['b2']))
          return (tf.matmul(layer_2, _weights['out']) + _biases['out'])
    複製代碼
  • 損失變量和優化器定義ide

  • softmax_cross_entropy_with_logits交叉熵損失函數(參數pred預測值),reduce_mean除以樣本總數。函數

  • GradientDescentOptimizer採用梯度降低優化求解oop

    # PREDICTION
      pred = multilayer_perceptron(x, weights, biases)
      
      # LOSS AND OPTIMIZER
      cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(pred, y)) 
      optm = tf.train.GradientDescentOptimizer(learning_rate=0.001).minimize(cost) 
      
      #準確率求解
      corr = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))    
      accr = tf.reduce_mean(tf.cast(corr, "float"))
      
      # INITIALIZER
      init = tf.global_variables_initializer()
      print ("FUNCTIONS READY")
    複製代碼
  • 按照Batch迭代優化

    training_epochs = 20
      batch_size      = 100
      display_step    = 4
      # LAUNCH THE GRAPH
      sess = tf.Session()
      sess.run(init)
      # OPTIMIZE
      for epoch in range(training_epochs):
          avg_cost = 0.
          total_batch = int(mnist.train.num_examples/batch_size)
          
          # ITERATION(按照Batch迭代,每一次迭代100)
          for i in range(total_batch):
              batch_xs, batch_ys = mnist.train.next_batch(batch_size)
              #填充值
              feeds = {x: batch_xs, y: batch_ys}
              #sess.run(模型訓練)
              sess.run(optm, feed_dict=feeds)
              avg_cost += sess.run(cost, feed_dict=feeds)
          avg_cost = avg_cost / total_batch
          # DISPLAY
          if (epoch+1) % display_step == 0:
              print ("Epoch: %03d/%03d cost: %.9f" % (epoch, training_epochs, avg_cost))
              feeds = {x: batch_xs, y: batch_ys}
              
              #sess.run(準確率求解)
              train_acc = sess.run(accr, feed_dict=feeds)
              print ("TRAIN ACCURACY: %.3f" % (train_acc))
              feeds = {x: mnist.test.images, y: mnist.test.labels}
              test_acc = sess.run(accr, feed_dict=feeds)
              print ("TEST ACCURACY: %.3f" % (test_acc))
      print ("OPTIMIZATION FINISHED")
    複製代碼

3 CNN神經網絡

  • 變量初始化spa

    import numpy as np
      import tensorflow as tf
      import matplotlib.pyplot as plt
      import input_data
      mnist = input_data.read_data_sets('data/', one_hot=True)
      trainimg   = mnist.train.images
      trainlabel = mnist.train.labels
      testimg    = mnist.test.images
      testlabel  = mnist.test.labels
      print ("MNIST ready")
      n_input  = 784
      
      
      n_output = 10
      ##wc1  [3, 3, 1, 64]   中3表示Filter寬度和深度,1表示深度,64表示outchannl最後獲得64張特徵圖。  14*14*128
      ##wc2 [3, 3, 64, 128] 中3表示Filter寬度和深度,1表示深度,64表示輸入64張特徵圖,輸出128張特徵圖。7*7*128   輸出1024向量
      ## 卷積層沒有減小挺像的大小。
      ## polling層把圖像減小到一半
      ## wd1 輸入7*7*128 輸出1024向量
      weights  = {
              'wc1': tf.Variable(tf.random_normal([3, 3, 1, 64], stddev=0.1)),
              'wc2': tf.Variable(tf.random_normal([3, 3, 64, 128], stddev=0.1)),
              'wd1': tf.Variable(tf.random_normal([7*7*128, 1024], stddev=0.1)),
              'wd2': tf.Variable(tf.random_normal([1024, n_output], stddev=0.1))
          }
      biases   = {
              'bc1': tf.Variable(tf.random_normal([64], stddev=0.1)),
              'bc2': tf.Variable(tf.random_normal([128], stddev=0.1)),
              'bd1': tf.Variable(tf.random_normal([1024], stddev=0.1)),
              'bd2': tf.Variable(tf.random_normal([n_output], stddev=0.1))
          }
    複製代碼
  • help方法的使用3d

  • 前向傳播rest

    def conv_basic(_input, _w, _b, _keepratio):
          # INPUT(轉換格式,轉換成4維 【n,h,w,c】 -1 batchSize大小,能夠讓TF推斷 ,輸出通道深度爲1)
          _input_r = tf.reshape(_input, shape=[-1, 28, 28, 1])
          
          # 第一層(nn模塊CNN, RNN)(conv2 中 strides ->【n,h,w,c】表示在各個上面滑動窗的大小 
          # padding 兩種選擇 SAME=>滑動窗不夠時填充,Valid不填充)。
          _conv1 = tf.nn.conv2d(_input_r, _w['wc1'], strides=[1, 1, 1, 1], padding='SAME')
          
          #_mean, _var = tf.nn.moments(_conv1, [0, 1, 2])
          #_conv1 = tf.nn.batch_normalization(_conv1, _mean, _var, 0, 1, 0.0001)
          # 激活函數relu
          _conv1 = tf.nn.relu(tf.nn.bias_add(_conv1, _b['bc1']))
          
          # max_pool層,ksize表示Window -1 batchSize大小,2*2窗口 1表示,輸出通道深度爲1
          _pool1 = tf.nn.max_pool(_conv1, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
          
          # dropout不讓全部的神經元參與計算比例
          _pool_dr1 = tf.nn.dropout(_pool1, _keepratio)
          
          # 第二層
          _conv2 = tf.nn.conv2d(_pool_dr1, _w['wc2'], strides=[1, 1, 1, 1], padding='SAME')
          #_mean, _var = tf.nn.moments(_conv2, [0, 1, 2])
          #_conv2 = tf.nn.batch_normalization(_conv2, _mean, _var, 0, 1, 0.0001)
          
          _conv2 = tf.nn.relu(tf.nn.bias_add(_conv2, _b['bc2']))
          _pool2 = tf.nn.max_pool(_conv2, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')
          _pool_dr2 = tf.nn.dropout(_pool2, _keepratio)
          
          #全鏈接層
          # VECTORIZE
          _dense1 = tf.reshape(_pool_dr2, [-1, _w['wd1'].get_shape().as_list()[0]])
          
          # FULLY CONNECTED LAYER 1
          _fc1 = tf.nn.relu(tf.add(tf.matmul(_dense1, _w['wd1']), _b['bd1']))
          _fc_dr1 = tf.nn.dropout(_fc1, _keepratio)
          
          # FULLY CONNECTED LAYER 2
          _out = tf.add(tf.matmul(_fc_dr1, _w['wd2']), _b['bd2'])
          # RETURN
          out = { 'input_r': _input_r, 'conv1': _conv1, 'pool1': _pool1, 'pool1_dr1': _pool_dr1,
              'conv2': _conv2, 'pool2': _pool2, 'pool_dr2': _pool_dr2, 'dense1': _dense1,
              'fc1': _fc1, 'fc_dr1': _fc_dr1, 'out': _out
          }
          return out
      print ("CNN READY")
    複製代碼
  • 模型訓練和評估

    a = tf.Variable(tf.random_normal([3, 3, 1, 64], stddev=0.1))
      print (a)
      a = tf.Print(a, [a], "a: ")
      init = tf.global_variables_initializer()
      sess = tf.Session()
      sess.run(init)
      
      Tensor("Variable_28/read:0", shape=(3, 3, 1, 64), dtype=float32)
      
      #print (help(tf.nn.conv2d))
      print (help(tf.nn.max_pool))
      
      x = tf.placeholder(tf.float32, [None, n_input])
      y = tf.placeholder(tf.float32, [None, n_output])
      keepratio = tf.placeholder(tf.float32)
      
      # FUNCTIONS
      
      _pred = conv_basic(x, weights, biases, keepratio)['out']
      cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(_pred, y))
      optm = tf.train.AdamOptimizer(learning_rate=0.001).minimize(cost)
      _corr = tf.equal(tf.argmax(_pred,1), tf.argmax(y,1)) 
      accr = tf.reduce_mean(tf.cast(_corr, tf.float32)) 
      init = tf.global_variables_initializer()
          
      # SAVER
      print ("GRAPH READY")
      
      sess = tf.Session()
      sess.run(init)
      
      training_epochs = 15
      batch_size      = 16
      display_step    = 1
      for epoch in range(training_epochs):
          avg_cost = 0.
          #total_batch = int(mnist.train.num_examples/batch_size)
          total_batch = 10
          # Loop over all batches
          for i in range(total_batch):
              batch_xs, batch_ys = mnist.train.next_batch(batch_size)
              # Fit training using batch data
              sess.run(optm, feed_dict={x: batch_xs, y: batch_ys, keepratio:0.7})
              # Compute average loss
              avg_cost += sess.run(cost, feed_dict={x: batch_xs, y: batch_ys, keepratio:1.})/total_batch
      
          # Display logs per epoch step
          if epoch % display_step == 0: 
              print ("Epoch: %03d/%03d cost: %.9f" % (epoch, training_epochs, avg_cost))
              train_acc = sess.run(accr, feed_dict={x: batch_xs, y: batch_ys, keepratio:1.})
              print (" Training accuracy: %.3f" % (train_acc))
              #test_acc = sess.run(accr, feed_dict={x: testimg, y: testlabel, keepratio:1.})
              #print (" Test accuracy: %.3f" % (test_acc))
      
      print ("OPTIMIZATION FINISHED")
    複製代碼
  • 結果展現

    Epoch: 000/015 cost: 30.928401661
       Training accuracy: 0.500
      Epoch: 001/015 cost: 12.954609606
       Training accuracy: 0.700
      Epoch: 002/015 cost: 10.392489696
       Training accuracy: 0.700
      Epoch: 003/015 cost: 7.254891634
       Training accuracy: 0.800
      Epoch: 004/015 cost: 4.977767670
       Training accuracy: 0.900
      Epoch: 005/015 cost: 5.414173813
       Training accuracy: 0.600
      Epoch: 006/015 cost: 3.057567777
       Training accuracy: 0.700
      Epoch: 007/015 cost: 4.929724103
       Training accuracy: 0.600
      Epoch: 008/015 cost: 3.192437538
       Training accuracy: 0.600
      Epoch: 009/015 cost: 3.224479928
       Training accuracy: 0.800
      Epoch: 010/015 cost: 2.720530389
       Training accuracy: 0.400
      Epoch: 011/015 cost: 3.000342276
       Training accuracy: 0.800
      Epoch: 012/015 cost: 0.639763238
       Training accuracy: 1.000
      Epoch: 013/015 cost: 1.897303332
       Training accuracy: 0.900
      Epoch: 014/015 cost: 2.295500937
       Training accuracy: 0.800
      OPTIMIZATION FINISHED
    複製代碼

4 模型持久化與加載

import tensorflow as tf

v1 = tf.Variable(tf.random_normal([1,2]), name="v1")
v2 = tf.Variable(tf.random_normal([2,3]), name="v2")
init_op = tf.global_variables_initializer()
saver = tf.train.Saver()
with tf.Session() as sess:
    sess.run(init_op)
    print ("V1:",sess.run(v1))  
    print ("V2:",sess.run(v2))
    saver_path = saver.save(sess, "save/model.ckpt")
    print ("Model saved in file: ", saver_path) 

V1: [[-0.61912751  0.10767912]]
V2: [[ 0.10039134 -1.51745009 -0.61548245]
 [ 0.6146487   0.66980863 -1.00977123]]
Model saved in file:  save/model.ckpt

import tensorflow as tf
v1 = tf.Variable(tf.random_normal([1,2]), name="v1")
v2 = tf.Variable(tf.random_normal([2,3]), name="v2")
saver = tf.train.Saver()

with tf.Session() as sess:
    saver.restore(sess, "save/model.ckpt")
    print ("V1:",sess.run(v1))  
    print ("V2:",sess.run(v2))
    print ("Model restored")

V1: [[-0.61912751  0.10767912]]
V2: [[ 0.10039134 -1.51745009 -0.61548245]
 [ 0.6146487   0.66980863 -1.00977123]]
Model restored
複製代碼

總結

基本的神經網絡案例,在於真正的入門神經網絡的構建。

相關文章
相關標籤/搜索