Tensorflow Basic Concepts

1. How tensorflow works

In order to run Tensorflow, we need two steps:

First, construct a Graph

Second, compute along the Graph in a Session.

For the second step, we can categorize it to train stage and execution stage.

 

2. Basic Tensorflow concepts

• Tensor
• Variable
• Placeholder
• Session

Tensor is just vector or matrix.

import tensorflow as tf
a = tf.zero(shape=[2,2]) # we get a tensor that is 2x2

Variable is similar to variables that we use in other programming languages. However, most of the time, it refers to the training parameters, such as Weight and Bias in Tensorflow.

W = tf.Variable(tf.zeros(shape=[1,2]))
B = tf.Variable(tf.zeros(shape=[1,2]))

The variables should be initialized by hand in a Session, for example:

init = tf.initialize_all_variables()

with tf.Session() as session:
    session.run(init)

Placeholder

We use placeholder to declare the shape a input tensors, but we can not only suply meaningful value during execution stage.

Train_x = tf.placeholder(tf.float32,[None, n_input, tensor_size],name='input')
Train_y = tf.placeholder(tf.float32,[None, n_classes],name='input')

Session

Session is used by Graph to conduct real computation. We can use Session to train, predict models and fetch Variables from the models and check their values.

 

3. Create a Model

# build the Graph
x = tf.placeholder(tf.float32, [None, 784]) # placeholder for input: x
y = tf.placeholder(tf.float32, [None, 10])  # placeholder for input: y
W = tf.Variable(tf.zeros([784, 10]))           # weight
b = tf.Variable(tf.zeros([10]))                    # bias
a = tf.nn.softmax(tf.matmul(x, W) + b)      # output of the model

#define loss function
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y * tf.log(a), reduction_indices=[1])) 
optimizer = tf.train.GradientDescentOptimizer(0.5) # learning rate=0.5
train = optimizer.minimize(cross_entropy)  # mininize loss

#check accuracy

correct_prediction = tf.equal(tf.argmax(a, 1), tf.argmax(y, 1)) # compare real target and prediction result accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32)) # caculate the average accuracy

4. Complete code Example

"""A very simple MNIST classifier.
See extensive documentation at
http://tensorflow.org/tutorials/mnist/beginners/index.md
"""
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

# Import data
from tensorflow.examples.tutorials.mnist import input_data

import tensorflow as tf

flags = tf.app.flags
FLAGS = flags.FLAGS
flags.DEFINE_string('data_dir', '/tmp/data/', 'Directory for storing data') # 把數據放在/tmp/data文件夾中

mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)   # 讀取數據集


# 創建抽象模型
x = tf.placeholder(tf.float32, [None, 784]) # 佔位符
y = tf.placeholder(tf.float32, [None, 10])
W = tf.Variable(tf.zeros([784, 10]))
b = tf.Variable(tf.zeros([10]))
a = tf.nn.softmax(tf.matmul(x, W) + b)

# 定義損失函數和訓練方法
cross_entropy = tf.reduce_mean(-tf.reduce_sum(y * tf.log(a), reduction_indices=[1]))  # 損失函數爲交叉熵
optimizer = tf.train.GradientDescentOptimizer(0.5) # 梯度降低法，學習速率爲0.5
train = optimizer.minimize(cross_entropy) # 訓練目標：最小化損失函數

# Test trained model
correct_prediction = tf.equal(tf.argmax(a, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

# Train
sess = tf.InteractiveSession()      # 創建交互式會話
tf.initialize_all_variables().run()
for i in range(1000):
    batch_xs, batch_ys = mnist.train.next_batch(100)
    train.run({x: batch_xs, y: batch_ys})
print(sess.run(accuracy,feed_dict={x:mnist.test.images,y:mnist.test.labels}))