tensorflow生成tfrecord格式的數據

tensorflow生成tfrecord格式的數據

tfrecord格式數據能高效的組織數據，提升訓練時的IO性能
1,2步驟定義了函數，3步驟生成tfrecord格式的數據 1.TF-Feature 將數據（values）封裝於tf.train.Feature

def int64_feature(values):
  """Returns a TF-Feature of int64s. Args: values: A scalar or list of values. Returns: A TF-Feature. """
  if not isinstance(values, (tuple, list)):
    values = [values]
  return tf.train.Feature(int64_list=tf.train.Int64List(value=values))


def bytes_feature(values):
  """Returns a TF-Feature of bytes. Args: values: A string. Returns: A TF-Feature. """
  return tf.train.Feature(bytes_list=tf.train.BytesList(value=[values]))


def float_feature(values):
  """Returns a TF-Feature of floats. Args: values: A scalar of list of values. Returns: A TF-Feature. """
  if not isinstance(values, (tuple, list)):
    values = [values]
  return tf.train.Feature(float_list=tf.train.FloatList(value=values))
複製代碼

2.tf.train.Example

def create_tf_example(features_dict):
    ''' :param features_dict: { img_query_bytes:bytes :return: tf.train.Example '''
    feature_map={  'img_query':bytes_feature(features_dict['img_query_bytes'])
                  }
    return tf.train.Example(features=tf.train.Features(feature=feature_map))
複製代碼

3.寫入tfrecord文件 使用tf.python_io.TFRecordWriter(out_path) 寫入 tf.train.Example

out_path = './dataset/train.record'
    with tf.python_io.TFRecordWriter(out_path) as writer:
            features_dict=dict()
            with tf.gfile.GFile(img_path,'rb') as fid:
                features_dict['img_query_bytes']=fid.read()

            example=create_tf_example(features_dict)
            writer.write(example.SerializeToString())
            if iter_num%1000==0:
                print('done : {} % {}'.format(iter_num,iter_steps))
複製代碼

一個Example結構

dict 表示字典類型

tf.train.Example {
    features: tf.train.Features{
        feature: dict{
            'feature_name':tf.train.Feature{
                int64_list:tf.train.Int64List{value:list}
                bytes_list:tf.train.BytesList{value:list}
                float_list:tf.train.FloatList{value:list}
            }
        }
    }
}
複製代碼