AI安全對抗賽第二名方案分享

時間 2020-04-09

原文原文鏈接

本項目爲AI安全對抗賽第二名方案介紹，可完美復現。團隊名爲:我不和大家玩了，隊伍成員一人，姓名張鑫，在讀於西安電子科技大學，目前研二，初賽排名第6，提交次數58次。決賽排名第2，提交次數84次。python

賽題背景

目前人工智能和機器學習技術被普遍應用在人機交互、推薦系統、安全防禦等各個領域，其受攻擊的可能性以及是否具有強抗打擊能力備受業界關注。具體場景包括語音，圖像識別，信用評估，防止欺詐，過濾惡意郵件，抵抗惡意代碼攻擊，網絡攻擊等等。攻擊者也試圖經過各類手段繞過，或直接對AI模型進行攻擊達到對抗目的。在人機交互這一環節，隨着移動設備的普及，語音、圖像做爲新興的人機輸入手段，其便捷和實用性被大衆所歡迎。所以圖像識別的準確性對人工智能產業相當重要。這一環節也是最容易被攻擊者利用，經過對數據源的細微修改，在用戶感知不到的狀況下，使機器作出了錯誤的操做。這種方法會致使AI系統被入侵、錯誤命令被執行，執行後的連鎖反應會形成的嚴重後果。git

本次競賽的題目和數據由百度安所有、百度大數據研究院提供，競賽平臺AI Studio由百度AI技術生態部提供。期待參賽者們可以以此爲契機，學習對抗樣本理論知識並提高深度學習工程實踐能力。歡迎全球範圍開發者積極參與，鼓勵高校教師積極參與指導。算法

賽題描述

初賽：初賽中，選手經過對指定圖像添加擾動，使目標模型（Target Model）（一個爲ResNeXt50模型並公開模型結構與參數（白盒）；一個爲MobileNetV2模型並公開模型結構與參數（白盒）；一個不公開模型結構與參數（黑盒）。）分類錯誤，例如對於一張分類爲A的圖片，目標模型只要判別擾動後的樣本不爲A，便可斷定成功。同時以生成擾動量越小越優。
複賽：複賽選手的目標與初賽相同：利用給定，將指定的120張圖片樣本生成爲攻擊樣本，主辦方根據選手提供的攻擊樣本在後臺使用上述5個Target Model（一個是與初賽相同的ResNeXt50白盒模型（白盒），一個是人工加固的模型（灰盒），另外三個均爲黑盒模型，其中包括由AutoDL技術訓練的模型。）進行評估，只要使Target Model分類結果與Label不一致，則斷定爲攻擊成功。樣本攻擊成功數越多、擾動越小，得分越高。

一熟悉baseline

方案將在baseline的基礎上增長函數進行修改，所以咱們先瀏覽一遍baseline。安全

1.1 熟悉baseline目錄結構

In[ ]

#解壓代碼壓縮包
import zipfile
tar = zipfile.ZipFile('/home/aistudio/data/data19725/attack_by_xin.zip','r')
tar.extractall()

In[ ]

cd baidu_attack_by_xin/

/home/aistudio/baidu_attack_by_xin

baseline存在如下目錄網絡

attack 存放核心算法代碼
models 存放模型結構定意
models_parameters 存放模型參數
input_imgae 存放120張輸入圖片，與標籤文件
output_image 存放輸出圖片
utils.py 定義了一些經常使用工具，好比讀入圖片處理，打印參數等
attack_FGSM.py 算法主函數，包含模型定義，算法調用。此文件將直接在下面notebook中展現

note： 加粗部分爲個人方案將要修改的部分app

1.2 baseline瀏覽

代碼大致分爲如下幾部分：dom

定義模型，導入參數。機器學習

採用模型爲ResNeXt50_32x4d，損失函數爲交叉熵，與訓練分類模型無異。區別在於如何更新參數，這點將在後文FGSM算法中介紹。ide
依次讀入圖片調用FGSM算法，生成對抗樣本函數

In[ ]

#coding=utf-8

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import argparse
import functools
import numpy as np
import paddle.fluid as fluid

#加載自定義文件
import models
##################################################
##################################################
#此處爲導入算法FGSM和PGD
#我方案中函數也將定義在attack/attack_pp.py中
from attack.attack_pp import FGSM, PGD
##################################################
##################################################
from utils import init_prog, save_adv_image, process_img, tensor2img, calc_mse, add_arguments, print_arguments

path = "/home/aistudio/baidu_attack_by_xin/"
######Init args
image_shape = [3,224,224]
class_dim=121
input_dir = path + "input_image/"
output_dir = path +  "output_image/"
model_name="ResNeXt50_32x4d"
pretrained_model= path + "models_parameters/86.45+88.81ResNeXt50_32x4d"

val_list = 'val_list.txt'
use_gpu=True

######Attack graph
adv_program=fluid.Program()
#完成初始化
with fluid.program_guard(adv_program):
    input_layer = fluid.layers.data(name='image', shape=image_shape, dtype='float32')
    #設置爲能夠計算梯度
    input_layer.stop_gradient=False

    # model definition
    model = models.__dict__[model_name]()
    out_logits = model.net(input=input_layer, class_dim=class_dim)
    out = fluid.layers.softmax(out_logits)

    place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
    exe = fluid.Executor(place)
    exe.run(fluid.default_startup_program())

    #記載模型參數
    fluid.io.load_persistables(exe, pretrained_model)

#設置adv_program的BN層狀態
init_prog(adv_program)

#建立測試用評估模式
eval_program = adv_program.clone(for_test=True)

#定義梯度
with fluid.program_guard(adv_program):
    label = fluid.layers.data(name="label", shape=[1] ,dtype='int64')
    loss = fluid.layers.cross_entropy(input=out, label=label)
    gradients = fluid.backward.gradients(targets=loss, inputs=[input_layer])[0]

######Inference
def inference(img):
    fetch_list = [out.name]

    result = exe.run(eval_program,
                     fetch_list=fetch_list,
                     feed={ 'image':img })
    result = result[0][0]
    pred_label = np.argmax(result)
    pred_score = result[pred_label].copy()
    return pred_label, pred_score

######FGSM attack
#untarget attack
def attack_nontarget_by_FGSM(img, src_label):
    pred_label = src_label

    step = 8.0/256.0
    eps = 32.0/256.0
    while pred_label == src_label:
        #生成對抗樣本
        adv=FGSM(adv_program=adv_program,eval_program=eval_program,gradients=gradients,o=img,
                 input_layer=input_layer,output_layer=out,step_size=step,epsilon=eps,
                 isTarget=False,target_label=0,use_gpu=use_gpu)

        pred_label, pred_score = inference(adv)
        step *= 2
        if step > eps:
            break

    print("Test-score: {0}, class {1}".format(pred_score, pred_label))

    adv_img=tensor2img(adv)
    return adv_img

####### Main #######
def get_original_file(filepath):
    with open(filepath, 'r') as cfile:
        full_lines = [line.strip() for line in cfile]
    cfile.close()
    original_files = []
    for line in full_lines:
        label, file_name = line.split()
        original_files.append([file_name, int(label)])
    return original_files

def gen_adv():
    ########若是你沒有頭緒能夠從這部分看起#######################
    mse = 0
    original_files = get_original_file(input_dir + val_list)

    for filename, label in original_files:
        img_path = input_dir + filename
        print("Image: {0} ".format(img_path))
        ##讀入圖像，轉換維度，歸一化##########
        img=process_img(img_path)
        ####將圖像輸入attack_nontarget_by_FGSM函數，獲得被攻擊後的圖像#######
        adv_img = attack_nontarget_by_FGSM(img, label)
        image_name, image_ext = filename.split('.')
        ##Save adversarial image(.png) 保存圖像
        save_adv_image(adv_img, output_dir+image_name+'.jpg')

        org_img = tensor2img(img)
        ##對比攻擊圖像與原圖像的差別，計算mse
        score = calc_mse(org_img, adv_img)
        mse += score
    print("ADV {} files, AVG MSE: {} ".format(len(original_files), mse/len(original_files)))


gen_adv()

Image: /home/aistudio/baidu_attack_by_xin/input_image/n02085620_10074.jpg 
Non-Targeted attack target_label=o_label=1
Non-Targeted attack target_label=o_label=1
Non-Targeted attack target_label=o_label=1
Test-score: 0.1829851120710373, class 1
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02085782_1039.jpg 
Non-Targeted attack target_label=o_label=2
Non-Targeted attack target_label=o_label=2
Non-Targeted attack target_label=o_label=2
Test-score: 0.7980572581291199, class 2
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02085936_10130.jpg 
Non-Targeted attack target_label=o_label=3
Test-score: 0.558245837688446, class 54
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02086079_10600.jpg 
Non-Targeted attack target_label=o_label=4
Test-score: 0.4213048815727234, class 5
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02086240_1059.jpg 
Non-Targeted attack target_label=o_label=5
Non-Targeted attack target_label=o_label=5
Non-Targeted attack target_label=o_label=5
Test-score: 0.6555399894714355, class 5
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02086646_1002.jpg 
Non-Targeted attack target_label=o_label=6
Non-Targeted attack target_label=o_label=6
Test-score: 0.13172577321529388, class 68
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02086910_1048.jpg 
Non-Targeted attack target_label=o_label=7
Test-score: 0.24971207976341248, class 1
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02087046_1206.jpg 
Non-Targeted attack target_label=o_label=8
Test-score: 0.7929812669754028, class 108
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02087394_11337.jpg 
Non-Targeted attack target_label=o_label=9
Test-score: 0.33581840991973877, class 93
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02088094_1003.jpg 
Non-Targeted attack target_label=o_label=10
Non-Targeted attack target_label=o_label=10
Non-Targeted attack target_label=o_label=10
Test-score: 0.24336178600788116, class 10
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02088238_10013.jpg 
Non-Targeted attack target_label=o_label=11
Non-Targeted attack target_label=o_label=11
Test-score: 0.18260332942008972, class 1
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02088364_10108.jpg 
Non-Targeted attack target_label=o_label=12
Test-score: 0.4022131860256195, class 6
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02088466_10083.jpg 
Non-Targeted attack target_label=o_label=13
Non-Targeted attack target_label=o_label=13
Test-score: 0.17899833619594574, class 96
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02088632_101.jpg 
Non-Targeted attack target_label=o_label=14
Non-Targeted attack target_label=o_label=14
Non-Targeted attack target_label=o_label=14
Test-score: 0.748593807220459, class 14
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02089078_1064.jpg 
Non-Targeted attack target_label=o_label=15
Test-score: 0.8527135848999023, class 13
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02089867_1029.jpg 
Non-Targeted attack target_label=o_label=16
Test-score: 0.9962345957756042, class 17
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02089973_1066.jpg 
Non-Targeted attack target_label=o_label=17
Test-score: 0.9775213003158569, class 16
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02090379_1272.jpg 
Non-Targeted attack target_label=o_label=18
Test-score: 0.9822365045547485, class 13
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02090622_10343.jpg 
Non-Targeted attack target_label=o_label=19
Test-score: 0.8759220242500305, class 22
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02090721_1292.jpg 
Non-Targeted attack target_label=o_label=20
Test-score: 0.9765266180038452, class 27
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02091032_10079.jpg 
Non-Targeted attack target_label=o_label=21
Test-score: 0.8322737812995911, class 22
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02091134_10107.jpg 
Non-Targeted attack target_label=o_label=22
Test-score: 0.0860852524638176, class 102
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02091244_1000.jpg 
Non-Targeted attack target_label=o_label=23
Non-Targeted attack target_label=o_label=23
Non-Targeted attack target_label=o_label=23
Test-score: 0.8737558126449585, class 23
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02091467_1110.jpg 
Non-Targeted attack target_label=o_label=24
Non-Targeted attack target_label=o_label=24
Test-score: 0.12693266570568085, class 99
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02091635_1319.jpg 
Non-Targeted attack target_label=o_label=25
Test-score: 0.16966181993484497, class 32
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02091831_10576.jpg 
Non-Targeted attack target_label=o_label=26
Test-score: 0.26041099429130554, class 8
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02092002_10699.jpg 
Non-Targeted attack target_label=o_label=27
Test-score: 0.9970411658287048, class 20
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02092339_1100.jpg 
Non-Targeted attack target_label=o_label=28
Test-score: 0.22998812794685364, class 59
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02093256_11023.jpg 
Non-Targeted attack target_label=o_label=29
Test-score: 0.3322082757949829, class 9
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02093428_10947.jpg 
Non-Targeted attack target_label=o_label=30
Non-Targeted attack target_label=o_label=30
Test-score: 0.4645201563835144, class 117
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02093647_1037.jpg 
Non-Targeted attack target_label=o_label=31
Non-Targeted attack target_label=o_label=31
Test-score: 0.09869368374347687, class 33
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02093754_1062.jpg 
Non-Targeted attack target_label=o_label=32
Test-score: 0.35046255588531494, class 111
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02093859_1003.jpg 
Non-Targeted attack target_label=o_label=33
Non-Targeted attack target_label=o_label=33
Test-score: 0.23188208043575287, class 83
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02093991_1026.jpg 
Non-Targeted attack target_label=o_label=34
Test-score: 0.6639878749847412, class 35
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02094114_1173.jpg 
Non-Targeted attack target_label=o_label=35
Test-score: 0.9812427759170532, class 20
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02094258_1004.jpg 
Non-Targeted attack target_label=o_label=36
Test-score: 0.990171492099762, class 35
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02094433_10126.jpg 
Non-Targeted attack target_label=o_label=37
Test-score: 0.7961386442184448, class 43
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02095314_1033.jpg 
Non-Targeted attack target_label=o_label=38
Test-score: 0.22626255452632904, class 17
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02095570_1031.jpg 
Non-Targeted attack target_label=o_label=39
Test-score: 0.3102056682109833, class 46
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02095889_1003.jpg 
Non-Targeted attack target_label=o_label=40
Non-Targeted attack target_label=o_label=40
Non-Targeted attack target_label=o_label=40
Test-score: 0.2282048910856247, class 38
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02096051_1110.jpg 
Non-Targeted attack target_label=o_label=41
Test-score: 0.6431247591972351, class 39
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02096177_10031.jpg 
Non-Targeted attack target_label=o_label=42
Test-score: 0.7510316967964172, class 116
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02096294_1111.jpg 
Non-Targeted attack target_label=o_label=43
Non-Targeted attack target_label=o_label=43
Non-Targeted attack target_label=o_label=43
Test-score: 0.1718287616968155, class 43
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02096437_1055.jpg 
Non-Targeted attack target_label=o_label=44
Test-score: 0.1869039088487625, class 25
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02096585_10604.jpg 
Non-Targeted attack target_label=o_label=45
Test-score: 0.517105758190155, class 8
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02097047_1412.jpg 
Non-Targeted attack target_label=o_label=46
Non-Targeted attack target_label=o_label=46
Test-score: 0.6849876642227173, class 48
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02097130_1193.jpg 
Non-Targeted attack target_label=o_label=47
Test-score: 0.41658449172973633, class 33
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02097209_1038.jpg 
Non-Targeted attack target_label=o_label=48
Test-score: 0.16146445274353027, class 47
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02097298_10676.jpg 
Non-Targeted attack target_label=o_label=49
Test-score: 0.609655499458313, class 33
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02097474_1070.jpg 
Non-Targeted attack target_label=o_label=50
Test-score: 0.9852504134178162, class 54
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02097658_1018.jpg 
Non-Targeted attack target_label=o_label=51
Test-score: 0.997583270072937, class 43
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02098105_1078.jpg 
Non-Targeted attack target_label=o_label=52
Test-score: 0.2030351161956787, class 50
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02098286_1009.jpg 
Non-Targeted attack target_label=o_label=53
Test-score: 0.43629899621009827, class 31
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02098413_11385.jpg 
Non-Targeted attack target_label=o_label=54
Test-score: 0.2666652798652649, class 50
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02099267_1018.jpg 
Non-Targeted attack target_label=o_label=55
Test-score: 0.24120940268039703, class 64
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02099429_1039.jpg 
Non-Targeted attack target_label=o_label=56
Test-score: 0.5451071858406067, class 105
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02099601_100.jpg 
Non-Targeted attack target_label=o_label=57
Non-Targeted attack target_label=o_label=57
Test-score: 0.2970142066478729, class 63
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02099712_1150.jpg 
Non-Targeted attack target_label=o_label=58
Test-score: 0.8002893924713135, class 29
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02099849_1068.jpg 
Non-Targeted attack target_label=o_label=59
Test-score: 0.4128360450267792, class 70
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02100236_1244.jpg 
Non-Targeted attack target_label=o_label=60
Test-score: 0.5225626826286316, class 70
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02100583_10249.jpg 
Non-Targeted attack target_label=o_label=61
Test-score: 0.7810189127922058, class 59
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02100735_10064.jpg 
Non-Targeted attack target_label=o_label=62
Test-score: 0.20395173132419586, class 12
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02100877_1062.jpg 
Non-Targeted attack target_label=o_label=63
Test-score: 0.1305573433637619, class 62
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02101006_135.jpg 
Non-Targeted attack target_label=o_label=64
Test-score: 0.2646207809448242, class 96
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02101388_10017.jpg 
Non-Targeted attack target_label=o_label=65
Non-Targeted attack target_label=o_label=65
Non-Targeted attack target_label=o_label=65
Test-score: 0.8180195689201355, class 65
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02101556_1116.jpg 
Non-Targeted attack target_label=o_label=66
Test-score: 0.966513991355896, class 69
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02102040_1055.jpg 
Non-Targeted attack target_label=o_label=67
Test-score: 0.9941025376319885, class 62
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02102177_1160.jpg 
Non-Targeted attack target_label=o_label=68
Non-Targeted attack target_label=o_label=68
Non-Targeted attack target_label=o_label=68
Test-score: 0.24618981778621674, class 68
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02102318_10000.jpg 
Non-Targeted attack target_label=o_label=69
Test-score: 0.4054173231124878, class 68
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02102480_101.jpg 
Non-Targeted attack target_label=o_label=70
Test-score: 0.3680818974971771, class 69
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02102973_1037.jpg 
Non-Targeted attack target_label=o_label=71
Test-score: 0.7708333730697632, class 116
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02104029_1075.jpg 
Non-Targeted attack target_label=o_label=72
Test-score: 0.10377514362335205, class 58
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02104365_10071.jpg 
Non-Targeted attack target_label=o_label=73
Test-score: 0.2432803213596344, class 64
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02105056_1165.jpg 
Non-Targeted attack target_label=o_label=74
Test-score: 0.40509727597236633, class 55
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02105162_10076.jpg 
Non-Targeted attack target_label=o_label=75
Test-score: 0.15848565101623535, class 85
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02105251_1588.jpg 
Non-Targeted attack target_label=o_label=76
Test-score: 0.10340812057256699, class 52
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02105412_1159.jpg 
Non-Targeted attack target_label=o_label=77
Test-score: 0.34471365809440613, class 87
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02105505_1018.jpg 
Non-Targeted attack target_label=o_label=78
Test-score: 0.5453677177429199, class 106
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02105641_10051.jpg 
Non-Targeted attack target_label=o_label=79
Test-score: 0.29593008756637573, class 40
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02105855_10095.jpg 
Non-Targeted attack target_label=o_label=80
Test-score: 0.4963936507701874, class 81
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02106030_11148.jpg 
Non-Targeted attack target_label=o_label=81
Test-score: 0.9998124241828918, class 80
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02106166_1205.jpg 
Non-Targeted attack target_label=o_label=81
Non-Targeted attack target_label=o_label=81
Non-Targeted attack target_label=o_label=81
Test-score: 0.49886953830718994, class 82
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02106382_1005.jpg 
Non-Targeted attack target_label=o_label=83
Test-score: 0.9096139669418335, class 101
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02106550_10048.jpg 
Non-Targeted attack target_label=o_label=84
Test-score: 0.9285635948181152, class 64
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02106662_10122.jpg 
Non-Targeted attack target_label=o_label=85
Non-Targeted attack target_label=o_label=85
Non-Targeted attack target_label=o_label=85
Test-score: 0.26929906010627747, class 85
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02107142_10952.jpg 
Non-Targeted attack target_label=o_label=86
Test-score: 0.09236325323581696, class 87
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02107312_105.jpg 
Non-Targeted attack target_label=o_label=87
Test-score: 0.9715318083763123, class 8
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02107574_1026.jpg 
Non-Targeted attack target_label=o_label=88
Test-score: 0.8894961476325989, class 91
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02107683_1003.jpg 
Non-Targeted attack target_label=o_label=89
Test-score: 0.2595520317554474, class 97
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02107908_1030.jpg 
Non-Targeted attack target_label=o_label=90
Test-score: 0.6221421957015991, class 91
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02108000_1087.jpg 
Non-Targeted attack target_label=o_label=91
Test-score: 0.9019123911857605, class 90
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02108089_1104.jpg 
Non-Targeted attack target_label=o_label=92
Test-score: 0.36616942286491394, class 45
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02108422_1096.jpg 
Non-Targeted attack target_label=o_label=93
Non-Targeted attack target_label=o_label=93
Test-score: 0.36447763442993164, class 103
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02108551_1025.jpg 
Non-Targeted attack target_label=o_label=94
Test-score: 0.715351939201355, class 64
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02108915_10564.jpg 
Non-Targeted attack target_label=o_label=95
Test-score: 0.5345490574836731, class 8
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02109047_10160.jpg 
Non-Targeted attack target_label=o_label=96
Test-score: 0.5757254362106323, class 13
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02109525_10032.jpg 
Non-Targeted attack target_label=o_label=97
Non-Targeted attack target_label=o_label=97
Non-Targeted attack target_label=o_label=97
Test-score: 0.19706158339977264, class 97
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02109961_11224.jpg 
Non-Targeted attack target_label=o_label=98
Test-score: 0.6545760631561279, class 99
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02110063_11105.jpg 
Non-Targeted attack target_label=o_label=99
Test-score: 0.6142775416374207, class 112
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02110185_10116.jpg 
Non-Targeted attack target_label=o_label=100
Test-score: 0.8787350058555603, class 98
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02110627_10147.jpg 
Non-Targeted attack target_label=o_label=101
Test-score: 0.3279683589935303, class 48
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02110806_1214.jpg 
Non-Targeted attack target_label=o_label=102
Test-score: 0.3767395615577698, class 91
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02110958_10378.jpg 
Non-Targeted attack target_label=o_label=103
Non-Targeted attack target_label=o_label=103
Non-Targeted attack target_label=o_label=103
Test-score: 0.5809877514839172, class 103
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02111129_1111.jpg 
Non-Targeted attack target_label=o_label=104
Test-score: 0.5515199899673462, class 94
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02111277_10237.jpg 
Non-Targeted attack target_label=o_label=105
Test-score: 0.5786005258560181, class 71
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02111500_1048.jpg 
Non-Targeted attack target_label=o_label=106
Non-Targeted attack target_label=o_label=106
Test-score: 0.07512383162975311, class 72
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02111889_10059.jpg 
Non-Targeted attack target_label=o_label=107
Test-score: 0.3027220666408539, class 72
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02112018_10158.jpg 
Non-Targeted attack target_label=o_label=108
Non-Targeted attack target_label=o_label=108
Non-Targeted attack target_label=o_label=108
Test-score: 0.8465248346328735, class 108
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02112137_1005.jpg 
Non-Targeted attack target_label=o_label=109
Non-Targeted attack target_label=o_label=109
Non-Targeted attack target_label=o_label=109
Test-score: 0.4224722981452942, class 80
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02112350_10079.jpg 
Non-Targeted attack target_label=o_label=110
Non-Targeted attack target_label=o_label=110
Non-Targeted attack target_label=o_label=110
Test-score: 0.5601344108581543, class 110
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02112706_105.jpg 
Non-Targeted attack target_label=o_label=111
Test-score: 0.3139760494232178, class 119
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02113023_1136.jpg 
Non-Targeted attack target_label=o_label=112
Test-score: 0.9765301942825317, class 113
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02113186_1030.jpg 
Non-Targeted attack target_label=o_label=113
Test-score: 0.9946261048316956, class 112
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02113624_1461.jpg 
Non-Targeted attack target_label=o_label=114
Test-score: 0.8437007069587708, class 115
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02113712_10525.jpg 
Non-Targeted attack target_label=o_label=115
Test-score: 0.9968068599700928, class 3
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02113799_1155.jpg 
Non-Targeted attack target_label=o_label=116
Test-score: 0.7794128656387329, class 71
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02113978_1034.jpg 
Non-Targeted attack target_label=o_label=117
Test-score: 0.05417395010590553, class 120
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02115641_10261.jpg 
Non-Targeted attack target_label=o_label=118
Test-score: 0.21042458713054657, class 117
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02115913_1010.jpg 
Non-Targeted attack target_label=o_label=119
Test-score: 0.5643325448036194, class 104
Image: /home/aistudio/baidu_attack_by_xin/input_image/n02116738_10024.jpg 
Non-Targeted attack target_label=o_label=120
Non-Targeted attack target_label=o_label=120
Non-Targeted attack target_label=o_label=120
Test-score: 0.19919021427631378, class 120
ADV 120 files, AVG MSE: 4.72762380070619

介紹FGSM以前，咱們回顧如下梯度降低算法更新公式：

θ:=θ−α∗▽J(θ)\theta := \theta - \alpha * \bigtriangledown J(\theta)θ:=θ−α∗▽J(θ)

其中，θ\thetaθ爲模型參數，α\alphaα爲步長，J(θ)J(\theta)J(θ)爲目標函數。

如上迭代可以使 J(θ)J(\theta)J(θ)不斷變小。

而上一code框中，定義目標爲模型輸出機率與標籤的交叉熵，按照如上公式迭代，會使模型預測更加準確。

而咱們的目的是讓模型迷惑，預測不出正確的標籤，所以，咱們只需改變一下符號：

θ:=θ+▽J(θ)\theta := \theta + \bigtriangledown J(\theta)θ:=θ+▽J(θ)

FGSM思想大概如此，此外

FGSM所用梯度會施以一sign函數,這意味這若是某一維度梯度爲-0.000000000001，通過sign函數後將爲-1。

In[ ]

""" Explaining and Harnessing Adversarial Examples, I. Goodfellow et al., ICLR 2015 實現了FGSM 支持定向和非定向攻擊的單步FGSM input_layer:輸入層 output_layer:輸出層 step_size:攻擊步長 adv_program：生成對抗樣本的prog eval_program:預測用的prog isTarget：是否認向攻擊 target_label：定向攻擊標籤 epsilon:約束linf大小 o:原始數據 use_gpu：是否使用GPU 返回： 生成的對抗樣本 """
def FGSM(adv_program,eval_program,gradients,o,input_layer,output_layer,step_size=16.0/256,epsilon=16.0/256,isTarget=False,target_label=0,use_gpu=False):
    
    place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
    exe = fluid.Executor(place)
   
    result = exe.run(eval_program,
                     fetch_list=[output_layer],
                     feed={ input_layer.name:o })
    result = result[0][0]
   
    o_label = np.argsort(result)[::-1][:1][0]
    
    if not isTarget:
        #無定向攻擊 target_label的值自動設置爲原標籤的值
        print("Non-Targeted attack target_label=o_label={}".format(o_label))
        target_label=o_label
    else:
        print("Targeted attack target_label={} o_label={}".format(target_label,o_label))
        
        
    target_label=np.array([target_label]).astype('int64')
    target_label=np.expand_dims(target_label, axis=0)
    
    #計算梯度
    g = exe.run(adv_program,
                     fetch_list=[gradients],
                     feed={ input_layer.name:o,'label': target_label  }
               )
    g = g[0][0]
    
    
    if isTarget:
        adv=o-np.sign(g)*step_size
    else:
        #################################
        #注意此處符號
        adv=o+np.sign(g)*step_size
    
    #實施linf約束
    adv=linf_img_tenosr(o,adv,epsilon)
    
    return adv

1.3 讓咱們看看生成的對抗樣本和原來有什麼區別

In[ ]

#定義一個觀察圖片區別的函數
def show_images_diff(original_img,adversarial_img):
    #original_img = np.array(Image.open(original_img))
    #adversarial_img = np.array(Image.open(adversarial_img))
    original_img=cv2.resize(original_img.copy(),(224,224))
    adversarial_img=cv2.resize(adversarial_img.copy(),(224,224))

    plt.figure(figsize=(10,10))

    #original_img=original_img/255.0
    #adversarial_img=adversarial_img/255.0

    plt.subplot(1, 3, 1)
    plt.title('Original Image')
    plt.imshow(original_img)
    plt.axis('off')

    plt.subplot(1, 3, 2)
    plt.title('Adversarial Image')
    plt.imshow(adversarial_img)
    plt.axis('off')

    plt.subplot(1, 3, 3)
    plt.title('Difference')
    difference = 0.0+adversarial_img - original_img
        
    l0 = np.where(difference != 0)[0].shape[0]*100/(224*224*3)
    l2 = np.linalg.norm(difference)/(256*3)
    linf=np.linalg.norm(difference.copy().ravel(),ord=np.inf)
    # print(difference)
    print("l0={}% l2={} linf={}".format(l0, l2,linf))
    
    #(-1,1) -> (0,1)
    #灰色打底 容易看出區別
    difference=difference/255.0
        
    difference=difference/2.0+0.5
   
    plt.imshow(difference)
    plt.axis('off')

    plt.show()
    

    #plt.savefig('fig_cat.png')

In[ ]

from PIL import Image, ImageOps
import cv2
import matplotlib.pyplot as plt
original_img=np.array(Image.open("/home/aistudio/baidu_attack_by_xin/input_image/n02085782_1039.jpg"))
adversarial_img=np.array(Image.open("/home/aistudio/baidu_attack_by_xin/output_image/n02085782_1039.jpg"))
show_images_diff(original_img,adversarial_img)

l0=92.0014880952381% l2=3.203206511496744 linf=31.0

在肉眼觀察下，並看不出對抗樣本有什麼變化，但其實模型已經認不出這條狗了

二模型改進

baseline採用了FGSM算法，而且只攻擊了一個模型，所以改進的思路爲兩路：

第一路，訓練更多的模型，攻擊更多的模型已尋求泛化能力。
第二路，改進算法，試試更發雜的，效果更好的算法。

2.1 更多的模型

集成模型選取思路爲多元，儘量多的不一樣模型，纔可能逼近賽題背後的黑盒模型。

橙色和藍色框中模型均採用pytorch進行遷移訓練，訓練集測試集爲原始Stanford Dogs數據集劃分，迭代次數均爲25，學習率均爲0.001。隨後將pytorch轉換爲oxnn模型，再用x2paddle轉換爲paddle模型。至於轉換的細節我將在稍後的系列文章中詳細介紹
紅色框中模型爲直接用paddlepaddle訓練而來，迭代次數爲20，其他參數與前述相同。
人工加固ResNeXt50_32x4d模型

決賽中的灰盒模型爲人工加固的模型，結構爲ResNeXt50。爲了攻擊黑盒模型，我在本地訓練了一個加固模型，做爲灰盒模型的逼近。訓練加固模型涉及到訓練集的選取和訓練方法的選取。訓練集的構成主要有兩部分，第一部分我採用不一樣的方法攻擊初賽的白盒模型(此白盒模型與灰盒模型具備相同的網絡結構)，將生成的n個樣本集做爲訓練集的一部分，思路如圖3.2所示。這樣基於的假設是，不一樣的攻擊方法生成的對抗樣本在真實的灰盒模型表現上會有差別，有些圖片依然能被灰盒模型正確識別。將n個樣本集集合，就能夠構建出徹底將灰盒攻擊成功的圖片集。

第二部分爲Stanford Dogs數據集中隨機選取的8000張圖片和原始的120張圖片，這些圖片的選取是爲了保持模型的泛化能力。

note： 這種方法效果很是明顯，由於從比賽來說，若是你訓練的模型和背後的黑盒具備相同的結構，遷移能力要比其餘結構的模型好不少

集成模型代碼實踐

幸虧不一樣模型參數的命名方式不一樣，所以一股腦讀進來不會出錯，代碼以下。

In[ ]

#coding=utf-8
from __future__ import absolute_import
from __future__ import division
from __future__ import print_function
import sys
import os
import numpy as np
import paddle.fluid as fluid
import pandas as pd
import models
from attack.attack_pp import FGSM, PGD,linf_img_tenosr,ensem_mom_attack_threshold_9model,\
ensem_mom_attack_threshold_9model2,ensem_mom_attack_threshold_9model_tarversion
from utils import init_prog, save_adv_image, process_img, tensor2img, calc_mse, add_arguments, print_arguments

image_shape = [3, 224, 224]
class_dim=121
input_dir = "./input_image/"
output_dir = "./output_image_attack/"
os.makedirs("./output_image_attack") 
#######################################################################
#這就是所用的全部模型
model_name1="ResNeXt50_32x4d"
pretrained_model1="./models_parameters/86.45+88.81ResNeXt50_32x4d"

model_name2="MobileNetV2"
pretrained_model2="./models_parameters/MobileNetV2"

model_name4="VGG16"
pretrained_model4="./models_parameters/VGG16"

model_name3="Densenet121"
pretrained_model3="./models_parameters/Densenet121"

model_name5="mnasnet1_0"
pretrained_model5="./models_parameters/mnasnet1_0"

model_name6="wide_resnet"
pretrained_model6="./models_parameters/wide_resnet"

model_name7="googlenet"
pretrained_model7="./models_parameters/googlenet"

model_name8="nas_mobile_net"
pretrained_model8="./models_parameters/nas_mobile_net"

model_name9="alexnet"
pretrained_model9="./models_parameters/alexnet"
########################################################################
val_list = 'val_list.txt'
use_gpu=True

mydict = {0: 1, 1: 10, 2: 100, 3: 101, 4: 102, 5: 103, 6: 104, 7: 105, 8: 106, 9: 107, 10: 108, 11: 109, 12: 11, 13: 110, 14: 111, 15: 112, 16: 113, 17: 114, 18: 115, 19: 116, 20: 117, 21: 118, 22: 119, 23: 12, 24: 120, 25: 13, 26: 14, 27: 15, 28: 16, 29: 17, 30: 18, 31: 19, 32: 2, 33: 20, 34: 21, 35: 22, 36: 23, 37: 24, 38: 25, 39: 26, 40: 27, 41: 28, 42: 29, 43: 3, 44: 30, 45: 31, 46: 32, 47: 33, 48: 34, 49: 35, 50: 36, 51: 37, 52: 38, 53: 39, 54: 4, 55: 40, 56: 41, 57: 42, 58: 43, 59: 44, 60: 45, 61: 46, 62: 47, 63: 48, 64: 49, 65: 5, 66: 50, 67: 51, 68: 52, 69: 53, 70: 54, 71: 55, 72: 56, 73: 57, 74: 58, 75: 59, 76: 6, 77: 60, 78: 61, 79: 62, 80: 63, 81: 64, 82: 65, 83: 66, 84: 67, 85: 68, 86: 69, 87: 7, 88: 70, 89: 71, 90: 72, 91: 73, 92: 74, 93: 75, 94: 76, 95: 77, 96: 78, 97: 79, 98: 8, 99: 80, 100: 81, 101: 82, 102: 83, 103: 84, 104: 85, 105: 86, 106: 87, 107: 88, 108: 89, 109: 9, 110: 90, 111: 91, 112: 92, 113: 93, 114: 94, 115: 95, 116: 96, 117: 97, 118: 98, 119: 99}
origdict = {1: 0, 2: 32, 3: 43, 4: 54, 5: 65, 6: 76, 7: 87, 8: 98, 9: 109, 10: 1, 11: 12, 12: 23, 13: 25, 14: 26, 15: 27, 16: 28, 17: 29, 18: 30, 19: 31, 20: 33, 21: 34, 22: 35, 23: 36, 24: 37, 25: 38, 26: 39, 27: 40, 28: 41, 29: 42, 30: 44, 31: 45, 32: 46, 33: 47, 34: 48, 35: 49, 36: 50, 37: 51, 38: 52, 39: 53, 40: 55, 41: 56, 42: 57, 43: 58, 44: 59, 45: 60, 46: 61, 47: 62, 48: 63, 49: 64, 50: 66, 51: 67, 52: 68, 53: 69, 54: 70, 55: 71, 56: 72, 57: 73, 58: 74, 59: 75, 60: 77, 61: 78, 62: 79, 63: 80, 64: 81, 65: 82, 66: 83, 67: 84, 68: 85, 69: 86, 70: 88, 71: 89, 72: 90, 73: 91, 74: 92, 75: 93, 76: 94, 77: 95, 78: 96, 79: 97, 80: 99, 81: 100, 82: 101, 83: 102, 84: 103, 85: 104, 86: 105, 87: 106, 88: 107, 89: 108, 90: 110, 91: 111, 92: 112, 93: 113, 94: 114, 95: 115, 96: 116, 97: 117, 98: 118, 99: 119, 100: 2, 101: 3, 102: 4, 103: 5, 104: 6, 105: 7, 106: 8, 107: 9, 108: 10, 109: 11, 110: 13, 111: 14, 112: 15, 113: 16, 114: 17, 115: 18, 116: 19, 117: 20, 118: 21, 119: 22, 120: 24}

adv_program=fluid.Program()
startup_program = fluid.Program()

new_scope = fluid.Scope()
#完成初始化
with fluid.program_guard(adv_program):
    place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
    exe = fluid.Executor(place)
    label = fluid.layers.data(name="label", shape=[1] ,dtype='int64')
    label2 = fluid.layers.data(name="label2", shape=[1] ,dtype='int64')
    adv_image = fluid.layers.create_parameter(name="adv_image",shape=(1,3,224,224),dtype='float32')
    
    model1 = models.__dict__[model_name1]()
    out_logits1 = model1.net(input=adv_image, class_dim=class_dim)
    out1 = fluid.layers.softmax(out_logits1)

    model2 = models.__dict__[model_name2](scale=2.0)
    out_logits2 = model2.net(input=adv_image, class_dim=class_dim)
    out2 = fluid.layers.softmax(out_logits2)

    _input1 = fluid.layers.create_parameter(name="_input_1", shape=(1,3,224,224),dtype='float32')
    
    model3 = models.__dict__[model_name3]()
    input_layer3,out_logits3 = model3.x2paddle_net(input =adv_image )
    out3 = fluid.layers.softmax(out_logits3[0])
    
    model4 = models.__dict__[model_name4]()
    input_layer4,out_logits4 = model4.x2paddle_net(input =adv_image )
    out4 = fluid.layers.softmax(out_logits4[0])


    model5 = models.__dict__[model_name5]()
    input_layer5,out_logits5 = model5.x2paddle_net(input =adv_image )
    out5 = fluid.layers.softmax(out_logits5[0])

    model6 = models.__dict__[model_name6]()
    input_layer6,out_logits6 = model6.x2paddle_net(input =adv_image)
    out6 = fluid.layers.softmax(out_logits6[0])

    model7 = models.__dict__[model_name7]()
    input_layer7,out_logits7 = model7.x2paddle_net(input =adv_image)
    out7 = fluid.layers.softmax(out_logits7[0])

    model8 = models.__dict__[model_name8]()
    input_layer8,out_logits8 = model8.x2paddle_net(input =adv_image)
    out8 = fluid.layers.softmax(out_logits8[0])
    
    
    model9 = models.__dict__[model_name9]()
    input_layer9,out_logits9 = model9.x2paddle_net(input =adv_image)
    out9 = fluid.layers.softmax(out_logits9[0])

    place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
    exe = fluid.Executor(place)
    exe.run(fluid.default_startup_program())

    one_hot_label = fluid.one_hot(input=label, depth=121)
    one_hot_label2 = fluid.one_hot(input=label2, depth=121)
    smooth_label = fluid.layers.label_smooth(label=one_hot_label, epsilon=0.1, dtype="float32")[0]
    smooth_label2 = fluid.layers.label_smooth(label=one_hot_label2, epsilon=0.1, dtype="float32")[0]



    ze = fluid.layers.fill_constant(shape=[1], value=-1, dtype='float32')
    loss = 1.2*fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out1, label=label[0]))\
    + 0.2*fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out2, label=label[0]))\
    + fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out3, label=label2[0]))\
    + fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out4, label=label2[0]))\
    + fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out5, label=label2[0]))\
    + fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out6, label=label2[0]))\
    + fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out7, label=label2[0]))\
    + fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out8, label=label2[0]))\
    + fluid.layers.matmul(ze, fluid.layers.cross_entropy(input=out9, label=label2[0]))
    
    avg_loss=fluid.layers.reshape(loss ,[1])#這裏修改loss

init_prog(adv_program)
eval_program = adv_program.clone(for_test=True)

with fluid.program_guard(adv_program): 
    #沒有解決變量重名的問題
    #此部分代碼爲加載模型參數
    def if_exist(var):
        b = os.path.exists(os.path.join(pretrained_model1, var.name))
        return b
    def if_exist2(var):
        b = os.path.exists(os.path.join(pretrained_model2, var.name))
        return b
    def if_exist3(var):
        b = os.path.exists(os.path.join(pretrained_model3, var.name))
        return b
    def if_exist4(var):
        b = os.path.exists(os.path.join(pretrained_model4, var.name))
        return b
    def if_exist5(var):
        b = os.path.exists(os.path.join(pretrained_model5, var.name))
        return b
    def if_exist6(var):
        b = os.path.exists(os.path.join(pretrained_model6, var.name))
        return b
    def if_exist7(var):
        b = os.path.exists(os.path.join(pretrained_model7, var.name))
        return b
    def if_exist8(var):
        b = os.path.exists(os.path.join(pretrained_model8, var.name))
        return b
    def if_exist9(var):
        b = os.path.exists(os.path.join(pretrained_model9, var.name))
        return b
    fluid.io.load_vars(exe,
                       pretrained_model1,
                       fluid.default_main_program(),
                       predicate=if_exist)
    fluid.io.load_vars(exe,
                       pretrained_model2,
                       fluid.default_main_program(),
                       predicate=if_exist2)
    fluid.io.load_vars(exe,
                       pretrained_model3,
                       fluid.default_main_program(),
                       predicate=if_exist3)
    fluid.io.load_vars(exe,
                       pretrained_model4,
                       fluid.default_main_program(),
                       predicate=if_exist4)
    fluid.io.load_vars(exe,
                       pretrained_model5,
                       fluid.default_main_program(),
                       predicate=if_exist5)
    fluid.io.load_vars(exe,
                       pretrained_model6,
                       fluid.default_main_program(),
                       predicate=if_exist6)
    fluid.io.load_vars(exe,
                       pretrained_model7,
                       fluid.default_main_program(),
                       predicate=if_exist7)
    fluid.io.load_vars(exe,
                       pretrained_model8,
                       fluid.default_main_program(),
                       predicate=if_exist8)

    fluid.io.load_vars(exe,
                       pretrained_model9,
                       fluid.default_main_program(),
                       predicate=if_exist9)
    gradients = fluid.backward.gradients(targets=avg_loss, inputs=[adv_image])[0]
    #gradients = fluid.backward.gradients(targets=avg_loss, inputs=[adv_image])
    #print(gradients.shape)
    
def attack_nontarget_by_ensemble(img, src_label,src_label2,label,momentum): #src_label2爲轉換後的標籤
    adv,m=ensem_mom_attack_threshold_9model_tarversion(adv_program=adv_program,eval_program=eval_program,gradients=gradients,o=img,
                src_label = src_label,
                src_label2 = src_label2,
                label = label,
                out1 = out1,out2 = out2 ,out3 = out3 ,out4 = out4,out5 = out5,out6 = out6,out7 = out7 ,out8 = out8,out9 = out9,mm = momentum)#添加了mm

    adv_img=tensor2img(adv)
    return adv_img,m

def get_original_file(filepath):
    with open(filepath, 'r') as cfile:
        full_lines = [line.strip() for line in cfile]
    cfile.close()
    original_files = []
    for line in full_lines:
        label, file_name = line.split()
        original_files.append([file_name, int(label)])
    return original_files
    
def gen_adv():
    mse = 0
    original_files = get_original_file(input_dir + val_list)
    #下一個圖片的初始梯度方向爲上一代的最後的值
    global momentum
    momentum=0
    
    for filename, label in original_files:
        img_path = input_dir + filename
        print("Image: {0} ".format(img_path))
        img=process_img(img_path)
        #adv_img = attack_nontarget_by_ensemble(img, label,origdict[label],label)
        adv_img,m = attack_nontarget_by_ensemble(img, label,origdict[label],label,momentum)
        #m爲上一個樣本最後一次梯度值
        momentum = m
        #adv_img 已經通過轉換了，範圍是0-255

        image_name, image_ext = filename.split('.')
        ##Save adversarial image(.png)
        save_adv_image(adv_img, output_dir+image_name+'.png')
        org_img = tensor2img(img)
        score = calc_mse(org_img, adv_img)
        print("Image:{0}, mase = {1} ".format(img_path,score))
        mse += score
    print("ADV {} files, AVG MSE: {} ".format(len(original_files), mse/len(original_files)))

note: 代碼中此處函數已替換爲個人實現，方案將在下一模塊介紹

2.2 改進算法

加入動量迭代

動量項是緩解局部最優的經常使用手段，咱們將對抗樣本的生成依然當作優化問題，那麼用到動量也就符合常理。

實際代碼以下：

隨機梯度反向

大粒度此方法受啓發於[2]，論文中採起雙路尋優，一路採用常規方法梯度上升，如圖中綠線所示。另外一路先採起梯度降低到達這一分類局部最優再進行梯度上升，以期找到更快的上升路徑，如圖3.3中藍線所示。而本人在實現過程當中，對其進行簡化，僅在迭代的第一步進行梯度降低。

實際代碼以下：

小粒度隨機選取梯度中5%進行取反，可視爲像素粒度的梯度反向，反轉比例爲一超參數。

代碼以下：

其中比例爲一超參數，而選取隨機選取必定%爲生成與梯度相同形狀的隨機數，設定閾值選取必定%乘以-1.

添加高斯噪聲

此方法受啓發於[6]，論文做者認爲攻擊模型的梯度具備噪聲，損害了遷移能力。論文做者用一組原始圖片加噪聲後的梯度的平均代替原來的梯度，效果獲得提高。而我與論文做者理解不一樣，添加噪聲意在增長梯度的噪聲，以越過局部最優，再者屢次計算梯度很是耗時，所以我選用了只加一次噪聲，均值爲0，方差爲超參數。

代碼以下：

攻擊後進行目標攻擊

此方法受啓發於[2]，做者在成功越過度界線後進行消除無用噪聲操做，做者認爲此舉能夠增強對抗樣本的遷移能力。個人作法與此不一樣，我認爲不只要越過邊界，還要走向這個錯誤分類的低谷。此舉依據的假設是:儘管不一樣模型的分界線存在差別，模型學到的特徵應是類似的。思路如圖3.4中紅色箭頭所示，帶有圓圈的數字表示迭代步數。所以，在成功攻擊以後，我又添加了兩步定向攻擊，目標爲攻擊成功時被錯分的類別。在集成攻擊時，目標爲被錯分類別的衆數。

目標攻擊代碼：選取的目標標籤爲9個模型預測的衆數

將上述策略集成，核心函數代碼全貌展現在下一代碼框中，我將本身設計的函數命名爲：ensem_mom_attack_threshold_9model_tarversion

In[ ]

def ensem_mom_attack_threshold_9model_tarversion(adv_program,eval_program,gradients,o,src_label2,src_label,out1,out2,out3,out4,out5,out6,out7,out8,out9,label,mm,iteration=20,use_gpu = True):
    origdict = {1: 0, 2: 32, 3: 43, 4: 54, 5: 65, 6: 76, 7: 87, 8: 98, 9: 109, 10: 1, 11: 12, 12: 23, 13: 25, 14: 26, 15: 27, 16: 28, 17: 29, 18: 30, 19: 31, 20: 33, 21: 34, 22: 35, 23: 36, 24: 37, 25: 38, 26: 39, 27: 40, 28: 41, 29: 42, 30: 44, 31: 45, 32: 46, 33: 47, 34: 48, 35: 49, 36: 50, 37: 51, 38: 52, 39: 53, 40: 55, 41: 56, 42: 57, 43: 58, 44: 59, 45: 60, 46: 61, 47: 62, 48: 63, 49: 64, 50: 66, 51: 67, 52: 68, 53: 69, 54: 70, 55: 71, 56: 72, 57: 73, 58: 74, 59: 75, 60: 77, 61: 78, 62: 79, 63: 80, 64: 81, 65: 82, 66: 83, 67: 84, 68: 85, 69: 86, 70: 88, 71: 89, 72: 90, 73: 91, 74: 92, 75: 93, 76: 94, 77: 95, 78: 96, 79: 97, 80: 99, 81: 100, 82: 101, 83: 102, 84: 103, 85: 104, 86: 105, 87: 106, 88: 107, 89: 108, 90: 110, 91: 111, 92: 112, 93: 113, 94: 114, 95: 115, 96: 116, 97: 117, 98: 118, 99: 119, 100: 2, 101: 3, 102: 4, 103: 5, 104: 6, 105: 7, 106: 8, 107: 9, 108: 10, 109: 11, 110: 13, 111: 14, 112: 15, 113: 16, 114: 17, 115: 18, 116: 19, 117: 20, 118: 21, 119: 22, 120: 24}
    mydict = {0: 1, 1: 10, 2: 100, 3: 101, 4: 102, 5: 103, 6: 104, 7: 105, 8: 106, 9: 107, 10: 108, 11: 109, 12: 11, 13: 110, 14: 111, 15: 112, 16: 113, 17: 114, 18: 115, 19: 116, 20: 117, 21: 118, 22: 119, 23: 12, 24: 120, 25: 13, 26: 14, 27: 15, 28: 16, 29: 17, 30: 18, 31: 19, 32: 2, 33: 20, 34: 21, 35: 22, 36: 23, 37: 24, 38: 25, 39: 26, 40: 27, 41: 28, 42: 29, 43: 3, 44: 30, 45: 31, 46: 32, 47: 33, 48: 34, 49: 35, 50: 36, 51: 37, 52: 38, 53: 39, 54: 4, 55: 40, 56: 41, 57: 42, 58: 43, 59: 44, 60: 45, 61: 46, 62: 47, 63: 48, 64: 49, 65: 5, 66: 50, 67: 51, 68: 52, 69: 53, 70: 54, 71: 55, 72: 56, 73: 57, 74: 58, 75: 59, 76: 6, 77: 60, 78: 61, 79: 62, 80: 63, 81: 64, 82: 65, 83: 66, 84: 67, 85: 68, 86: 69, 87: 7, 88: 70, 89: 71, 90: 72, 91: 73, 92: 74, 93: 75, 94: 76, 95: 77, 96: 78, 97: 79, 98: 8, 99: 80, 100: 81, 101: 82, 102: 83, 103: 84, 104: 85, 105: 86, 106: 87, 107: 88, 108: 89, 109: 9, 110: 90, 111: 91, 112: 92, 113: 93, 114: 94, 115: 95, 116: 96, 117: 97, 118: 98, 119: 99}
    
    place = fluid.CUDAPlace(0) if use_gpu else fluid.CPUPlace()
    exe = fluid.Executor(place)
    
    target_label=np.array([src_label]).astype('int64')
    target_label=np.expand_dims(target_label, axis=0)
    target_label2=np.array([src_label2]).astype('int64')
    target_label2=np.expand_dims(target_label2, axis=0)
    
    img = o.copy()
    decay_factor = 0.90
    steps=90
    epsilons = np.linspace(5, 388, num=75)
    flag_traget = 0#表示非目標攻擊
    flag2=0 #退出的標誌
    for epsilon in epsilons[:]:
        #print("now momentum is {}".format(momentum))
        if flag_traget==0:
            #momentum = mm
            momentum = 0
            adv=img.copy()
            for i in range(steps):
                
                if i<50:
                    adv_noise = (adv+np.random.normal(loc=0.0, scale=0.5+epsilon/90,size = (3,224,224))).astype('float32')
                else:
                    adv_noise = (adv+np.random.normal(loc=0.0, scale=0.1,size = (3,224,224))).astype('float32')
                g,resul1,resul2,resul3,resul4,resul5,resul6,resul7,resul8,resul9 = exe.run(adv_program,
                             fetch_list=[gradients,out1,out2,out3,out4,out5,out6,out7,out8,out9],
                             feed={'label2':target_label2,'adv_image':adv_noise,'label': target_label })
               
                #print(g[0][0].shape,g[0][1].shape,g[0][2].shape)
                g = (g[0][0]+g[0][1]+g[0][2])/3 #三通道梯度平均
                #print(g.shape)
                velocity = g / (np.linalg.norm(g.flatten(),ord=1) + 1e-10)
                momentum = decay_factor * momentum + velocity
                #print(momentum.shape)
                norm_m = momentum / (np.linalg.norm(momentum.flatten(),ord=2) + 1e-10)
                #print(norm_m.shape)
                _max = np.max(abs(norm_m))
                tmp = np.percentile(abs(norm_m), [25, 99.45, 99.5])#將圖片變更的像素點限定在0.5%
                thres = tmp[2]
                mask = abs(norm_m)>thres
                norm_m_m = np.multiply(norm_m,mask)
                if i<50: #前50步，2%的梯度反響,隨着i遞減 試試5%
                    dir_mask = np.random.rand(3,224,224)
                    #print(dir_mask)
                    dir_mask = dir_mask>(0.15-i/900)  
                    #print(dir_mask)
                    dir_mask[dir_mask==0] = -1
                    #print(dir_mask)
                    norm_m_m = np.multiply(norm_m_m,dir_mask)
                    #print(norm_m_m.shape)
                #步長也隨着step衰減
                if i==0:
                    adv=adv+epsilon*norm_m_m 
                else:
                    adv=adv-epsilon*norm_m_m 
                    #adv=adv-(epsilon-i/30)*norm_m_m 
                #實施linf約束
                adv=linf_img_tenosr(img,adv,epsilon)
        else:
            for i in range(2):
                adv_noise = (adv+np.random.normal(loc=0.0, scale=0.1,size = (3,224,224))).astype('float32')
                target_label=np.array([t_label]).astype('int64')
                target_label=np.expand_dims(target_label, axis=0)
                target_label2=np.array([origdict[t_label]]).astype('int64')
                target_label2=np.expand_dims(target_label2, axis=0)
                g,resul1,resul2,resul3,resul4,resul5,resul6,resul7,resul8,resul9 = exe.run(adv_program,
                         fetch_list=[gradients,out1,out2,out3,out4,out5,out6,out7,out8,out9],
                         feed={'label2':target_label2,'adv_image':adv_noise,'label': target_label }
                          )
                g = (g[0][0]+g[0][1]+g[0][2])/3 #三通道梯度平均
                velocity = g / (np.linalg.norm(g.flatten(),ord=1) + 1e-10)
                momentum = decay_factor * momentum + velocity
                #print(momentum.shape)
                norm_m = momentum / (np.linalg.norm(momentum.flatten(),ord=2) + 1e-10)
                #print(norm_m.shape)
                _max = np.max(abs(norm_m))
                tmp = np.percentile(abs(norm_m), [25, 99.45, 99.5])#將圖片變更的像素點限定在0.5%
                thres = tmp[2]
                mask = abs(norm_m)>thres
                norm_m_m = np.multiply(norm_m,mask)
                adv=adv+epsilon*norm_m_m
                #實施linf約束
                adv=linf_img_tenosr(img,adv,epsilon)
            flag2=1
            
        print("epsilon is {}".format(epsilon))
        print("label is:{}; model1:{}; model2:{}; model3:{}; model4:{}; model5:{}; model6:{}; model7:{}; model8:{} ; model9:{} ".format(label,resul1.argmax(),resul2.argmax(),mydict[resul3.argmax()],mydict[resul4.argmax()],\
        mydict[resul5.argmax()],mydict[resul6.argmax()],mydict[resul7.argmax()],mydict[resul8.argmax()],mydict[resul9.argmax()]))#模型3標籤到真正標籤
        

        if((label!=resul1.argmax()) and(label!=resul2.argmax())and(origdict[label]!=resul3.argmax())and(origdict[label]!=resul4.argmax())and(origdict[label]!=resul5.argmax())\
        and(origdict[label]!=resul6.argmax())and(origdict[label]!=resul7.argmax())and(origdict[label]!=resul8.argmax())and(origdict[label]!=resul9.argmax())):
            res_list = [resul1.argmax(),resul2.argmax(),mydict[resul3.argmax()],mydict[resul4.argmax()],mydict[resul5.argmax()],mydict[resul6.argmax()],mydict[resul7.argmax()],mydict[resul8.argmax()],mydict[resul9.argmax()]]
            ser = pd.Series(res_list)
            t_label = ser.mode()[0]#取衆數做爲target_label
            flag_traget=1
            if(flag2 == 1):
                break
    return adv,momentum

3 方案復現

算法介紹完畢，讓咱們運行一下完整的方案，生成對抗樣本。

note: 因爲集成模型較多，代碼將運行1個小時左右。可提早停止運行(點擊notebook右上角運行菜單，選中中斷執行)，查看已成的對抗樣本。

In[ ]

gen_adv()

接來看看對抗樣本與原樣本有什麼區別

In[ ]

#定義一個觀察圖片區別的函數
def show_images_diff(original_img,adversarial_img):
    #original_img = np.array(Image.open(original_img))
    #adversarial_img = np.array(Image.open(adversarial_img))
    original_img=cv2.resize(original_img.copy(),(224,224))
    adversarial_img=cv2.resize(adversarial_img.copy(),(224,224))

    plt.figure(figsize=(10,10))

    #original_img=original_img/255.0
    #adversarial_img=adversarial_img/255.0

    plt.subplot(1, 3, 1)
    plt.title('Original Image')
    plt.imshow(original_img)
    plt.axis('off')

    plt.subplot(1, 3, 2)
    plt.title('Adversarial Image')
    plt.imshow(adversarial_img)
    plt.axis('off')

    plt.subplot(1, 3, 3)
    plt.title('Difference')
    difference = 0.0+adversarial_img - original_img
        
    l0 = np.where(difference != 0)[0].shape[0]*100/(224*224*3)
    l2 = np.linalg.norm(difference)/(256*3)
    linf=np.linalg.norm(difference.copy().ravel(),ord=np.inf)
    # print(difference)
    print("l0={}% l2={} linf={}".format(l0, l2,linf))
    
    #(-1,1) -> (0,1)
    #灰色打底 容易看出區別
    difference=difference/255.0
        
    difference=difference/2.0+0.5
   
    plt.imshow(difference)
    plt.axis('off')

    plt.show()
    

    #plt.savefig('fig_cat.png')#plt.savefig('fig_cat.png')10model_ensemble_attack.#plt.savefig('fig_cat.png')#plt.savefig('fig_cat.png')10model_ensemble_attack.py10model_ensemble_attack.py

In[ ]

from PIL import Image, ImageOps
import cv2
import matplotlib.pyplot as plt
#########################################
##此處的pname可替換爲你想查看的圖片
pname = "n02085620_10074.jpg"
#########################################
image_name, image_ext = pname.split('.')
pname_attack = image_name + ".png"
original_img=np.array(Image.open("/home/aistudio/baidu_attack_by_xin/input_image/" + pname))
adversarial_img=np.array(Image.open("/home/aistudio/baidu_attack_by_xin/output_image_attack/" + pname_attack))
show_images_diff(original_img,adversarial_img)

l0=22.507440476190474% l2=5.516279998781479 linf=221.0

此方案生成的對抗樣本具備良好的遷移能力，能夠在AI安全對抗賽中取得第二名，以此標準

其中M表示防護模型，y表示樣本I的真實標籤。若是防護算法對樣本識別正確，這次攻擊不成功，擾動量直接置爲上限128。若是攻擊成功，計算對抗樣本和原始樣本的擾動量，採用平均L2距離。每一個對抗樣本都會在m個防護模型上計算擾動量，n表明樣本個數，最後對全部的擾動量進行平均，作爲本次攻擊的總體距離得分，得分越小越好。

爲衡量標準，我方案生成的圖片能夠達到3.78089。

4 臨門一腳

然而這不夠，做爲一個競賽，人人都虎視眈眈盯着獎金的時候，還須要不斷的提高。所以還須要臨門一腳，一種後處理方法。

小擾動截斷

使用上述方法後，個人結果在95-96分之間波動，爲進一步提高成績，我選用最高分96.53分圖片進行後處理。後處理方法爲：將攻擊後的圖片與原圖片進行對比，對必定閾值如下的擾動進行截斷。 通過不斷上探閾值，發現閾值爲17(圖片的像素範圍爲0-255)的時候效果最好。此方法提分0.3左右。

代碼以下：

提早預警： 運行下面代碼須要生成所有對抗樣本。

In[ ]

#coding=utf-8

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import argparse
import functools
import numpy as np
import paddle.fluid as fluid

#加載自定義文件
import models
from attack.attack_pp import FGSM, PGD
from utils import *

######Init args
image_shape = [3,224,224]
class_dim=121
input_dir = "./input_image/"
attacked_dir = "./output_image_attack/"
output_dir = "./posopt_output_image/"
drop_thres = 10
os.makedirs("./posopt_output_image") 
val_list = 'val_list.txt'
use_gpu=True


####### Main #######
def get_original_file(filepath):
    with open(filepath, 'r') as cfile:
        full_lines = [line.strip() for line in cfile]
    cfile.close()
    original_files = []
    for line in full_lines:
        label, file_name = line.split()
        original_files.append([file_name, int(label)])
    return original_files

def gen_diff():
    original_files = get_original_file(input_dir + val_list)

    for filename, label in original_files:
        image_name, image_ext = filename.split('.')
        img_path = input_dir + filename
        print("Image: {0} ".format(img_path))
        img=process_img(img_path)
        adv_img_path = attacked_dir + image_name+'.png'
        adv=process_img(adv_img_path)
        
        org_img = tensor2img(img)
        adv_img = tensor2img(adv)
        #10/256 如下的擾動所有截斷
        diff = abs(org_img-adv_img)<drop_thres   #<10的爲1
        diff_max = abs(org_img-adv_img)>=drop_thres  #>=10的爲1
        #<10的保留org_img
        tmp1 = np.multiply(org_img,diff)
        #>10的保留adv_img
        tmp2 = np.multiply(adv_img,diff_max)
        final_img = tmp1+tmp2
        
        save_adv_image(final_img, output_dir+image_name+'.png')


gen_diff()