fcn是深度學習用於圖像分割的鼻祖.後續的不少網絡結構都是在此基礎上演進而來.html
圖像分割即像素級別的分類.前端
語義分割的基本框架:
前端fcn(以及在此基礎上的segnet,deconvnet,deeplab等) + 後端crf/mrfgit
FCN是分割網絡的鼻祖,後面的不少網絡都是在此基礎上提出的.
論文地址github
和傳統的分類網絡相比,就是將傳統分類網絡的全鏈接層用反捲積層替代.獲得一個和圖像大小一致的feature map。本篇文章用的網絡是VGG.
後端
主要關注兩點網絡
關於反捲積(也叫轉置卷積)的詳細推導,能夠參考:<https://blog.csdn.net/LoseInVain/article/details/81098502>框架
簡單滴說就是:卷積的反向操做.以4x4矩陣A爲例,卷積核C(3x3,stride=1),經過卷積操做獲得一個2x2的矩陣B. 轉置卷積即已知B,要獲得A,咱們要找到卷積核C,使得B至關於A經過C作正向卷積,獲得B.ide
轉置卷積是一種上採樣的方法.函數
若是隻用特徵提取部分(也就是VGG全鏈接層以前的部分)獲得的feature map作上採樣將feature map還原到圖像輸入的size的話,feature不夠精確.因此採用不一樣layer的feature map作上採樣再組合起來.學習
源碼:https://github.com/pochih/FCN-pytorch
其中的核心代碼以下:
class FCNs(nn.Module): def __init__(self, pretrained_net, n_class): super().__init__() self.n_class = n_class self.pretrained_net = pretrained_net self.relu = nn.ReLU(inplace=True) self.deconv1 = nn.ConvTranspose2d(512, 512, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) self.bn1 = nn.BatchNorm2d(512) self.deconv2 = nn.ConvTranspose2d(512, 256, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) self.bn2 = nn.BatchNorm2d(256) self.deconv3 = nn.ConvTranspose2d(256, 128, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) self.bn3 = nn.BatchNorm2d(128) self.deconv4 = nn.ConvTranspose2d(128, 64, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) self.bn4 = nn.BatchNorm2d(64) self.deconv5 = nn.ConvTranspose2d(64, 32, kernel_size=3, stride=2, padding=1, dilation=1, output_padding=1) self.bn5 = nn.BatchNorm2d(32) self.classifier = nn.Conv2d(32, n_class, kernel_size=1) def forward(self, x): output = self.pretrained_net(x) x5 = output['x5'] # size=(N, 512, x.H/32, x.W/32) x4 = output['x4'] # size=(N, 512, x.H/16, x.W/16) x3 = output['x3'] # size=(N, 256, x.H/8, x.W/8) x2 = output['x2'] # size=(N, 128, x.H/4, x.W/4) x1 = output['x1'] # size=(N, 64, x.H/2, x.W/2) score = self.bn1(self.relu(self.deconv1(x5))) # size=(N, 512, x.H/16, x.W/16) score = score + x4 # element-wise add, size=(N, 512, x.H/16, x.W/16) score = self.bn2(self.relu(self.deconv2(score))) # size=(N, 256, x.H/8, x.W/8) score = score + x3 # element-wise add, size=(N, 256, x.H/8, x.W/8) score = self.bn3(self.relu(self.deconv3(score))) # size=(N, 128, x.H/4, x.W/4) score = score + x2 # element-wise add, size=(N, 128, x.H/4, x.W/4) score = self.bn4(self.relu(self.deconv4(score))) # size=(N, 64, x.H/2, x.W/2) score = score + x1 # element-wise add, size=(N, 64, x.H/2, x.W/2) score = self.bn5(self.relu(self.deconv5(score))) # size=(N, 32, x.H, x.W) score = self.classifier(score) # size=(N, n_class, x.H/1, x.W/1) return score # size=(N, n_class, x.H/1, x.W/1)
train.py中
vgg_model = VGGNet(requires_grad=True, remove_fc=True) fcn_model = FCNs(pretrained_net=vgg_model, n_class=n_class)
這裏咱們重點看FCN
的forward函數
def forward(self, x): output = self.pretrained_net(x) x5 = output['x5'] # size=(N, 512, x.H/32, x.W/32) x4 = output['x4'] # size=(N, 512, x.H/16, x.W/16) x3 = output['x3'] # size=(N, 256, x.H/8, x.W/8) x2 = output['x2'] # size=(N, 128, x.H/4, x.W/4) x1 = output['x1'] # size=(N, 64, x.H/2, x.W/2) score = self.bn1(self.relu(self.deconv1(x5))) # size=(N, 512, x.H/16, x.W/16) score = score + x4 # element-wise add, size=(N, 512, x.H/16, x.W/16) score = self.bn2(self.relu(self.deconv2(score))) # size=(N, 256, x.H/8, x.W/8) score = score + x3 # element-wise add, size=(N, 256, x.H/8, x.W/8) score = self.bn3(self.relu(self.deconv3(score))) # size=(N, 128, x.H/4, x.W/4) score = score + x2 # element-wise add, size=(N, 128, x.H/4, x.W/4) score = self.bn4(self.relu(self.deconv4(score))) # size=(N, 64, x.H/2, x.W/2) score = score + x1 # element-wise add, size=(N, 64, x.H/2, x.W/2) score = self.bn5(self.relu(self.deconv5(score))) # size=(N, 32, x.H, x.W) score = self.classifier(score) # size=(N, n_class, x.H/1, x.W/1) return score # size=(N, n_class, x.H/1, x.W/1)
可見FCN的輸入爲(batch_size,c,h,w),輸出爲(batch_size,class,h,w).
首先是通過vgg的特徵提取層,能夠獲得feature map. 5個max_pool後的feature map的size分別爲
x5 = output['x5'] # size=(N, 512, x.H/32, x.W/32) x4 = output['x4'] # size=(N, 512, x.H/16, x.W/16) x3 = output['x3'] # size=(N, 256, x.H/8, x.W/8) x2 = output['x2'] # size=(N, 128, x.H/4, x.W/4) x1 = output['x1'] # size=(N, 64, x.H/2, x.W/2)
以後每個pool layer的feature map都通過一次2倍上採樣,並與前一個pool layer的輸出進行element-wise add.(resnet也有相似操做).從而使得上採樣後的feature map信息更充分更精準,模型的魯棒性會更好.
例如以輸入圖片尺寸爲224x224爲例,pool4的輸出爲(,512,14,14),pool5的輸出爲(,512,7,7),反捲積後獲得(,512,14,14),再與pool4的輸出作element-wise add。獲得的仍然是(,512,14,14). 對這個輸出作上採樣獲得(,256,28,28)再與pool3的輸出相加. 依次類推,最終獲得(,64,112,112).
此後,再作一次反捲積上採樣獲得(,32,224,224),以後卷積獲得(,n_class,224,224)。即獲得n_class張224x224的feature map。
下圖顯示了隨着上採樣的進行,獲得的feature map細節愈來愈豐富.
criterion = nn.BCEWithLogitsLoss()
損失函數採用二分類交叉熵.torch中有2個計算二分類交叉熵的函數
後者只是在前者的基礎上,對輸入先作一個sigmoid將輸入轉換到0-1之間.即BCEWithLogitsLoss = Sigmoid + BCELoss
一個具體的例子能夠參考:https://blog.csdn.net/qq_22210253/article/details/85222093