隨着微服務概念的深刻人心,隨着docker開發的持續進行,咱們在生產的過程當中將會產生大量的docker鏡像,這些鏡像會隨着版本迭代的過程當中,這些鏡像將會佔用大量的存儲空間,本文將分析影響鏡像大小的因素,隨後提供鏡像瘦身的思路。python
不可避免地,咱們在docker學習的過程當中必定繞不開理解這三者的關係,從研發流程的角度來看來看,Dockerfile 是軟件的原材料,Docker 鏡像是軟件的交付品,而 Docker 容器則能夠認爲是軟件的運行的狀態。從應用軟件的角度來看,Dockerfile、Docker 鏡像與 Docker 容器分別表明軟件的三個不一樣階段,Dockerfile 面向開發,Docker 鏡像成爲交付標準,Docker 容器則涉及部署與運維,三者缺一不可,協力充當 Docker 體系的基石。
簡單來說,Dockerfile構建出Docker鏡像,經過Docker鏡像運行Docker容器。後續有機會再詳細分析這三者的關係linux
如下是咱們精簡鏡像的目的:git
簡單來說就是:最基礎的鏡像+合併操做指令(減小操做指令)sql
咱們即將經過一系列的dockerfile開發,對鏡像打包進行一次次的試驗,以盡最大努力的減少鏡像的大小
原始文件
這是一份很是糟糕的Dockerfile文件打包以後,僅做爲分析用例docker
FROM python:3.5.6-slim-stretch RUN apt-get update RUN echo python -V RUN mkdir /code RUN mkdir /code/db ADD . /code/ WORKDIR /code RUN pip install -r requirement
運行命令:docker build -t spidermax:test_01 .
將會對鏡像進行打包,docker相關程序會讀取該文件(Dockerfile),如下是鏡像層的部署過程django
Sending build context to Docker daemon 289.8kB
Step 1/9 : FROM python:3.5.6-slim-stretch
---> 86669b8e5771
Step 2/9 : RUN apt-get update
---> Using cache
---> 1b77542b82f4
Step 3/9 : RUN echo python -V
---> Using cache
---> de6cc0a916d7
Step 4/9 : RUN mkdir /code
---> Using cache
---> 996828abe04e
Step 5/9 : RUN mkdir /code/db
---> Using cache
---> 2db7c4dfb918
Step 6/9 : ADD . /code/
---> d9d92db45b29
Step 7/9 : WORKDIR /code
---> Running in 5ae405de28fc
Removing intermediate container 5ae405de28fc
---> a7e3696c6a5b
Step 8/9 : RUN pip install -U pip
---> Running in dcf5b22f6f33
Collecting pip
Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 18.1
Uninstalling pip-18.1:
Successfully uninstalled pip-18.1
Successfully installed pip-19.0.3
Removing intermediate container dcf5b22f6f33
---> 949d89aabe2b
Step 9/9 : RUN pip install -r requirement
---> Running in 697c65e4161a
Collecting Django==2.1.7 (from -r requirement (line 1))
Downloading https://files.pythonhosted.org/packages/c7/87/fbd666c4f87591ae25b7bb374298e8629816e87193c4099d3608ef11fab9/Django-2.1.7-py3-none-any.whl (7.3MB)
Collecting EasyProcess==0.2.5 (from -r requirement (line 2))
Downloading https://files.pythonhosted.org/packages/45/3a/4eecc0c7995a13a64739bbedc0d3691fc574245b7e79cff81905aa0c2b38/EasyProcess-0.2.5.tar.gz
Collecting PyMySQL==0.9.3 (from -r requirement (line 3))
Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB)
Collecting pytz==2018.9 (from -r requirement (line 4))
Downloading https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl (510kB)
Collecting selenium==3.141.0 (from -r requirement (line 5))
Downloading https://files.pythonhosted.org/packages/80/d6/4294f0b4bce4de0abf13e17190289f9d0613b0a44e5dd6a7f5ca98459853/selenium-3.141.0-py2.py3-none-any.whl (904kB)
Collecting urllib3==1.24.1 (from -r requirement (line 6))
Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)
Building wheels for collected packages: EasyProcess
Building wheel for EasyProcess (setup.py): started
Building wheel for EasyProcess (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/41/22/19/af15ef6264c58b625a82641ed7483ad05e258fbd8925505227
Successfully built EasyProcess
Installing collected packages: pytz, Django, EasyProcess, PyMySQL, urllib3, selenium
Successfully installed Django-2.1.7 EasyProcess-0.2.5 PyMySQL-0.9.3 pytz-2018.9 selenium-3.141.0 urllib3-1.24.1
Removing intermediate container 697c65e4161a
---> f59692faac28
Successfully built f59692faac28
Successfully tagged spidermax:test_01緩存
經過docker images查看鏡像大小
經過docker history咱們來觀察鏡像層打包的過程
這裏有若干個missing層,實際上是docker打包程序程序自動優化的過程。Docker爲了加快鏡像構建速度,也會將每個鏡像層緩存下來。利用這個特性,能夠提高工做效率,不過能用到這個特性倒是有點尷尬:安全
接下來咱們對相同屬性指令進行合併,好比RUN命令等運維
FROM python:3.5.6-slim-stretch RUN apt-get update &&\ echo python -V &&\ mkdir /code &&\ mkdir /code/db ADD . /code/ WORKDIR /code RUN pip install -U pip &&\ pip install -r requirement
如下是編譯日誌ide
docker build -t spidermax:test_02 .
Sending build context to Docker daemon 289.8kB
Step 1/5 : FROM python:3.5.6-slim-stretch
---> 86669b8e5771
Step 2/5 : RUN apt-get update && echo python -V && mkdir /code && mkdir /code/db
---> Running in 2a879b85221f
Get:1 http://security.debian.org/debian-security stretch/updates InRelease [94.3 kB]
Get:2 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [481 kB]
Ign:3 http://deb.debian.org/debian stretch InRelease
Get:4 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]
Get:5 http://deb.debian.org/debian stretch Release [118 kB]
Get:6 http://deb.debian.org/debian stretch-updates/main amd64 Packages [11.1 kB]
Get:7 http://deb.debian.org/debian stretch Release.gpg [2434 B]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7084 kB]
Fetched 7881 kB in 8s (966 kB/s)
Reading package lists...
python -V
Removing intermediate container 2a879b85221f
---> 5ce22d4611e9
Step 3/5 : ADD . /code/
---> af87134918ca
Step 4/5 : WORKDIR /code
---> Running in 2a5c72670be3
Removing intermediate container 2a5c72670be3
---> 0324df92356d
Step 5/5 : RUN pip install -U pip && pip install -r requirement
---> Running in 439cc1586076
Collecting pip
Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 18.1
Uninstalling pip-18.1:
Successfully uninstalled pip-18.1
Successfully installed pip-19.0.3
Collecting Django==2.1.7 (from -r requirement (line 1))
Downloading https://files.pythonhosted.org/packages/c7/87/fbd666c4f87591ae25b7bb374298e8629816e87193c4099d3608ef11fab9/Django-2.1.7-py3-none-any.whl (7.3MB)
Collecting EasyProcess==0.2.5 (from -r requirement (line 2))
Downloading https://files.pythonhosted.org/packages/45/3a/4eecc0c7995a13a64739bbedc0d3691fc574245b7e79cff81905aa0c2b38/EasyProcess-0.2.5.tar.gz
Collecting PyMySQL==0.9.3 (from -r requirement (line 3))
Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB)
Collecting pytz==2018.9 (from -r requirement (line 4))
Downloading https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl (510kB)
Collecting selenium==3.141.0 (from -r requirement (line 5))
Downloading https://files.pythonhosted.org/packages/80/d6/4294f0b4bce4de0abf13e17190289f9d0613b0a44e5dd6a7f5ca98459853/selenium-3.141.0-py2.py3-none-any.whl (904kB)
Collecting urllib3==1.24.1 (from -r requirement (line 6))
Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)
Building wheels for collected packages: EasyProcess
Building wheel for EasyProcess (setup.py): started
Building wheel for EasyProcess (setup.py): finished with status 'done'
Stored in directory: /root/.cache/pip/wheels/41/22/19/af15ef6264c58b625a82641ed7483ad05e258fbd8925505227
Successfully built EasyProcess
Installing collected packages: pytz, Django, EasyProcess, PyMySQL, urllib3, selenium
Successfully installed Django-2.1.7 EasyProcess-0.2.5 PyMySQL-0.9.3 pytz-2018.9 selenium-3.141.0 urllib3-1.24.1
Removing intermediate container 439cc1586076
---> d767a8a2d9f2
Successfully built d767a8a2d9f2
Successfully tagged spidermax:test_02
雖然在大小和原來沒有啥區別
可是在鏡像打包上,步驟上少了不少,這樣在必定程度上就縮短鏡像打包的時間,這一點從剛纔的打包日誌也有所體現,由於test01版本是須要9步,優化1版本是僅須要5步,打包效率提高將近1倍
咱們所使用的基礎鏡像是python:3.5.6-slim-stretch,該基礎鏡像致使咱們每次打包都須要在其之上進行操做,這樣也致使咱們的獎項將會變得很是的大,畢竟基礎就已經達到135M之大,雖說咱們不該該重複造輪子,可是這樣的輪子咱們要不起啊
咱們不妨從新反思一下咱們所使用的鏡像功能,無非就是python的編譯器、pip以及apt功能,至於說另外的django等文件,是咱們經過requirement來安裝的,編譯器自帶的若干依賴咱們沒怎麼用到(這麼描述,不是很嚴謹),那咱們爲何不乾脆本身部署一個python環境,只要有apt,還有什麼事情沒有不能作的?
這裏推薦使用apline這款鏡像,簡直是惡魔!居然只有4M左右
Alpine 的意思是「高山的」,好比 Alpine plants高山植物,Alpine skiing高山滑雪、the alpine resort阿爾卑斯山勝地。
Alpine Linux 網站首頁註明「Small!Simple!Secure!Alpine Linux is a security-oriented, lightweight Linux distribution based on musl libc and busybox.」歸納瞭如下特色:
FROM alpine:3.7 RUN apk update && apk upgrade &&\ apk add python3 &&\ apk add python3-dev &&\ python3 -m ensurepip &&\ if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi &&\ if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi &&\ mkdir /code && mkdir /code/db ADD . /code/ WORKDIR /code RUN pip install -U pip && pip install -r requirement
由於安裝python3的時候默認會是pip3,爲了安裝方便咱們須要將pip3修改成pip,此外還有將python3修改成python,方便咱們對鏡像的啓動,如下是運行的日誌
docker build -t spidermax:test_03 .
Sending build context to Docker daemon 290.3kB
Step 1/5 : FROM alpine:3.7
---> 6d1ef012b567
Step 2/5 : RUN apk update && apk upgrade && apk add python3 && apk add python3-dev && python3 -m ensurepip && if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && mkdir /code && mkdir /code/db
---> Running in ef033831c1a9
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
v3.7.3-6-g3eacd5f9a6 [http://dl-cdn.alpinelinux.org/alpine/v3.7/main]
v3.7.3-4-g7296a289a6 [http://dl-cdn.alpinelinux.org/alpine/v3.7/community]
OK: 9049 distinct packages available
OK: 4 MiB in 13 packages
(1/11) Installing libbz2 (1.0.6-r6)
(2/11) Installing expat (2.2.5-r0)
(3/11) Installing libffi (3.2.1-r4)
(4/11) Installing gdbm (1.13-r1)
由於安裝python3的時候默認會是pip3,爲了安裝方便咱們須要將pip3修改成pip,此外還有將python3修改成python,方便咱們對鏡像的啓動,如下是運行的日誌
docker build -t spidermax:test_03 .
Sending build context to Docker daemon 290.3kB
Step 1/5 : FROM alpine:3.7
---> 6d1ef012b567
Step 2/5 : RUN apk update && apk upgrade && apk add python3 && apk add python3-dev && python3 -m ensurepip && if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi && if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi && mkdir /code && mkdir /code/db
---> Running in ef033831c1a9
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.7/community/x86_64/APKINDEX.tar.gz
v3.7.3-6-g3eacd5f9a6 [http://dl-cdn.alpinelinux.org/alpine/v3.7/main]
v3.7.3-4-g7296a289a6 http://dl-cdn.alpinelinux.org/alpine/v3.7/community Installing xz-libs (5.2.3-r1)
(6/11) Installing ncurses-terminfo-base (6.0_p20171125-r1)
(7/11) Installing ncurses-terminfo (6.0_p20171125-r1)
(8/11) Installing ncurses-libs (6.0_p20171125-r1)
(9/11) Installing readline (7.0.003-r0)
(10/11) Installing sqlite-libs (3.25.3-r0)
(11/11) Installing python3 (3.6.5-r0)
Executing busybox-1.27.2-r11.trigger
OK: 66 MiB in 24 packages
(1/2) Installing pkgconf (1.3.10-r0)
(2/2) Installing python3-dev (3.6.5-r0)
Executing busybox-1.27.2-r11.trigger
OK: 79 MiB in 26 packages
Requirement already satisfied: setuptools in /usr/lib/python3.6/site-packages
Requirement already satisfied: pip in /usr/lib/python3.6/site-packages
Removing intermediate container ef033831c1a9
---> 4250724684eb
Step 3/5 : ADD . /code/
---> 8f1a939f8ac1
Step 4/5 : WORKDIR /code
---> Running in 137e7b42decb
Removing intermediate container 137e7b42decb
---> 29586daada18
Step 5/5 : RUN pip install -U pip && pip install -r requirement
---> Running in 36baec208714
Collecting pip
Downloading https://files.pythonhosted.org/packages/d8/f3/413bab4ff08e1fc4828dfc59996d721917df8e8583ea85385d51125dceff/pip-19.0.3-py2.py3-none-any.whl (1.4MB)
Installing collected packages: pip
Found existing installation: pip 9.0.3
Uninstalling pip-9.0.3:
Successfully uninstalled pip-9.0.3
Successfully installed pip-19.0.3
Collecting Django==2.1.7 (from -r requirement (line 1))
Downloading https://files.pythonhosted.org/packages/c7/87/fbd666c4f87591ae25b7bb374298e8629816e87193c4099d3608ef11fab9/Django-2.1.7-py3-none-any.whl (7.3MB)
Collecting EasyProcess==0.2.5 (from -r requirement (line 2))
Downloading https://files.pythonhosted.org/packages/45/3a/4eecc0c7995a13a64739bbedc0d3691fc574245b7e79cff81905aa0c2b38/EasyProcess-0.2.5.tar.gz
Collecting PyMySQL==0.9.3 (from -r requirement (line 3))
Downloading https://files.pythonhosted.org/packages/ed/39/15045ae46f2a123019aa968dfcba0396c161c20f855f11dea6796bcaae95/PyMySQL-0.9.3-py2.py3-none-any.whl (47kB)
Collecting pytz==2018.9 (from -r requirement (line 4))
Downloading https://files.pythonhosted.org/packages/61/28/1d3920e4d1d50b19bc5d24398a7cd85cc7b9a75a490570d5a30c57622d34/pytz-2018.9-py2.py3-none-any.whl (510kB)
Collecting selenium==3.141.0 (from -r requirement (line 5))
Downloading https://files.pythonhosted.org/packages/80/d6/4294f0b4bce4de0abf13e17190289f9d0613b0a44e5dd6a7f5ca98459853/selenium-3.141.0-py2.py3-none-any.whl (904kB)
Collecting urllib3==1.24.1 (from -r requirement (line 6))
Downloading https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl (118kB)
Installing collected packages: pytz, Django, EasyProcess, PyMySQL, urllib3, selenium
Running setup.py install for EasyProcess: started
Running setup.py install for EasyProcess: finished with status 'done'
Successfully installed Django-2.1.7 EasyProcess-0.2.5 PyMySQL-0.9.3 pytz-2018.9 selenium-3.141.0 urllib3-1.24.1
Removing intermediate container 36baec208714
---> e59b7105697a
Successfully built e59b7105697a
Successfully tagged spidermax:test_03
檢查鏡像大小,會發現這個鏡像和原來相比咱們將其縮小了近一倍
那這樣已經達到目的了嗎?不還不夠!做爲工程師,優化永不止步!
每一次的鏡像打包,docker都會去雲端請求,下載相關的軟件,並安裝,那安裝以後的安裝包會怎麼處理呢?——還算存留在鏡像中,也就是所謂的緩存,這些是相對應用層來講的,他們只會佔用咱們存儲空間,並無過多的用處,所以須要的就是將這個緩存的內容刪除,實現起來也很簡單的帶上--no-cache便可,如圖見apk操做詳情
修改以後的腳本爲
FROM alpine:3.7 RUN apk update --no-cache && apk upgrade --update-cache --available &&\ apk add --no-cache python3 &&\ apk add --no-cache python3-dev &&\ python3 -m ensurepip &&\ if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi &&\ if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi &&\ rm -rf /var/lib/apk/* &&\ mkdir /code && mkdir /code/db ADD . /code/ WORKDIR /code RUN pip install -U pip && pip install -r requirement
這樣的話,會致使一個問題,就是每次打包都須要一次又一次的從新下載資源,這個其實也是一個效率上的矛盾,一個互相博弈的過程,鏡像大小變化
固然了,咱們可刪除apk的緩存固然能夠刪除pip的緩存啦,咱們能夠經過直接刪除pip的安裝包的形式來減少文件的大小,代碼以下:
FROM alpine:3.7 RUN apk update && apk upgrade --update-cache --available &&\ apk add python3 &&\ apk add python3-dev &&\ python3 -m ensurepip &&\ rm -r /usr/lib/python*/ensurepip &&\ if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi &&\ if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi &&\ rm -rf /var/lib/apk/* &&\ mkdir /code && mkdir /code/db ADD . /code/ WORKDIR /code RUN pip install -U pip && pip install -r requirement && rm -rf ~/.cache/pip
鏡像的變化以下:
經過咱們的一次次優化以後,咱們將一個高達203M的鏡像進行了優化,縮小了將近一倍,在這一次次的編譯過程當中,也體會到了打包速度一次比一次的提升,這樣纔是優化思想的應用,一次又一次的超越本身
最後給這個dockerfile標記維護者的信息,表示說這個鏡像是我提供的,若是有問題了須要幫忙解決能夠聯繫我,也表示我對這個鏡像擁有著做權,雖然我已經將這個鏡像開源了
FROM alpine:3.7 MAINTAINER yerikyu "yerik_shu@139.com" RUN apk update && apk upgrade --update-cache --available &&\ apk add python3 &&\ apk add python3-dev &&\ python3 -m ensurepip &&\ rm -r /usr/lib/python*/ensurepip &&\ if [ ! -e /usr/bin/pip ]; then ln -s pip3 /usr/bin/pip ; fi &&\ if [[ ! -e /usr/bin/python ]]; then ln -sf /usr/bin/python3 /usr/bin/python; fi &&\ rm -rf /var/lib/apk/* &&\ mkdir /code && mkdir /code/db ADD . /code/ WORKDIR /code RUN pip install -U pip && pip install -r requirement && rm -rf ~/.cache/pip
打包結果以下:
docker鏡像層:http://blog.daocloud.io/principle-of-docker-image/
alpine官方資料:https://alpinelinux.org/