本文使用「署名 4.0 國際 (CC BY 4.0)」許可協議,歡迎轉載、或從新修改使用,但須要註明來源。 署名 4.0 國際 (CC BY 4.0)html
本文做者: 蘇洋node
建立時間: 2019年09月08日 統計字數: 15348字 閱讀時間: 31分鐘閱讀 本文連接: soulteary.com/2019/09/08/…linux
去年的時候,我曾經寫過如何簡單搭建 Kubernetes 集羣,當時使用的是官方的工具箱:Kubeadm,這個方案對於只是想試試的同窗來講,仍是過於複雜。這裏介紹一款簡單的工具:MicroK8s。nginx
官方給這款工具的人設是「無需運維的 Kubernetes ,服務於工做站、物聯網。」最大的價值在於能夠快速搭建單節點的容器編排系統,用於生產試驗。git
官方網站裏的文檔有簡單介紹如何安裝使用,可是卻不曾考慮安裝過程存在網絡問題的神州大陸的同窗們,本文將結合這種狀況聊聊。github
官方在早些時候宣佈接下來將會使用 Containerd
替換 docker
:docker
The upcoming release of v1.14 Kubernetes will mark the MicroK8s switch to Containerd and enhanced security. As this is a big step forward we would like to give you a heads up and offer you a preview of what is coming. Give it a test drive with:編程
snap install microk8s --classic --channel=1.13/edge/secure-containerdubuntu
You can read more in our blog 117, and the respective pill request 13 Please, let us know 5 how we can make this transition smoother for you. Thanksvim
社區裏已經有用戶諮詢/吐槽過了,這裏考慮減小變化,暫時仍是以使用 docker 做爲容器封裝的 1.13 ,新版本留給下一篇「折騰」吧。
snap 是 **canonical ** 公司給出的更「高級」的包管理的解決方案,最先應用在 Ubuntu Phone 上。
使用 snap 安裝 K8s 確實很簡單,就像下面同樣,一條命令解決問題:
snap install microk8s --classic --channel=1.13/stable
複製代碼
可是這條命令若是不是在海外主機上執行,應該會遇到安裝緩慢的問題。
snap install microk8s --classic --channel=1.13/stable
Download snap "microk8s" (581) from channel "1.13/stable" 0% 25.9kB/s 2h32m
複製代碼
想要解決這個問題,暫時只能給 snap 添加代理來解決問題,snap 不會讀取系統的環境變量,只讀取應用的變量文件。
使用下面的命令能夠方便的修改 snap 的環境變量,可是默認編輯器是 ** nano **,很是難用。
systemctl edit snapd.service
複製代碼
這裏能夠先更新編輯器爲咱們熟悉的 ** vim **:
sudo update-alternatives --install "$(which editor)" editor "$(which vim)" 15
sudo update-alternatives --config editor
複製代碼
交互式終端須要咱們手動輸入數字,而後按下回車確認選擇:
There are 5 choices for the alternative editor (providing /usr/bin/editor).
Selection Path Priority Status
------------------------------------------------------------
* 0 /bin/nano 40 auto mode
1 /bin/ed -100 manual mode
2 /bin/nano 40 manual mode
3 /usr/bin/vim 15 manual mode
4 /usr/bin/vim.basic 30 manual mode
5 /usr/bin/vim.tiny 15 manual mode
Press <enter> to keep the current choice[*], or type selection number: 5
update-alternatives: using /usr/bin/vim.tiny to provide /usr/bin/editor (editor) in manual mode
複製代碼
再次執行上面編輯環境變量的命令,添加一段代理配置:
[Service]
Environment="HTTP_PROXY=http://10.11.12.123:10240"
Environment="HTTPS_PROXY=http://10.11.12.123:10240"
Environment="NO_PROXY=localhost,127.0.0.1,192.168.0.0/24,*.domain.ltd"
複製代碼
再次執行安裝,安裝進度起飛:
snap install microk8s --classic --channel=1.13/stable
Download snap "microk8s" (581) from channel "1.13/stable" 31% 14.6MB/s 11.2s
複製代碼
若是速度沒有變化,能夠考慮重載 snap 服務。
systemctl daemon-reload && systemctl restart snapd
複製代碼
若是上面的操做一切順利,你將會看到相似下面的日誌:
snap install microk8s --classic --channel=1.13/stable
microk8s (1.13/stable) v1.13.6 from Canonical✓ installed
複製代碼
執行列表命令,能夠看到當前 snap 已經安裝好的工具:
snap list
Name Version Rev Tracking Publisher Notes
core 16-2.40 7396 stable canonical✓ core
microk8s v1.13.6 581 1.13 canonical✓ classic
複製代碼
以前 獨立安裝 K8s 須要先安裝 docker,而使用 snap 安裝的話,這一切都是默認就緒的。
microk8s.docker version
Client:
Version: 18.09.2
API version: 1.39
Go version: go1.10.4
Git commit: 6247962
Built: Tue Feb 26 23:56:24 2019
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 18.09.2
API version: 1.39 (minimum version 1.12)
Go version: go1.10.4
Git commit: 6247962
Built: Tue Feb 12 22:47:29 2019
OS/Arch: linux/amd64
Experimental: false
複製代碼
想要使用 Kubernetes,除了安裝 MicroK8s 外,還須要獲取它依賴的工具鏡像。然而鏡像的獲取仍是須要費點功夫的,首先獲取 1.13
版本的 MicroK8s 代碼:
git clone --single-branch --branch=1.13 https://github.com/ubuntu/microk8s.git
複製代碼
而後獲取其中聲明的容器鏡像列表:
grep -ir 'image:' * | awk '{print $3 $4}' | uniq
複製代碼
由於官方代碼的奔放,咱們會得到長得奇形怪狀的鏡像名稱:
localhost:32000/my-busybox
elasticsearch:6.5.1
alpine:3.6
docker.elastic.co/kibana/kibana-oss:6.3.2
time="2016-02-04T07:53:57.505612354Z"level=error
cdkbot/registry-$ARCH:2.6
...
...
quay.io/prometheus/prometheus
quay.io/coreos/kube-rbac-proxy:v0.4.0
k8s.gcr.io/metrics-server-$ARCH:v0.2.1
cdkbot/addon-resizer-$ARCH:1.8.1
cdkbot/microbot-$ARCH
"k8s.gcr.io/cuda-vector-add:v0.1"
nginx:latest
istio/examples-bookinfo-details-v1:1.8.0
busybox
busybox:1.28.4
複製代碼
根據咱們要部署的目標服務器的具體需求,替換掉 $ARCH
變量,去掉無心義的鏡像名稱,整理好的列表以下:
k8s.gcr.io/fluentd-elasticsearch:v2.2.0
elasticsearch:6.5.1
alpine:3.6
docker.elastic.co/kibana/kibana-oss:6.3.2
cdkbot/registry-amd64:2.6
gcr.io/google_containers/defaultbackend-amd64:1.4
quay.io/kubernetes-ingress-controller/nginx-ingress-controller-amd64:0.22.0
jaegertracing/jaeger-operator:1.8.1
gcr.io/google_containers/k8s-dns-kube-dns-amd64:1.14.7
gcr.io/google_containers/k8s-dns-dnsmasq-nanny-amd64:1.14.7
gcr.io/google_containers/k8s-dns-sidecar-amd64:1.14.7
k8s.gcr.io/kubernetes-dashboard-amd64:v1.8.3
k8s.gcr.io/heapster-influxdb-amd64:v1.3.3
k8s.gcr.io/heapster-grafana-amd64:v4.4.3
k8s.gcr.io/heapster-amd64:v1.5.2
cdkbot/addon-resizer-amd64:1.8.1
cdkbot/hostpath-provisioner-amd64:latest
quay.io/coreos/k8s-prometheus-adapter-amd64:v0.3.0
grafana/grafana:5.2.4
quay.io/coreos/kube-rbac-proxy:v0.4.0
quay.io/coreos/kube-state-metrics:v1.4.0
quay.io/coreos/addon-resizer:1.0
quay.io/prometheus/prometheus
quay.io/coreos/prometheus-operator:v0.25.0
quay.io/prometheus/alertmanager
quay.io/prometheus/node-exporter:v0.16.0
quay.io/coreos/kube-rbac-proxy:v0.4.0
k8s.gcr.io/metrics-server-amd64:v0.2.1
cdkbot/addon-resizer-amd64:1.8.1
nvidia/k8s-device-plugin:1.11
cdkbot/microbot-amd64
k8s.gcr.io/cuda-vector-add:v0.1
nginx:latest
istio/examples-bookinfo-details-v1:1.8.0
istio/examples-bookinfo-ratings-v1:1.8.0
istio/examples-bookinfo-reviews-v1:1.8.0
istio/examples-bookinfo-reviews-v2:1.8.0
istio/examples-bookinfo-reviews-v3:1.8.0
istio/examples-bookinfo-productpage-v1:1.8.0
busybox
busybox:1.28.4
複製代碼
將上面的列表保存爲 package-list.txt 在網絡通暢的雲服務器上使用下面的腳本,能夠將 K8s 依賴的工具鏡像離線保存:
PACKAGES=`cat ./package-list.txt`;
for package in $PACKAGES; do docker pull "$package"; done
docker images | tail -n +2 | grep -v "<none>" | awk '{printf("%s:%s\n", $1, $2)}' | while read IMAGE; do
for package in $PACKAGES;
do
if [[ $package != *[':']* ]];then package="$package:latest"; fi
if [ $IMAGE == $package ];then
echo "[find image] $IMAGE"
filename="$(echo $IMAGE| tr ':' '-' | tr '/' '-').tar"
echo "[save as] $filename"
docker save ${IMAGE} -o $filename
fi
done
done
複製代碼
將鏡像轉存待部署服務器的方式多種多樣,這裏提一種最簡單的方案:scp
PACKAGES=`cat ./package-list.txt`;
for package in $PACKAGES;
do
if [[ $package != *[':']* ]];then package="$package:latest";fi
filename="$(echo $package| tr ':' '-' | tr '/' '-').tar"
# 根據本身實際場景修改地址
scp "mirror-server:~/images/$filename" .
scp "./$filename" "deploy-server:"
done
複製代碼
若是順利你將看到相似下面的日誌:
k8s.gcr.io-fluentd-elasticsearch-v2.2.0.tar 100% 140MB 18.6MB/s 00:07
elasticsearch-6.5.1.tar 100% 748MB 19.4MB/s 00:38
alpine-3.6.tar 100% 4192KB 15.1MB/s 00:00
docker.elastic.co-kibana-kibana-oss-6.3.2.tar 100% 614MB 22.8MB/s 00:26
cdkbot-registry-amd64-2.6.tar 100% 144MB 16.1MB/s 00:08
gcr.io-google_containers-defaultbackend-amd64-1.4.tar 100% 4742KB 13.3MB/s 00:00
...
...
複製代碼
最後在目標服務器使用 docker load
命令導入鏡像便可。
ls *.tar | xargs -I {} microk8s.docker load -i {}
複製代碼
接下來能夠正式安裝 K8s 啦。
使用 MicroK8s 配置各類組件很簡單,只須要一條命令:
microk8s.enable dashboard dns ingress istio registry storage
複製代碼
完整的組件列表能夠經過 microk8s.enable --help
來查看:
microk8s.enable --help
Usage: microk8s.enable ADDON...
Enable one or more ADDON included with microk8s
Example: microk8s.enable dns storage
Available addons:
dashboard
dns
fluentd
gpu
ingress
istio
jaeger
metrics-server
prometheus
registry
storage
複製代碼
執行 enable
順利的話,你將看到相似下面的日誌:
logentry.config.istio.io/accesslog created
logentry.config.istio.io/tcpaccesslog created
rule.config.istio.io/stdio created
rule.config.istio.io/stdiotcp created
...
...
Istio is starting
Enabling the private registry
Enabling default storage class
deployment.extensions/hostpath-provisioner created
storageclass.storage.k8s.io/microk8s-hostpath created
Storage will be available soon
Applying registry manifest
namespace/container-registry created
persistentvolumeclaim/registry-claim created
deployment.extensions/registry created
service/registry created
The registry is enabled
Enabling default storage class
deployment.extensions/hostpath-provisioner unchanged
storageclass.storage.k8s.io/microk8s-hostpath unchanged
Storage will be available soon
複製代碼
使用 microk8s.status
檢查各個組件的狀態:
microk8s is running
addons:
jaeger: disabled
fluentd: disabled
gpu: disabled
storage: enabled
registry: enabled
ingress: enabled
dns: enabled
metrics-server: disabled
prometheus: disabled
istio: enabled
dashboard: enabled
複製代碼
可是組件就緒,不表明 K8s 已經安裝就緒,使用 microk8s.inspect
排查下安裝部署結果:
Inspecting services
Service snap.microk8s.daemon-containerd is running
Service snap.microk8s.daemon-docker is running
Service snap.microk8s.daemon-apiserver is running
Service snap.microk8s.daemon-proxy is running
Service snap.microk8s.daemon-kubelet is running
Service snap.microk8s.daemon-scheduler is running
Service snap.microk8s.daemon-controller-manager is running
Service snap.microk8s.daemon-etcd is running
Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system info
Copy network configuration to the final report tarball
Copy processes list to the final report tarball
Copy snap list to the final report tarball
Inspect kubernetes cluster
WARNING: IPtables FORWARD policy is DROP. Consider enabling traffic forwarding with: sudo iptables -P FORWARD ACCEPT
複製代碼
解決方法很簡單,使用 ufw
添加幾條規則便可:
sudo ufw allow in on cbr0 && sudo ufw allow out on cbr0
sudo ufw default allow routed
sudo iptables -P FORWARD ACCEPT
複製代碼
再次使用 microk8s.inspect
命令檢查,會發現 WARNING 已經消失了。
可是 Kubernetes 真的安裝就緒了嗎?跟隨下一小節尋找答案吧。
在上面的操做順利以後完畢後,使用 microk8s.kubectl get pods
查看當前 Kubernetes pods 狀態,若是看到 ContainerCreating
,那麼說明 Kubernetes 還須要一些額外的「修補工做」。
NAME READY STATUS RESTARTS AGE
default-http-backend-855bc7bc45-t4st8 0/1 ContainerCreating 0 16m
nginx-ingress-microk8s-controller-kgjtl 0/1 ContainerCreating 0 16m
複製代碼
使用 microk8s.kubectl get pods --all-namespaces
查看詳細的狀態,不出意外的話,將看到相似下面的日誌輸出:
NAMESPACE NAME READY STATUS RESTARTS AGE
container-registry registry-7fc4594d64-rrgs9 0/1 Pending 0 15m
default default-http-backend-855bc7bc45-t4st8 0/1 ContainerCreating 0 16m
default nginx-ingress-microk8s-controller-kgjtl 0/1 ContainerCreating 0 16m
...
...
複製代碼
首要的問題就是解決掉這個處於 Pending 狀態的容器。使用 microk8s.kubectl describe pod
能夠快速查看當前這個問題 pod 的詳細狀態:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 22m default-scheduler Successfully assigned default/default-http-backend-855bc7bc45-t4st8 to ubuntu-basic-18-04
Warning FailedCreatePodSandBox 21m kubelet, ubuntu-basic-18-04 Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
Warning FailedCreatePodSandBox 43s (x45 over 21m) kubelet, ubuntu-basic-18-04 Failed create pod sandbox: rpc error: code = Unknown desc = failed pulling image "k8s.gcr.io/pause:3.1": Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
複製代碼
參考日誌輸出,能夠發現,以前整理的依賴鏡像列表,還存在「漏網之魚」: MicroK8s 中還包含未寫在程序中,從遠程配置中獲取的鏡像。
對於這種狀況,咱們只能經過給 docker
添加代理來解決問題(或者手動一個一個來)。
編輯 MicroK8s 使用的 docker 環境變量配置文件 vi /var/snap/microk8s/current/args/dockerd-env
,在其中添加代理配置,好比:
HTTPS_PROXY=http://10.11.12.123:10555
HTTPS_PROXY=http://10.11.12.123:10555
NO_PROXY=127.0.0.1
複製代碼
接着重啓 docker :
sudo systemctl restart snap.microk8s.daemon-docker.service
複製代碼
這一切就緒以後,執行下面的命令,重置 MicroK8s 並再次嘗試安裝各類組件:
microk8s.reset
microk8s.enable dashboard dns ingress istio registry storage
複製代碼
命令以後完畢以後,片刻以後再次執行 microk8s.kubectl get pods
會發現全部的 pod 的狀態就都是 Running:
NAME READY STATUS RESTARTS AGE
default-http-backend-855bc7bc45-w62jd 1/1 Running 0 46s
nginx-ingress-microk8s-controller-m9lc2 1/1 Running 0 46s
複製代碼
使用 microk8s.kubectl get pods --all-namespaces
繼續進行驗證:
NAMESPACE NAME READY STATUS RESTARTS AGE
container-registry registry-7fc4594d64-whjnl 1/1 Running 0 2m
default default-http-backend-855bc7bc45-w62jd 1/1 Running 0 2m
default nginx-ingress-microk8s-controller-m9lc2 1/1 Running 0 2m
istio-system grafana-59b8896965-xtc27 1/1 Running 0 2m
istio-system istio-citadel-856f994c58-fbc7c 1/1 Running 0 2m
istio-system istio-cleanup-secrets-9q8tw 0/1 Completed 0 2m
istio-system istio-egressgateway-5649fcf57-cbqlv 1/1 Running 0 2m
istio-system istio-galley-7665f65c9c-l7grc 1/1 Running 0 2m
istio-system istio-grafana-post-install-sl6mb 0/1 Completed 0 2m
istio-system istio-ingressgateway-6755b9bbf6-hvnld 1/1 Running 0 2m
istio-system istio-pilot-698959c67b-zts2v 2/2 Running 0 2m
istio-system istio-policy-6fcb6d655f-mx68m 2/2 Running 0 2m
istio-system istio-security-post-install-5d7bb 0/1 Completed 0 2m
istio-system istio-sidecar-injector-768c79f7bf-qvcjd 1/1 Running 0 2m
istio-system istio-telemetry-664d896cf5-jz22s 2/2 Running 0 2m
istio-system istio-tracing-6b994895fd-z8jn9 1/1 Running 0 2m
istio-system prometheus-76b7745b64-fqvn9 1/1 Running 0 2m
istio-system servicegraph-5c4485945b-spf77 1/1 Running 0 2m
kube-system heapster-v1.5.2-64874f6bc6-8ghnr 4/4 Running 0 2m
kube-system hostpath-provisioner-599db8d5fb-kxtjw 1/1 Running 0 2m
kube-system kube-dns-6ccd496668-98mvt 3/3 Running 0 2m
kube-system kubernetes-dashboard-654cfb4879-vzgk5 1/1 Running 0 2m
kube-system monitoring-influxdb-grafana-v4-6679c46745-68vn7 2/2 Running 0 2m
複製代碼
若是你看到的結果相似上面這樣,說明 Kubernetes 是真的就緒了。
安都安完了,總得試着玩玩看吧,固然,這裏不會隨大流的展現下管理後臺就匆匆擱筆。
使用 kubectl
基於現成的容器建立一個 deployment:
microk8s.kubectl create deployment microbot --image=dontrebootme/microbot:v1
複製代碼
既然用上了最早進的編排系統,不體驗下自動擴容豈不是太惋惜了:
microk8s.kubectl scale deployment microbot --replicas=2
複製代碼
將服務暴露出來,建立流量轉發:
microk8s.kubectl expose deployment microbot --type=NodePort --port=80 --name=microbot-service
複製代碼
使用 get
命令查看服務狀態:
microk8s.kubectl get all
複製代碼
若是一切順利的話,你將會看到相似下面的日誌輸出:
NAME READY STATUS RESTARTS AGE
pod/default-http-backend-855bc7bc45-w62jd 1/1 Running 0 64m
pod/microbot-7c7594fb4-dxgg7 1/1 Running 0 13m
pod/microbot-7c7594fb4-v9ztg 1/1 Running 0 13m
pod/nginx-ingress-microk8s-controller-m9lc2 1/1 Running 0 64m
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/default-http-backend ClusterIP 10.152.183.13 <none> 80/TCP 64m
service/kubernetes ClusterIP 10.152.183.1 <none> 443/TCP 68m
service/microbot-service NodePort 10.152.183.15 <none> 80:31354/TCP 13m
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
daemonset.apps/nginx-ingress-microk8s-controller 1 1 1 1 1 <none> 64m
NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/default-http-backend 1/1 1 1 64m
deployment.apps/microbot 2/2 2 2 13m
NAME DESIRED CURRENT READY AGE
replicaset.apps/default-http-backend-855bc7bc45 1 1 1 64m
replicaset.apps/microbot-7c7594fb4 2 2 2 13m
複製代碼
能夠看到咱們剛剛建立的 Service 地址是 10.11.12.234:31354
。使用瀏覽器訪問,能夠看到應用已經跑起來啦。
本着「誰製造誰收拾」的綠色環保理念,除了「無腦」建立外,咱們也須要學會如何治理(銷燬),使用 delete
命令,先銷燬 deployment :
microk8s.kubectl delete deployment microbot
複製代碼
執行完畢後日志輸出會是下面同樣:
deployment.extensions "microbot" deleted
複製代碼
在銷燬 service 前,咱們須要使用 get
命令先獲取全部的 service 的名稱:
microk8s.kubectl get services microbot-service
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
microbot-service NodePort 10.152.183.15 <none> 80:31354/TCP 24m
複製代碼
獲得了 service 名稱後,依舊是使用 delete
命令,刪除不須要的資源:
microk8s.kubectl delete service microbot-service
複製代碼
執行結果以下:
service "microbot-service" deleted
複製代碼
估計仍是有同窗會想一窺 Dashboard 的情況。
能夠經過 microk8s.config
命令,先得到當前服務器監聽的 IP 地址:
microk8s.config
apiVersion: v1
clusters:
- cluster:
server: http://10.11.12.234:8080
name: microk8s-cluster
contexts:
- context:
cluster: microk8s-cluster
user: admin
name: microk8s
current-context: microk8s
kind: Config
preferences: {}
users:
- name: admin
user:
username: admin
複製代碼
能夠看到,當前監聽的服務 IP 地址爲 10.11.12.234
,使用 proxy
命令,打開流量轉發:
microk8s.kubectl proxy --accept-hosts=.* --address=0.0.0.0
複製代碼
接着訪問下面的地址,就能看到咱們熟悉的 Dashboard 啦:
http://10.11.12.234:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/#!/login
複製代碼
完整安裝下來,連着系統一共花費了接近 8G 的儲存空間,因此若是你打算持續使用的話,能夠提早規劃下磁盤空間,好比參考 遷移 Docker 容器儲存位置 把將來持續膨脹的 docker 鏡像包地址先作個遷移。
df -h
Filesystem Size Used Avail Use% Mounted on
udev 7.8G 0 7.8G 0% /dev
tmpfs 1.6G 1.7M 1.6G 1% /run
/dev/sda2 79G 16G 59G 21% /
複製代碼
這篇文章成文於一個月以前,因爲使用的仍是 「Docker」 方案,理論來講時效性仍是靠譜的,若是你遇到了什麼問題,歡迎討論溝通。
看着草稿箱堆積愈來愈多的有趣內容,或許應該考慮「合做撰寫」的模式了。
—EOF
我如今有一個小小的折騰羣,裏面彙集了一些喜歡折騰的小夥伴。
在不發廣告的狀況下,咱們在裏面會一塊兒聊聊軟件、HomeLab、編程上的一些問題,也會在羣裏不按期的分享一些技術沙龍的資料。
喜歡折騰的小夥伴歡迎掃碼添加好友。(請註明來源和目的,不然不會經過審覈)