K8S 1.10.1 高可用環境搭建實戰
服務器信息以下：node
| 主機名稱 | IP | 備註 |
|---|---|---|
| node01 | 10.150.27.51 | master and etcd |
| rode02 | 10.150.27.65 | master and etcd |
| node03 | 10.150.27.66 | node |
| VIP | 10.150.27.99 |
軟件版本：linux
docker17.03.2-ce
socat-1.7.3.2-2.el7.x86_64
kubelet-1.10.0-0.x86_64
kubernetes-cni-0.6.0-0.x86_64
kubectl-1.10.0-0.x86_64
kubeadm-1.10.0-0.x86_64git
參考文檔：https://github.com/cookeem/kubeadm-ha/blob/master/README_CN.mdgithub
1:環境初始化
1：分別在三臺主機設置主機名稱docker
hostnamectl set-hostname node01 hostnamectl set-hostname node02 hostnamectl set-hostname node03
2:配置主機映射express
cat <<EOF > /etc/hosts 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost localhost.localdomain localhost6 localhost6.localdomain6 ## k8s 10.150.27.51 node01 10.150.27.65 node02 10.150.27.66 node03 EOF
3：node01上執行ssh免密碼登錄配置apache
ssh-keygen #一路回車便可 ssh-copy-id node02 ssh-copy-id node03
4：三臺主機配置、停防火牆、關閉Swap、關閉Selinux、設置內核、K8S的yum源、安裝依賴包、配置ntp（配置完後建議重啓一次）json
systemctl stop firewalld systemctl disable firewalld swapoff -a sed -i 's/.*swap.*/#&/' /etc/fstab setenforce 0 sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/sysconfig/selinux sed -i "s/^SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config sed -i "s/^SELINUX=permissive/SELINUX=disabled/g" /etc/sysconfig/selinux sed -i "s/^SELINUX=permissive/SELINUX=disabled/g" /etc/selinux/config modprobe br_netfilter cat <<EOF > /etc/sysctl.d/k8s.conf net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF sysctl -p /etc/sysctl.d/k8s.conf ls /proc/sys/net/bridge cat <<EOF > /etc/yum.repos.d/kubernetes.repo [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/ enabled=1 gpgcheck=1 repo_gpgcheck=1 gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOF yum install -y epel-release yum install -y yum-utils device-mapper-persistent-data lvm2 net-tools conntrack-tools wget vim ntpdate libseccomp libtool-ltdl systemctl enable ntpdate.service echo '*/30 * * * * /usr/sbin/ntpdate time7.aliyun.com >/dev/null 2>&1' > /tmp/crontab2.tmp crontab /tmp/crontab2.tmp systemctl start ntpdate.service echo "* soft nofile 65536" >> /etc/security/limits.conf echo "* hard nofile 65536" >> /etc/security/limits.conf echo "* soft nproc 65536" >> /etc/security/limits.conf echo "* hard nproc 65536" >> /etc/security/limits.conf echo "* soft memlock unlimited" >> /etc/security/limits.conf echo "* hard memlock unlimited" >> /etc/security/limits.conf
2：安裝、配置keepalived（主節點）
1:安裝keepalivedvim
yum install -y keepalived systemctl enable keepalived
==node01的keepalived.conf==centos
cat <<EOF > /etc/keepalived/keepalived.conf
global_defs {
router_id LVS_k8s
}
#設置一個腳原本檢測K8S API的運行狀態，腳本執行間隔爲3s。
vrrp_script CheckK8sMaster {
script "curl -k https://10.150.27.99:6443"
interval 3
timeout 9
fall 2
rise 2
}
vrrp_instance VI_1 { ##集羣的名稱
state MASTER ##指定keepalived的角色
interface eno16780032 ## VIP綁定的網卡位置
virtual_router_id 61 ## VRID兩臺機器必須一致
priority 100 ##優先級，數字越大，優先級越高
advert_int 1 ##主備之間同步檢查的時間間隔
mcast_src_ip 10.150.27.51
nopreempt ##不搶佔模式
authentication {
auth_type PASS
auth_pass sqP05dQgMSlzrxHj ##驗證密碼，主備保持一致
}
unicast_peer {
10.150.27.65
}
virtual_ipaddress {
10.150.27.99/24 ## VIP
}
track_script {
CheckK8sMaster ##執行監控的服務
}
}
==node02的keepalived.conf==
cat <<EOF > /etc/keepalived/keepalived.conf
global_defs {
router_id LVS_k8s
}
vrrp_script CheckK8sMaster {
script "curl -k https://10.150.27.99:6443"
interval 3
timeout 9
fall 2
rise 2
}
vrrp_instance VI_1 {
state BACKUP
interface eno16780032
virtual_router_id 61
priority 90
advert_int 1
mcast_src_ip 10.150.27.65
nopreempt
authentication {
auth_type PASS
auth_pass sqP05dQgMSlzrxHj
}
unicast_peer {
10.150.27.51
}
virtual_ipaddress {
10.150.27.99/24
}
track_script {
CheckK8sMaster
}
}
2：啓動keepalived
systemctl restart keepalived
能夠看到VIP已經綁定到node01上面了
2: eno16780032: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:50:56:aa:5b:e8 brd ff:ff:ff:ff:ff:ff
inet 10.150.27.51/24 brd 10.150.27.255 scope global eno16780032
valid_lft forever preferred_lft forever
inet 10.150.27.99/24 scope global secondary eno16780032
valid_lft forever preferred_lft forever
3：建立etcd證書(node01上執行便可)
1：設置cfssl環境
wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 wget https://pkg.cfssl.org/R1.2/cfssl-certinfo_linux-amd64 chmod +x cfssl_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl chmod +x cfssljson_linux-amd64 mv cfssljson_linux-amd64 /usr/local/bin/cfssljson chmod +x cfssl-certinfo_linux-amd64 mv cfssl-certinfo_linux-amd64 /usr/local/bin/cfssl-certinfo export PATH=/usr/local/bin:$PATH
2：建立 CA 配置文件（下面配置的IP爲etc節點的IP）
mkdir /root/ssl
cd /root/ssl
cat > ca-config.json <<EOF
{
"signing": {
"default": {
"expiry": "8760h"
},
"profiles": {
"kubernetes-Soulmate": {
"usages": [
"signing",
"key encipherment",
"server auth",
"client auth"
],
"expiry": "8760h"
}
}
}
}
EOF
cat > ca-csr.json <<EOF
{
"CN": "kubernetes-Soulmate",
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "shanghai",
"L": "shanghai",
"O": "k8s",
"OU": "System"
}
]
}
EOF
cfssl gencert -initca ca-csr.json | cfssljson -bare ca
cat > etcd-csr.json <<EOF
{
"CN": "etcd",
"hosts": [
"127.0.0.1",
"10.150.27.51",
"10.150.27.65"
],
"key": {
"algo": "rsa",
"size": 2048
},
"names": [
{
"C": "CN",
"ST": "shanghai",
"L": "shanghai",
"O": "k8s",
"OU": "System"
}
]
}
EOF
cfssl gencert -ca=ca.pem \
-ca-key=ca-key.pem \
-config=ca-config.json \
-profile=kubernetes-Soulmate etcd-csr.json | cfssljson -bare etcd
3:node01分發etcd證書到node02上面
mkdir -p /etc/etcd/ssl cp etcd.pem etcd-key.pem ca.pem /etc/etcd/ssl/ ssh -n node02 "mkdir -p /etc/etcd/ssl && exit" scp -r /etc/etcd/ssl/*.pem node02:/etc/etcd/ssl/
4：安裝配置etcd (兩主節點）實際使用須要奇數節點，類同zookeeper
1:安裝etcd
yum install etcd -y mkdir -p /var/lib/etcd
==node01的etcd.service==
cat <<EOF >/etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \
--name node01 \
--cert-file=/etc/etcd/ssl/etcd.pem \
--key-file=/etc/etcd/ssl/etcd-key.pem \
--peer-cert-file=/etc/etcd/ssl/etcd.pem \
--peer-key-file=/etc/etcd/ssl/etcd-key.pem \
--trusted-ca-file=/etc/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \
--initial-advertise-peer-urls https://10.150.27.51:2380 \
--listen-peer-urls https://10.150.27.51:2380 \
--listen-client-urls https://10.150.27.51:2379,http://127.0.0.1:2379 \
--advertise-client-urls https://10.150.27.51:2379 \
--initial-cluster-token etcd-cluster-0 \
--initial-cluster node01=https://10.150.27.51:2380,node02=https://10.150.27.65:2380 \
--initial-cluster-state new \
--data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
==node02的etcd.service==
cat <<EOF >/etc/systemd/system/etcd.service
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target
Documentation=https://github.com/coreos
[Service]
Type=notify
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/bin/etcd \
--name node02 \
--cert-file=/etc/etcd/ssl/etcd.pem \
--key-file=/etc/etcd/ssl/etcd-key.pem \
--peer-cert-file=/etc/etcd/ssl/etcd.pem \
--peer-key-file=/etc/etcd/ssl/etcd-key.pem \
--trusted-ca-file=/etc/etcd/ssl/ca.pem \
--peer-trusted-ca-file=/etc/etcd/ssl/ca.pem \
--initial-advertise-peer-urls https://10.150.27.65:2380 \
--listen-peer-urls https://10.150.27.65:2380 \
--listen-client-urls https://10.150.27.65:2379,http://127.0.0.1:2379 \
--advertise-client-urls https://10.150.27.65:2379 \
--initial-cluster-token etcd-cluster-0 \
--initial-cluster node01=https://10.150.27.51:2380,node02=https://10.150.27.65:2380 \
--initial-cluster-state new \
--data-dir=/var/lib/etcd
Restart=on-failure
RestartSec=5
LimitNOFILE=65536
[Install]
WantedBy=multi-user.target
EOF
2：添加自啓動（etcd集羣最少2個節點才能啓動，啓動報錯看mesages日誌）
mv etcd.service /usr/lib/systemd/system/ systemctl daemon-reload systemctl enable etcd systemctl start etcd systemctl status etcd
3：在兩個etcd節點執行一下命令檢查
etcdctl --endpoints=https://10.150.27.51:2379,https://10.150.27.65:2379,https://192.168.150.183:2379 \ --ca-file=/etc/etcd/ssl/ca.pem \ --cert-file=/etc/etcd/ssl/etcd.pem \ --key-file=/etc/etcd/ssl/etcd-key.pem cluster-health
顯示健康狀態即爲正常：
member 753a005b7804171f is healthy: got healthy result from https://10.150.27.65:2379 member e8aa5c83cd4f744a is healthy: got healthy result from https://10.150.27.51:2379 cluster is healthy
5：全部節點安裝配置docker
1:安裝docker（kubeadm目前支持docker最高版本是17.03.x）
yum install https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/stable/Packages/docker-ce-selinux-17.03.2.ce-1.el7.centos.noarch.rpm -y yum install https://mirrors.aliyun.com/docker-ce/linux/centos/7/x86_64/stable/Packages/docker-ce-17.03.2.ce-1.el7.centos.x86_64.rpm -y
修改配置文件
vim /usr/lib/systemd/system/docker.service
ExecStart=/usr/bin/dockerd -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock --registry-mirror=https://ms3cfraz.mirror.aliyuncs.com
啓動docker
systemctl daemon-reload systemctl restart docker systemctl enable docker systemctl status docker
6: 安裝、配置kubeadm
1：全部節點安裝kubelet kubeadm kubectl
yum install -y kubelet kubeadm kubectl systemctl enable kubelet
2: 全部節點修改kubelet配置文件
/etc/systemd/system/kubelet.service.d/10-kubeadm.conf
#修改這一行 Environment="KUBELET_CGROUP_ARGS=--cgroup-driver=cgroupfs" #添加這一行 Environment="KUBELET_EXTRA_ARGS=--v=2 --fail-swap-on=false --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/k8sth/pause-amd64:3.0"
3：全部節點修改完配置文件必定要從新加載配置
systemctl daemon-reload systemctl enable kubelet
4: 命令補全
yum install -y bash-completion source /usr/share/bash-completion/bash_completion source <(kubectl completion bash) echo "source <(kubectl completion bash)" >> ~/.bashrc
7：初始化集羣
1：node0一、node02添加集羣初始配置文件（集羣配置文件同樣）
cat <<EOF > config.yaml apiVersion: kubeadm.k8s.io/v1alpha1 kind: MasterConfiguration etcd: endpoints: - https://10.150.27.51:2379 - https://10.150.27.65:2379 caFile: /etc/etcd/ssl/ca.pem certFile: /etc/etcd/ssl/etcd.pem keyFile: /etc/etcd/ssl/etcd-key.pem dataDir: /var/lib/etcd networking: podSubnet: 10.244.0.0/16 kubernetesVersion: 1.10.0 api: advertiseAddress: "10.150.27.99" token: "b99a00.a144ef80536d4344" tokenTTL: "0s" ## token 永不失效 apiServerCertSANs: - node01 - node02 - node03 - 10.150.27.51 - 10.150.27.65 - 10.150.27.66 - 10.150.27.99 featureGates: CoreDNS: true imageRepository: "registry.cn-hangzhou.aliyuncs.com/k8sth" EOF
2：首先node01初始化集羣
配置文件定義podnetwork是172.16.30.0/16
kubeadm init --hlep能夠看出，service默認網段是10.96.0.0/12
/etc/systemd/system/kubelet.service.d/10-kubeadm.conf默認dns地址cluster-dns=10.96.0.10
kubeadm init --config config.yaml
==初始化失敗後處理辦法==
kubeadm reset
#或
rm -rf /etc/kubernetes/*.conf
rm -rf /etc/kubernetes/manifests/*.yaml
docker ps -a |awk '{print $1}' |xargs docker rm -f
systemctl stop kubelet
==初始化正常的結果以下==
Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join 10.150.27.99:6443 --token b99a00.a144ef80536d4344 --discovery-token-ca-cert-hash sha256:f79b68fb698c92b9336474eb3bf184e847f967dc58a6296911892662b98b1315
3：node01上面執行以下命令
mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
4：kubeadm生成證書密碼文件分發到node02上面去
scp -r /etc/kubernetes/pki node02:/etc/kubernetes/
5：部署flannel網絡，只須要在node01執行就行
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml #版本信息：quay.io/coreos/flannel:v0.10.0-amd64 kubectl create -f kube-flannel.yml
執行命令
[root@node01 ~]# kubectl get node NAME STATUS ROLES AGE VERSION node01 Ready master 31m v1.10.0
[root@node01 ~]# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-7997f8864c-4x7mg 1/1 Running 0 29m kube-system coredns-7997f8864c-zfcck 1/1 Running 0 29m kube-system kube-apiserver-node01 1/1 Running 0 29m kube-system kube-controller-manager-node01 1/1 Running 0 30m kube-system kube-flannel-ds-hw2xb 1/1 Running 0 1m kube-system kube-proxy-s265b 1/1 Running 0 29m kube-system kube-scheduler-node01 1/1 Running 0 30m
6：部署dashboard
kubectl create -f kubernetes-dashboard.yaml
獲取token,經過令×××登錄
kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
經過firefox訪問dashboard，輸入token,便可登錄
https://10.150.27.99:30000/
kubernetes-dashboard.yaml文件內容以下
# Copyright 2017 The Kubernetes Authors.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# Configuration to deploy release version of the Dashboard UI compatible with
# Kubernetes 1.8.
#
# Example usage: kubectl create -f <this_file>
# ------------------- Dashboard Secret ------------------- #
apiVersion: v1
kind: Secret
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard-certs
namespace: kube-system
type: Opaque
---
# ------------------- Dashboard Service Account ------------------- #
apiVersion: v1
kind: ServiceAccount
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
---
# ------------------- Dashboard Role & Role Binding ------------------- #
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
rules:
# Allow Dashboard to create 'kubernetes-dashboard-key-holder' secret.
- apiGroups: [""]
resources: ["secrets"]
verbs: ["create"]
# Allow Dashboard to create 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
verbs: ["create"]
# Allow Dashboard to get, update and delete Dashboard exclusive secrets.
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["kubernetes-dashboard-key-holder", "kubernetes-dashboard-certs"]
verbs: ["get", "update", "delete"]
# Allow Dashboard to get and update 'kubernetes-dashboard-settings' config map.
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["kubernetes-dashboard-settings"]
verbs: ["get", "update"]
# Allow Dashboard to get metrics from heapster.
- apiGroups: [""]
resources: ["services"]
resourceNames: ["heapster"]
verbs: ["proxy"]
- apiGroups: [""]
resources: ["services/proxy"]
resourceNames: ["heapster", "http:heapster:", "https:heapster:"]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: kubernetes-dashboard-minimal
namespace: kube-system
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: kubernetes-dashboard-minimal
subjects:
- kind: ServiceAccount
name: kubernetes-dashboard
namespace: kube-system
---
# ------------------- Dashboard Deployment ------------------- #
kind: Deployment
apiVersion: apps/v1beta2
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
k8s-app: kubernetes-dashboard
template:
metadata:
labels:
k8s-app: kubernetes-dashboard
spec:
nodeSelector:
node-role.kubernetes.io/master: ""
containers:
- name: kubernetes-dashboard
image: registry.cn-hangzhou.aliyuncs.com/k8sth/kubernetes-dashboard-amd64:v1.8.3
ports:
- containerPort: 8443
protocol: TCP
args:
- --auto-generate-certificates
# Uncomment the following line to manually specify Kubernetes API server Host
# If not specified, Dashboard will attempt to auto discover the API server and connect
# to it. Uncomment only if the default does not work.
# - --apiserver-host=http://my-address:port
volumeMounts:
- name: kubernetes-dashboard-certs
mountPath: /certs
# Create on-disk volume to store exec logs
- mountPath: /tmp
name: tmp-volume
livenessProbe:
httpGet:
scheme: HTTPS
path: /
port: 8443
initialDelaySeconds: 30
timeoutSeconds: 30
volumes:
- name: kubernetes-dashboard-certs
secret:
secretName: kubernetes-dashboard-certs
- name: tmp-volume
emptyDir: {}
serviceAccountName: kubernetes-dashboard
# Comment the following tolerations if Dashboard must not be deployed on master
tolerations:
- key: node-role.kubernetes.io/master
effect: NoSchedule
---
# ------------------- Dashboard Service ------------------- #
kind: Service
apiVersion: v1
metadata:
labels:
k8s-app: kubernetes-dashboard
name: kubernetes-dashboard
namespace: kube-system
spec:
type: NodePort
ports:
- port: 443
targetPort: 8443
nodePort: 30000
selector:
k8s-app: kubernetes-dashboard
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: admin-user
namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
name: admin-user
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: cluster-admin
subjects:
- kind: ServiceAccount
name: admin-user
namespace: kube-system
7：安裝heapster
[root@node01 ~]# kubectl create -f kube-heapster/influxdb/ deployment.extensions "monitoring-grafana" created service "monitoring-grafana" created serviceaccount "heapster" created deployment.extensions "heapster" created service "heapster" created deployment.extensions "monitoring-influxdb" created service "monitoring-influxdb" created [root@node01 ~]# kubectl create -f kube-heapster/rbac/ clusterrolebinding.rbac.authorization.k8s.io "heapster" created [root@node01 ~]# kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system coredns-7997f8864c-4x7mg 1/1 Running 0 1h kube-system coredns-7997f8864c-zfcck 1/1 Running 0 1h kube-system heapster-647b89cd4b-wmvmw 1/1 Running 0 39s kube-system kube-apiserver-node01 1/1 Running 0 1h kube-system kube-controller-manager-node01 1/1 Running 0 1h kube-system kube-flannel-ds-hw2xb 1/1 Running 0 49m kube-system kube-proxy-s265b 1/1 Running 0 1h kube-system kube-scheduler-node01 1/1 Running 0 1h kube-system kubernetes-dashboard-7b44ff9b77-26fkj 1/1 Running 0 44m kube-system monitoring-grafana-74bdd98b7d-szvqg 1/1 Running 0 40s kube-system monitoring-influxdb-55bbd4b96-95tw7 1/1 Running 0 40s
訪問https://10.150.27.99:30000/#!/login便可看到監控信息
heapster文件信息
[root@node01 ~]# tree kube-heapster/
kube-heapster/
├── influxdb
│ ├── grafana.yaml
│ ├── heapster.yaml
│ └── influxdb.yaml
└── rbac
└── heapster-rbac.yaml
grafana.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: monitoring-grafana
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
task: monitoring
k8s-app: grafana
spec:
nodeSelector:
node-role.kubernetes.io/master: ""
containers:
- name: grafana
image: registry.cn-hangzhou.aliyuncs.com/k8sth/heapster-grafana-amd64:v4.4.3
imagePullPolicy: IfNotPresent
ports:
- containerPort: 3000
protocol: TCP
volumeMounts:
- mountPath: /etc/ssl/certs
name: ca-certificates
readOnly: true
- mountPath: /var
name: grafana-storage
env:
- name: INFLUXDB_HOST
value: monitoring-influxdb
- name: GF_SERVER_HTTP_PORT
value: "3000"
# The following env variables are required to make Grafana accessible via
# the kubernetes api-server proxy. On production clusters, we recommend
# removing these env variables, setup auth for grafana, and expose the grafana
# service using a LoadBalancer or a public IP.
- name: GF_AUTH_BASIC_ENABLED
value: "false"
- name: GF_AUTH_ANONYMOUS_ENABLED
value: "true"
- name: GF_AUTH_ANONYMOUS_ORG_ROLE
value: Admin
- name: GF_SERVER_ROOT_URL
# If you're only using the API Server proxy, set this value instead:
# value: /api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
value: /
volumes:
- name: ca-certificates
hostPath:
path: /etc/ssl/certs
- name: grafana-storage
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
labels:
# For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
# If you are NOT using this as an addon, you should comment out this line.
kubernetes.io/cluster-service: 'true'
kubernetes.io/name: monitoring-grafana
name: monitoring-grafana
namespace: kube-system
spec:
# In a production setup, we recommend accessing Grafana through an external Loadbalancer
# or through a public IP.
# type: LoadBalancer
# You could also use NodePort to expose the service at a randomly-generated port
# type: NodePort
ports:
- port: 80
targetPort: 3000
selector:
k8s-app: grafana
heapster.yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: heapster
namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: heapster
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
task: monitoring
k8s-app: heapster
spec:
serviceAccountName: heapster
nodeSelector:
node-role.kubernetes.io/master: ""
containers:
- name: heapster
image: registry.cn-hangzhou.aliyuncs.com/k8sth/heapster-amd64:v1.4.2
imagePullPolicy: IfNotPresent
command:
- /heapster
- --source=kubernetes:https://kubernetes.default
- --sink=influxdb:http://monitoring-influxdb.kube-system.svc:8086
---
apiVersion: v1
kind: Service
metadata:
labels:
task: monitoring
# For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
# If you are NOT using this as an addon, you should comment out this line.
kubernetes.io/cluster-service: 'true'
kubernetes.io/name: Heapster
name: heapster
namespace: kube-system
spec:
ports:
- port: 80
targetPort: 8082
selector:
k8s-app: heapster
influxdb.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
name: monitoring-influxdb
namespace: kube-system
spec:
replicas: 1
template:
metadata:
labels:
task: monitoring
k8s-app: influxdb
spec:
nodeSelector:
node-role.kubernetes.io/master: ""
containers:
- name: influxdb
image: registry.cn-hangzhou.aliyuncs.com/k8sth/heapster-influxdb-amd64:v1.3.3
imagePullPolicy: IfNotPresent
volumeMounts:
- mountPath: /data
name: influxdb-storage
volumes:
- name: influxdb-storage
emptyDir: {}
---
apiVersion: v1
kind: Service
metadata:
labels:
task: monitoring
# For use as a Cluster add-on (https://github.com/kubernetes/kubernetes/tree/master/cluster/addons)
# If you are NOT using this as an addon, you should comment out this line.
kubernetes.io/cluster-service: 'true'
kubernetes.io/name: monitoring-influxdb
name: monitoring-influxdb
namespace: kube-system
spec:
ports:
- port: 8086
targetPort: 8086
selector:
k8s-app: influxdb
heapster-rbac.yaml
kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1beta1 metadata: name: heapster roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: system:heapster subjects: - kind: ServiceAccount name: heapster namespace: kube-system
9：在node02上面執行初始化
kubeadm init --config config.yaml #初始化的結果和node01的結果徹底同樣 mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config
10：查看節點信息
[root@node01 ~]# kubectl get nodes NAME STATUS ROLES AGE VERSION node01 Ready master 1h v1.10.0 node02 Ready master 1h v1.10.0
[root@node01 ~]# kubectl get pods --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE kube-system coredns-7997f8864c-cr725 1/1 Running 0 40d 10.244.0.2 node01 kube-system coredns-7997f8864c-qp79g 1/1 Running 0 40d 10.244.0.3 node01 kube-system elasticsearch-logging-1 1/1 Running 0 7d 10.244.0.18 node01 kube-system heapster-647b89cd4b-pmlwh 1/1 Running 0 11d 10.244.0.15 node01 kube-system kube-apiserver-node02 1/1 Running 1 7d 10.150.27.65 node02 kube-system kube-apiserver-node01 1/1 Running 0 40d 10.150.27.51 node01 kube-system kube-controller-manager-node02 1/1 Running 2 7d 10.150.27.65 node02 kube-system kube-controller-manager-node01 1/1 Running 1 40d 10.150.27.51 node01 kube-system kube-flannel-ds-7f67k 1/1 Running 1 40d 10.150.27.65 node02 kube-system kube-flannel-ds-mjl2d 1/1 Running 0 40d 10.150.27.51 node01 kube-system kube-proxy-75t65 1/1 Running 1 40d 10.150.27.65 node02 kube-system kube-proxy-mtnnw 1/1 Running 0 40d 10.150.27.51 node01 kube-system kube-scheduler-node02 1/1 Running 1 7d 10.150.27.65 node02 kube-system kube-scheduler-node01 1/1 Running 1 40d 10.150.27.51 node01 kube-system kubernetes-dashboard-7b44ff9b77-zx448 1/1 Running 0 40d 10.244.0.4 node01 kube-system monitoring-grafana-74bdd98b7d-2grhz 1/1 Running 0 11d 10.244.0.16 node01 kube-system monitoring-influxdb-55bbd4b96-xxfrr 1/1 Running 0 11d 10.244.0.17 node01
11：讓master也運行pod（默認master不運行pod）
kubectl taint nodes --all node-role.kubernetes.io/master-
八：添加node03節點到集羣
在node03節點執行以下命令，便可將節點添加進集羣
kubeadm join 10.150.27.99:6443 --token b99a00.a144ef80536d4344 --discovery-token-ca-cert-hash sha256:f79b68fb698c92b9336474eb3bf184e847f967dc58a6296911892662b98b1315
[root@node01 ~]# kubectl get node NAME STATUS ROLES AGE VERSION node01 Ready master 45m v1.10.0 node02 Ready master 15m v1.10.0 node03 Ready <none> 13m v1.10.0
12: Dashboard樣例
https://10.150.27.99:30000/
見下圖。
至此K8S1.1.10雙Master高可用模式搭建完成，在Node01上關閉網卡模擬故障，VIP會自動漂移到Node02。
相關文章
- 1. k8s環境搭建
- 2. mongoDB高可用集羣環境搭建
- 3. hadoop環境搭建：高可用
- 4. neo4j高可用環境搭建
- 5. Kafka高可用環境搭建
- 6. Elasticsearch Docker高可用環境搭建
- 7. Kafaka高可用集羣環境搭建
- 8. MySQL&Redis+keepalived高可用環境搭建
- 9. MHA+keepalive高可用環境搭建
- 10. Django實戰——環境搭建
- 更多相關文章...
- • Swift 環境搭建 - Swift 教程
- • Rust 環境搭建 - RUST 教程
- • Java Agent入門實戰（一）-Instrumentation介紹與使用
- • Java Agent入門實戰（三）-JVM Attach原理與使用
相關標籤/搜索
每日一句
-
每一个你不满意的现在，都有一个你没有努力的曾经。
最新文章
- 1. 在windows下的虛擬機中,安裝華爲電腦的deepin操作系統
- 2. 強烈推薦款下載不限速解析神器
- 3. 【區塊鏈技術】孫宇晨：區塊鏈技術帶來金融服務的信任變革
- 4. 搜索引起的鏈接分析-計算網頁的重要性
- 5. TiDB x 微衆銀行 | 耗時降低 58%，分佈式架構助力實現普惠金融
- 6. 《數字孿生體技術白皮書》重磅發佈（附完整版下載）
- 7. 雙十一「避坑」指南：區塊鏈電子合同爲電商交易保駕護航！
- 8. 區塊鏈產業，怎樣「鏈」住未來？
- 9. OpenglRipper使用教程
- 10. springcloud請求一次好用一次不好用zuul Name or service not known
歡迎關注本站公眾號,獲取更多信息
相關文章