kubernetes实践(二)MAC OS系统下k8s集群的搭建

By prince No comments

1.版本统一

 

centos7

Docker 18.09.0

kubeadm-1.14.0-0 
kubelet-1.14.0-0 
kubectl-1.14.0-0

k8s.gcr.io/kube-apiserver:v1.14.0
k8s.gcr.io/kube-controller-manager:v1.14.0
k8s.gcr.io/kube-scheduler:v1.14.0
k8s.gcr.io/kube-proxy:v1.14.0
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.3.10
k8s.gcr.io/coredns:1.3.1

calico:v3.9

2.准备3台centos7

1.vargrant准备centos机器

mkdir k8s-docker-centos7 
cd  k8s-docker-centos7

vi Vagrantfile


#======================================================================
boxes = [
        {
                :name => "m",
                :eth1 => "192.168.50.14",
                :mem => "4096",
                :cpu => "2"
        },
        {
                :name => "w1",
                :eth1 => "192.168.50.15",
                :mem => "4096",
                :cpu => "2"
        },
        {
                :name => "w2",
                :eth1 => "192.168.50.16",
                :mem => "4096",
                :cpu => "2"
        }
]
Vagrant.configure(2) do |config|
        config.vm.box = "centos/7"
        boxes.each do |opts|
                config.vm.define opts[:name] do |config|
                        config.vm.hostname = opts[:name]
                        config.vm.network :public_network, bridge: "en0: Wi-Fi (Wireless)", ip: opts[:eth1]
                        config.vm.provider "vmware_fusion" do |v|
                                v.vmx["memsize"] = opts[:mem]
                                v.vmx["numvcpus"] = opts[:cpu]
                        end
                        config.vm.provider "virtualbox" do |v|
                                v.customize ["modifyvm", :id, "--memory", opts[:mem]]
                        v.customize ["modifyvm", :id, "--cpus", opts[:cpu]]
                                v.customize ["modifyvm", :id, "--name", opts[:name]]
                        end
                end
        end
end
#======================================================================

2.配置机器登陆权限

(1).启动之后进入到对应的centos里面,使得root账户能够登陆

cd  k8s-docker-centos7
vagrant up [启动虚拟机]
vagrant ssh master-kubeadm-k8s [进入manager-node]
vagrant ssh worker01-kubeadm-k8s [进入worker01-node]
vagrant ssh worker02-kubeadm-k8s [进入worker02-node]

(2).分别登陆三个节点,执行下面操作,改成可以密码登陆root账户

sudo -i [进入root账户]
vi /etc/ssh/sshd_config [设置root账户可以密码登陆]
修改PasswordAuthentication yes
passwd [修改密码]
systemctl restart sshd [重启sshd服务]

3.更新并安装依赖

3台机器都执行

yum -y update
yum install -y conntrack ipvsadm ipset jq sysstat curl iptables libseccomp

4.安装docker

3台机器都需要执行,安装版本为18.09.0

(1).安装依赖

sudo yum install -y yum-utils device-mapper-persistent-data lvm2

(2).添加源信息

sudo yum-config-manager \
--add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

sudo yum makecache fast

(3).刷新yum源缓存

yum clean all 
yum makecache 
yum repolist

(4).设置加速镜像

sudo mkdir -p /etc/docker

vi /etc/docker/daemon.json

{
"registry-mirrors": ["这边替换成自己的实际地址"]
}

(5).安装docker

sudo yum install -y docker-ce-18.09.0 docker-ce-cli-18.09.0 containerd.io

(6).启动docker

sudo systemctl start docker && sudo systemctl enable docker

5.修改host文件

(1)master

# 设置master的hostname,并且修改hosts文件
sudo hostnamectl set-hostname m

vi /etc/hosts
192.168.50.14 m
192.168.50.15 w1
192.168.50.16 w2

(2)两个worker

# 设置worker01/02的hostname,并且修改hosts文件
sudo hostnamectl set-hostname w1
sudo hostnamectl set-hostname w2

vi /etc/hosts
192.168.50.14 m
192.168.50.15 w1
192.168.50.16 w2

(3)使用ping测试一下

ping m
ping w1
ping w2

6.centos7系统设置

(1)关闭防火墙

systemctl stop firewalld && systemctl disable firewalld

(2)关闭selinux

setenforce 0
sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config

(3)关闭swap

swapoff -a
sed -i '/swap/s/^\(.*\)$/#\1/g' /etc/fstab

(4)配置iptables的ACCEPT规则

强制在机器启动时候在forward 链中增加一条全转发规则:

a.写一个脚本

#!/bin/bash
sleep 10
iptables -I FORWARD  -s 0.0.0.0/0 -d 0.0.0.0/0  -j ACCEPT

为了简单可以直接用

iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && iptables -P FORWARD ACCEPT

b.在systemd中增加一个service并开机启动

[Unit]
Description=enable forward all for forward chain in filter table
After=network.target
[Service]
ExecStart=/bin/bash /etc/sysconfig/add-forward-iptable-rule.sh
[Install]
WantedBy=multi-user.target
systemctl enable add-iptable-rule.service

(5)设置系统参数

cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF

sysctl --system

7.安装kubeadm, kubelet和 kubectl

(1).配置yum源

cat <<EOF > /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

(2).刷新yum源缓存

yum clean all 
yum makecache 
yum repolist

(3).安装kubeadm,kubelet,kubectl

yum install -y kubeadm-1.14.0-0 kubelet-1.14.0-0 kubectl-1.14.0-0

(4)docker和k8s设置同一个cgroup

# docker
vi /etc/docker/daemon.json
"exec-opts": ["native.cgroupdriver=systemd"],

systemctl restart docker
# kubelet
sed -i "s/cgroup-driver=systemd/cgroup-driver=cgroupfs/g" /etc/systemd/system/kubelet.service.d/10-kubeadm.conf

systemctl enable kubelet && systemctl start kubelet

8.下载k8s镜像

(1).查询需要的镜像

kubeadm config images list

k8s.gcr.io/kube-apiserver:v1.14.0
k8s.gcr.io/kube-controller-manager:v1.14.0
k8s.gcr.io/kube-scheduler:v1.14.0
k8s.gcr.io/kube-proxy:v1.14.0
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.3.10
k8s.gcr.io/coredns:1.3.1

(2)解决国外镜像不能访问的问题

a.创建kubeadm.sh脚本,用于拉取镜像/打tag/删除原有镜像

#!/bin/bash

set -e

KUBE_VERSION=v1.14.0
KUBE_PAUSE_VERSION=3.1
ETCD_VERSION=3.3.10
CORE_DNS_VERSION=1.3.1

GCR_URL=k8s.gcr.io
ALIYUN_URL=registry.cn-hangzhou.aliyuncs.com/google_containers

images=(kube-proxy:${KUBE_VERSION}
kube-scheduler:${KUBE_VERSION}
kube-controller-manager:${KUBE_VERSION}
kube-apiserver:${KUBE_VERSION}
pause:${KUBE_PAUSE_VERSION}
etcd:${ETCD_VERSION}
coredns:${CORE_DNS_VERSION})

for imageName in ${images[@]} ; do
docker pull $ALIYUN_URL/$imageName
docker tag $ALIYUN_URL/$imageName $GCR_URL/$imageName
docker rmi $ALIYUN_URL/$imageName
done

b.运行脚本和查看镜像

# 运行脚本
sh ./kubeadm.sh

# 查看镜像
docker images

9.kube init 初始化

(1).初始化master节点

kubeadm init --kubernetes-version=1.14.0 --apiserver-advertise-address=192.168.50.14 --pod-network-cidr=10.244.0.0/16

【若要重新初始化集群状态:kubeadm reset,然后再进行上述操作】

执行结果

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube

  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.

Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.50.14:6443 --token rzrqpo.1hdg1yc2eikpunx8 \

    --discovery-token-ca-cert-hash sha256:32fb668badcee3bef094b7d69d7beb97ad2e26a7106236f078c3be369e89ecbc

初始化master执行流程

01-进行一系列检查,以确定这台机器可以部署kubernetes

02-生成kubernetes对外提供服务所需要的各种证书可对应目录
/etc/kubernetes/pki/*

03-为其他组件生成访问kube-ApiServer所需的配置文件
ls /etc/kubernetes/
admin.conf controller-manager.conf kubelet.conf scheduler.conf

04-为 Master组件生成Pod配置文件。
ls /etc/kubernetes/manifests/*.yaml
kube-apiserver.yaml 
kube-controller-manager.yaml
kube-scheduler.yaml

05-生成etcd的Pod YAML文件。
ls /etc/kubernetes/manifests/*.yaml
kube-apiserver.yaml 
kube-controller-manager.yaml
kube-scheduler.yaml
etcd.yaml

06-一旦这些 YAML 文件出现在被 kubelet 监视的/etc/kubernetes/manifests/目录下,kubelet就会自动创建这些yaml文件定义的pod,即master组件的容器。master容器启动后,kubeadm会通过检查localhost:6443/healthz这个master组件的健康状态检查URL,等待master组件完全运行起来

07-为集群生成一个bootstrap token

08-将ca.crt等 Master节点的重要信息,通过ConfigMap的方式保存在etcd中,工后续部署node节点使用

09-最后一步是安装默认插件,kubernetes默认kube-proxy和DNS两个插件是必须安装的

 

(2).根据日志执行

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

(3).验证是否成功

kubectl get pods -n kube-system 【查看kube-system的pods】
kubectl get pods --all-namespaces 【查看所有pods】

注意:coredns没有启动,需要安装网络插件

(4).健康检查

curl -k https://localhost:6443/healthz

10.部署calico网络插件

(1).查询calico插件镜像

curl https://docs.projectcalico.org/v3.9/manifests/calico.yaml | grep image

结果

image: calico/cni:v3.9.5
image: calico/pod2daemon-flexvol:v3.9.5
image: calico/node:v3.9.5
image: calico/kube-controllers:v3.9.5

(2).手动拉取calico插件镜像

docker pull calico/cni:v3.9.5
docker pull calico/pod2daemon-flexvol:v3.9.5
docker pull calico/node:v3.9.5
docker pull calico/kube-controllers:v3.9.5

(3).安装calico插件

calico如果有节点是多网卡,所以需要在资源清单文件中指定内网网卡

# DaemonSet中添加该环境变量
...
spec:
  containers:
  - env:
    - name: DATASTORE_TYPE
      value: kubernetes
    # 指定内网网卡
    - name: IP_AUTODETECTION_METHOD  
      value: interface=eth0    
    - name: WAIT_FOR_DATASTORE
      value: "true"

安装插件

yum install -y wget
wget https://docs.projectcalico.org/v3.9/manifests/calico.yaml
kubectl apply -f calico.yaml

(4).确认calico插件是否安装成功

kubectl get pods --all-namespaces -w

11.work节点加入

(1)在woker01和worker02上执行上述命令

kubeadm join 192.168.50.14:6443 --token rzrqpo.1hdg1yc2eikpunx8 \

    --discovery-token-ca-cert-hash sha256:32fb668badcee3bef094b7d69d7beb97ad2e26a7106236f078c3be369e89ecbc

(2)在master节点上检查集群信息

kubectl get nodes

NAME STATUS ROLES AGE VERSION
master-kubeadm-k8s Ready master 19m v1.14.0
worker01-kubeadm-k8s Ready <none> 3m6s v1.14.0
worker02-kubeadm-k8s Ready <none> 2m41s v1.14.0

发表评论

 

此站点使用Akismet来减少垃圾评论。了解我们如何处理您的评论数据