本篇主要用于记录在实施docker和kubenetes过程中遇到的一个问题和解决办法.
本节部分内容摘自互联网,有些部分为自己在测试环境中遇到到实际问题,后面还会根据实际情况不断分享关于docker/k8s在开发和维护过程中出现的种种问题,以便后来者少走弯路.
kubernets nodeport 无法访问环境:
Os: centos7.1 Kubelet: 1.6.7 Docker: 17.06-ce Calico: 2.3 K8s Cluster: master, node-1, node-2问题:
现有 service A， 为了能使外部访问，故将 service type 设为NodePort。端口为 31246。
A 所对应的 pod 运行在 node-1 上。
经过测试发现，外部访问 master:31246 和 node-2:31246 时均出现失败，只有通过 node-1:31246 才可正常访问。
起因:
为了安全起见， docker 在 1.13 版本之后，将系统iptables 中 FORWARD 链的默认策略设置为 DROP，并为连接到 docker0 网桥的容器添加了放行规则。这里引用 moby issue#14041 中的描述：
When docker starts, it enables net.ipv4.ip_forward without changing the iptables FORWARD chain default policy to DROP. This means that another machine on the same network as the docker host can add a route to their routing table, and directly address any containers running on that docker host. For example, if the docker0 subnet is 172.17.0.0/16 (the default subnet), and the docker host’s IP address is 192.168.0.10, from another host on the network run: $ ip route add 172.17.0.0/16 via 192.168.0.10 $ nmap 172.17.0.0/16 1 2 The above will scan for containers running on the host, and report IP addresses & running services found. To fix this, docker needs to set the FORWARD policy to DROP when it enables the net.ipv4.ip_forward sysctl parameter.kubernetes 使用的 cni 插件会因此受影响（cni并不会在 FORWARD 链中生成相应规则），由此导致除 pod 所在 host 以外节点无法转发报文而访问失败。
解决办法:
如果对安全要求较低，可将 FORWARD 链的默认规则设为 ACCEPT
iptables -P FORWARD ACCEPT google 网络不可达 https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64/repodata/repomd.xml: [Errno 14] curl#7 - "Failed to connect to 2404:6800:4005:809::200e: 网络不可达"经典问题，需要自备梯子哦！
CentOS中设置yum的proxy
没有梯子的换阿里源
cat > /etc/yum.repos.d/kubernetes.repo <<EOF [kubernetes] name=Kubernetes baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled=1 gpgcheck=0 repo_gpgcheck=0 EOF 关闭Swap F1213 10:20:53.304755 2266 server.go:261] failed to run Kubelet: Running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained:执行 swapoff -a
设定master错误 [ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1 [ERROR FileContent--proc-sys-net-ipv4-ip_forward]: /proc/sys/net/ipv4/ip_forward contents are not set to 1 [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`按提示设定为1
echo "1" >/proc/sys/net/ipv4/ip_forward echo "1" >/proc/sys/net/bridge/bridge-nf-call-iptables Kubeadm init 安装镜像卡住 # kubeadm config images list --kubernetes-version v1.13.0 # 看下该版本下的镜像名 # 拉取镜像 docker pull mirrorgooglecontainers/kube-apiserver:v1.13.0 docker pull mirrorgooglecontainers/kube-controller-manager:v1.13.0 docker pull mirrorgooglecontainers/kube-scheduler:v1.13.0 docker pull mirrorgooglecontainers/kube-proxy:v1.13.0 docker pull mirrorgooglecontainers/pause:3.1 docker pull mirrorgooglecontainers/etcd:3.2.24 docker pull coredns/coredns:1.2.6 # 重命名镜像标签 docker tag docker.io/mirrorgooglecontainers/kube-proxy:v1.13.0 k8s.gcr.io/kube-proxy:v1.13.0 docker tag docker.io/mirrorgooglecontainers/kube-scheduler:v1.13.0 k8s.gcr.io/kube-scheduler:v1.13.0 docker tag docker.io/mirrorgooglecontainers/kube-apiserver:v1.13.0 k8s.gcr.io/kube-apiserver:v1.13.0 docker tag docker.io/mirrorgooglecontainers/kube-controller-manager:v1.13.0 k8s.gcr.io/kube-controller-manager:v1.13.0 docker tag docker.io/mirrorgooglecontainers/etcd:3.2.24 k8s.gcr.io/etcd:3.2.24 docker tag docker.io/mirrorgooglecontainers/pause:3.1 k8s.gcr.io/pause:3.1 docker tag docker.io/coredns/coredns:1.2.6 k8s.gcr.io/coredns:1.2.6 # 删除旧镜像 docker rmi docker.io/mirrorgooglecontainers/kube-proxy:v1.13.0 docker rmi docker.io/mirrorgooglecontainers/kube-scheduler:v1.13.0 docker rmi docker.io/mirrorgooglecontainers/kube-apiserver:v1.13.0 docker rmi docker.io/mirrorgooglecontainers/kube-controller-manager:v1.13.0 docker rmi docker.io/mirrorgooglecontainers/etcd:3.2.24 docker rmi docker.io/mirrorgooglecontainers/pause:3.1 docker rmi docker.io/coredns/coredns:1.2.6终于看到了
Your Kubernetes master has initialized successfully! To start using your cluster, you need to run the following as a regular user: mkdir -p $HOME/.kube sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config sudo chown $(id -u):$(id -g) $HOME/.kube/config You should now deploy a pod network to the cluster. Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at: https://kubernetes.io/docs/concepts/cluster-administration/addons/ You can now join any number of machines by running the following on each node as root: kubeadm join 192.168.232.204:6443 --token m2hxkd.scxjrxgew6pyhvmb --discovery-token-ca-cert-hash sha256:8b94cefbe54ae4b3d7201012db30966c53870aad55be80a2888ec0da178c3610 network配置 # 这是我的虚机/etc/hosts的配置 192.168.232.204 k8a204 192.168.232.203 k8a203 192.168.232.202 k8a202