conditions信息中 ContainerReady为True, 但是Ready却为False, message中提示"对应的readiness gate condition还不存在", 那我们只需要patch上对应的condition即可， 如下所示:
status: conditions: - lastProbeTime: null lastTransitionTime: "2020-03-14T11:38:03Z" message: LB synced successfully reason: LBHealthy status: "True" type: cloudnativestation.net/load-balancer-ready # <--- 增加readiness gate condtion - lastProbeTime: null lastTransitionTime: "2020-03-14T11:38:03Z" status: "True" type: Initialized - lastProbeTime: null lastTransitionTime: "2020-03-14T11:38:05Z" status: "True" type: Ready # <--- pod状态变为ready - lastProbeTime: null lastTransitionTime: "2020-03-14T11:38:05Z" status: "True" type: ContainersReady - lastProbeTime: null lastTransitionTime: "2020-03-14T11:38:03Z" status: "True" type: PodScheduled containerStatuses: - containerID: docker://65e894a7ef4e53c982bd02da9aee2ddae7c30e652c5bba0f36141876f4c30a01 image: nginx:latest imageID: docker-pullable://nginx@sha256:2e6775f4300fc79b9d7fe6bb60c83b5fefe584258d9318ed4087467手动设置完readiness gate的condtion之后整个pod才能变为ready。
容器退出过程对于容器退出的过程中， 我们需要及时将流量从LB上面摘除。 一个pod典型的退出流程为: 我们从控制台下达删除pod的命令时，apiserver会记录pod deletionTimestamp 标记在pod的manifest中， 随后开始执行删除逻辑，首先发送SIGTERM 信号， 然后最大等待terminationGracePeriodSeconds发送SIGKILL信号强制清理， terminationGracePeriodSeconds该值用户可以自行在pod的manifest中指定。
结合整个退出过程，我们需要在监听到容器退出开始时(也就是deletionTimestamp被标记时) 在LB上将该pod流量权重置为0, 这样新建连接就不到达该容器，同时已有连接不受影响，可以继续提供服务。等到容器真正退出时才将该pod从LB上面摘除。用户如果想要更加安全的流量退出逻辑，可以设置一个稍长一点的terminationGracePeriodSeconds， 甚至设置prestop逻辑或者处理SIGTERM信号， 让pod在退出前等待足够长的时间将流量彻底断掉，
明确了整个架构中的关键点后，就是具体的实现环节了。 这部分我们可以借鉴社区提供的service controller及各个云厂商LB在kubernetes中的应用。 社区为了屏蔽掉不同云厂商产品的差异，开发了cloud-controller-manager, 其内部定义了很多接口， 各个云厂商只需要实现其中的接口就可以在合适的时候被调用。 对于LoadBalancer定义接口如下：
// LoadBalancer is an abstract, pluggable interface for load balancers. type LoadBalancer interface { // TODO: Break this up into different interfaces (LB, etc) when we have more than one type of service // GetLoadBalancer returns whether the specified load balancer exists, and // if so, what its status is. // Implementations must treat the *v1.Service parameter as read-only and not modify it. // Parameter 'clusterName' is the name of the cluster as presented to kube-controller-manager GetLoadBalancer(ctx context.Context, clusterName string, service *v1.Service) (status *v1.LoadBalancerStatus, exists bool, err error) // GetLoadBalancerName returns the name of the load balancer. Implementations must treat the // *v1.Service parameter as read-only and not modify it. GetLoadBalancerName(ctx context.Context, clusterName string, service *v1.Service) string // EnsureLoadBalancer creates a new load balancer 'name', or updates the existing one. Returns the status of the balancer // Implementations must treat the *v1.Service and *v1.Node // parameters as read-only and not modify them. // Parameter 'clusterName' is the name of the cluster as presented to kube-controller-manager EnsureLoadBalancer(ctx context.Context, clusterName string, service *v1.Service, nodes []*v1.Node) (*v1.LoadBalancerStatus, error) // UpdateLoadBalancer updates hosts under the specified load balancer. // Implementations must treat the *v1.Service and *v1.Node // parameters as read-only and not modify them. // Parameter 'clusterName' is the name of the cluster as presented to kube-controller-manager UpdateLoadBalancer(ctx context.Context, clusterName string, service *v1.Service, nodes []*v1.Node) error // EnsureLoadBalancerDeleted deletes the specified load balancer if it // exists, returning nil if the load balancer specified either didn't exist or // was successfully deleted. // This construction is useful because many cloud providers' load balancers // have multiple underlying components, meaning a Get could say that the LB // doesn't exist even if some part of it is still laying around. // Implementations must treat the *v1.Service parameter as read-only and not modify it. // Parameter 'clusterName' is the name of the cluster as presented to kube-controller-manager EnsureLoadBalancerDeleted(ctx context.Context, clusterName string, service *v1.Service) error }