调用cri client请求cri server端,在这里server端就是docker shim (docker Service对象)
// Exec prepares a streaming endpoint to execute a command in the container, and returns the address. func (ds *dockerService) Exec(_ context.Context, req *runtimeapi.ExecRequest) (*runtimeapi.ExecResponse, error) { if ds.streamingServer == nil { return nil, streaming.NewErrorStreamingDisabled("exec") } _, err := checkContainerStatus(ds.client, req.ContainerId) if err != nil { return nil, err } return ds.streamingServer.GetExec(req) }调用dockerService.StreamingSerer的GetExec方法,streamingServer的所有方法都定义在:pkg/kubelet/server/streaming/server.go里
func (s *server) GetExec(req *runtimeapi.ExecRequest) (*runtimeapi.ExecResponse, error) { if err := validateExecRequest(req); err != nil { return nil, err } token, err := s.cache.Insert(req) if err != nil { return nil, err } return &runtimeapi.ExecResponse{ Url: s.buildURL("exec", token), }, nil }可以看到这里只是返回一个简单的token组合成的url, 之所以生成一个token是因为用户的命令中可能包含各种各样的字符,各种长度的字符,需要格式化为一个简单的token。 该token会缓存在本地,后面真正的exec请求会携带这该token,通过该token找到之前的具体请求。
处理streaming请求在获取到该exec真正的URL后,就需要通过该URL来获取真正的数据了。为该URL提供服务的sever一般位于CRI的实现之中,例如docker shim会创建一个streamingServer来提供服务。
各个运行时 streaming server 的处理框架都是类似的,kublet为了方便各runtime实现CRI接口,提供了统一的包,位于:pkg/kubelet/server/streaming/server.go。 各种底层runtime只需要实现其中的steaming.Runtime接口就可以简单创建一个streamingServer:
// Runtime is the interface to execute the commands and provide the streams. type Runtime interface { Exec(containerID string, cmd []string, in io.Reader, out, err io.WriteCloser, tty bool, resize <-chan remotecommand.TerminalSize) error Attach(containerID string, in io.Reader, out, err io.WriteCloser, tty bool, resize <-chan remotecommand.TerminalSize) error PortForward(podSandboxID string, port int32, stream io.ReadWriteCloser) error }目前kubelet内置了docker runtime的实现:dockershim,在dockershim中streaming.Runtime的实现结构体为streamingRuntime:
type streamingRuntime struct { client libdocker.Interface execHandler ExecHandler } var _ streaming.Runtime = &streamingRuntime{}其中docker client作为成员函数,以便后面请求docker获取数据。
NOTE
注意区分kubelet中各种runtime的定义,说实话各种runtime确实挺乱的,我们需要明确各种Runtime是定义在哪个scope下的,streaming.Runtime是个interface, 位于pkg/kubelet/server/streaming/server.go,用来定义流处理请求的所需要的动作。streamingRuntime是dockershim对streaming.Runtime interface的具体实现,位于pkg/kubelet/dockershim/docker_streaming.go,该结构体为private的。 另一个比较容易混淆的是pkg/kubelet/container/runtime.go (俗称kubecontainer)中的StreamingRuntime, 该interface为public的,用来定义GetExec/GetAttach/GetPortForward接口
利用该streaming.Runtime就可以创建streamingServer了:
// NewServer creates a new Server for stream requests. // TODO(tallclair): Add auth(n/z) interface & handling. func NewServer(config Config, runtime Runtime) (Server, error) { s := &server{ config: config, runtime: &criAdapter{runtime}, cache: newRequestCache(), } if s.config.BaseURL == nil { s.config.BaseURL = &url.URL{ Scheme: "http", Host: s.config.Addr, } if s.config.TLSConfig != nil { s.config.BaseURL.Scheme = "https" } } ws := &restful.WebService{} endpoints := []struct { path string handler restful.RouteFunction }{ {"/exec/{token}", s.serveExec}, {"/attach/{token}", s.serveAttach}, {"/portforward/{token}", s.servePortForward}, } // If serving relative to a base path, set that here. pathPrefix := path.Dir(s.config.BaseURL.Path) for _, e := range endpoints { for _, method := range []string{"GET", "POST"} { ws.Route(ws. Method(method). Path(path.Join(pathPrefix, e.path)). To(e.handler)) } } handler := restful.NewContainer() handler.Add(ws) s.handler = handler s.server = &http.Server{ Addr: s.config.Addr, Handler: s.handler, TLSConfig: s.config.TLSConfig, } return s, nil }在NewServer中会注册对应的handler来处理/exec/{token}类接口。这里exec的handdler为ServerExec方法:
func (s *server) serveExec(req *restful.Request, resp *restful.Response) { token := req.PathParameter("token") cachedRequest, ok := s.cache.Consume(token) if !ok { http.NotFound(resp.ResponseWriter, req.Request) return } exec, ok := cachedRequest.(*runtimeapi.ExecRequest) if !ok { http.NotFound(resp.ResponseWriter, req.Request) return } streamOpts := &remotecommandserver.Options{ Stdin: exec.Stdin, Stdout: exec.Stdout, Stderr: exec.Stderr, TTY: exec.Tty, } remotecommandserver.ServeExec( resp.ResponseWriter, req.Request, s.runtime, "", // unused: podName "", // unusued: podUID exec.ContainerId, exec.Cmd, streamOpts, s.config.StreamIdleTimeout, s.config.StreamCreationTimeout, s.config.SupportedRemoteCommandProtocols) }