sample-apiserver目录结构如下,可参考编写自己的aggregated server:
staging/src/k8s.io/sample-apiserver ├── artifacts │ ├── example │ │ ├── apiservice.yaml ... ├── hack ├── main.go └── pkg ├── admission ├── apis ├── apiserver ├── cmd ├── generated │ ├── clientset │ │ └── versioned ... │ │ └── typed │ │ └── wardle │ │ ├── v1alpha1 │ │ └── v1beta1 │ ├── informers │ │ └── externalversions │ │ └── wardle │ │ ├── v1alpha1 │ │ └── v1beta1 │ ├── listers │ │ └── wardle │ │ ├── v1alpha1 │ │ └── v1beta1 └── registry其中,artifacts用于部署yaml示例
hack目录存放自动脚本(eg: update-codegen)
main.go是aggregated server启动入口;pkg/cmd负责启动aggregated server具体逻辑;pkg/apiserver用于aggregated server初始化以及路由注册
pkg/apis负责相关CR的结构体定义,自动生成(update-codegen)
pkg/admission负责准入的相关代码
pkg/generated负责生成访问CR的clientset,informers,以及listers
pkg/registry目录负责CR相关的RESTStorage实现
更多代码原理详情,参考 kubernetes-reading-notes 。
apiExtensionsServerapiExtensionsServer主要负责CustomResourceDefinition(CRD)apiResources以及apiVersions的注册,同时处理CRD以及相应CustomResource(CR)的REST请求(如果对应CR不能被处理的话则会返回404),也是apiserver Delegation的最后一环
原理总结如下:
Custom Resource,简称CR,是Kubernetes自定义资源类型,与之相对应的就是Kubernetes内置的各种资源类型,例如Pod、Service等。利用CR我们可以定义任何想要的资源类型
CRD通过yaml文件的形式向Kubernetes注册CR实现自定义api-resources,属于第二种扩展Kubernetes API资源的方式,也是普遍使用的一种
APIExtensionServer负责CustomResourceDefinition(CRD)apiResources以及apiVersions的注册,同时处理CRD以及相应CustomResource(CR)的REST请求(如果对应CR不能被处理的话则会返回404),也是apiserver Delegation的最后一环
crdRegistrationController负责将CRD GroupVersions自动注册到APIServices中。具体逻辑为:枚举所有CRDs,然后根据CRD定义的crd.Spec.Group以及crd.Spec.Versions字段构建APIService,并添加到autoRegisterController.apiServicesToSync中,由autoRegisterController进行创建以及维护操作。这也是为什么创建完CRD后会产生对应的APIService对象
APIExtensionServer包含的controller以及功能如下所示:
openapiController:将 crd 资源的变化同步至提供的 OpenAPI 文档,可通过访问 /openapi/v2 进行查看;
crdController:负责将 crd 信息注册到 apiVersions 和 apiResources 中,两者的信息可通过 kubectl api-versions 和 kubectl api-resources 查看;
kubectl api-versions命令返回所有Kubernetes集群资源的版本信息(实际发出了两个请求,分别是https://127.0.0.1:6443/api以及https://127.0.0.1:6443/apis,并在最后将两个请求的返回结果进行了合并)
$ kubectl -v=8 api-versions I1211 11:44:50.276446 22493 loader.go:375] Config loaded from file: /root/.kube/config I1211 11:44:50.277005 22493 round_trippers.go:420] GET https://127.0.0.1:6443/api?timeout=32s ... I1211 11:44:50.290265 22493 request.go:1068] Response Body: {"kind":"APIVersions","versions":["v1"],"serverAddressByClientCIDRs":[{"clientCIDR":"0.0.0.0/0","serverAddress":"x.x.x.x:6443"}]} I1211 11:44:50.293673 22493 round_trippers.go:420] GET https://127.0.0.1:6443/apis?timeout=32s ... I1211 11:44:50.298360 22493 request.go:1068] Response Body: {"kind":"APIGroupList","apiVersion":"v1","groups":[{"name":"apiregistration.k8s.io","versions":[{"groupVersion":"apiregistration.k8s.io/v1","version":"v1"},{"groupVersion":"apiregistration.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"apiregistration.k8s.io/v1","version":"v1"}},{"name":"extensions","versions":[{"groupVersion":"extensions/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"extensions/v1beta1","version":"v1beta1"}},{"name":"apps","versions":[{"groupVersion":"apps/v1","version":"v1"}],"preferredVersion":{"groupVersion":"apps/v1","version":"v1"}},{"name":"events.k8s.io","versions":[{"groupVersion":"events.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"events.k8s.io/v1beta1","version":"v1beta1"}},{"name":"authentication.k8s.io","versions":[{"groupVersion":"authentication.k8s.io/v1","version":"v1"},{"groupVersion":"authentication.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"authentication.k8s.io/v1"," [truncated 4985 chars] apiextensions.k8s.io/v1 apiextensions.k8s.io/v1beta1 apiregistration.k8s.io/v1 apiregistration.k8s.io/v1beta1 apps/v1 authentication.k8s.io/v1beta1 ... storage.k8s.io/v1 storage.k8s.io/v1beta1 v1