本文预设了几个前提,会关注到kubectl在干什么的同学,基本上已经对k8s有了一定的了解,至少也是有使用kubectl探索过k8s的一些基础的功能。
上面是一张k8s核心组件的架构图,由图可知 kubectl只与apiserver打交道。探索kubectl在干什么约等于探索apiserver提供了什么。
那么我们先来看一眼apiserver大致提供了什么。
kubectl proxy Starting to serve on 127.0.0.1:8001 复制代码
上面这一条命令是在kubectl可执行的node节点,开启一个 apiserver的proxy,可以免鉴权直接通过curl方式访问apiserver。
curl http://127.0.0.1:8001/ { "paths": [ "/api", "/api/v1", "/apis", "...", "/apis/apiextensions.k8s.io/v1", "/healthz", "/...", "/healthz/poststarthook/apiservice-openapi-controller", "...", "/healthz/poststarthook/start-kube-apiserver-admission-initializer", "/livez", "/...", "/livez/poststarthook/start-kube-apiserver-admission-initializer", "/logs", "/metrics", "/openapi/v2", "/readyz", "....", "/readyz/shutdown", "/version" ] } 复制代码
直接通过 curl 访问 apiserver的根路径,可以看到打印了一个可访问的path路径列表。大致归类可以分为:
- api/v1
- apis/*
- healthz/*
- livez/*
- logs
- metrics
- Openapi/v2
- readyz
- Version
再次归类整理为:
- api resource 相关接口
- api/*
- apis/*
- openapi
- 检查类接口
- 健康检查 /healthz/*
- 存活检查 /livez/*
- 就绪检查 /readyz/*
- 其他
- 日志
- 指标
- 版本
根据上述的接口归类来看,主要提供业务逻辑相处理的接口是 api resource 相关的接口。
根据 kubernetes/staging/src/k8s.io/client-go/discovery/discovery-cient.go:482
// NewDiscoveryClientForConfig creates a new DiscoveryClient for the given config. This client // can be used to discover supported resources in the API server. func NewDiscoveryClientForConfig(c *restclient.Config) (*DiscoveryClient, error) { config := *c if err := setDiscoveryDefaults(&config); err != nil { return nil, err } client, err := restclient.UnversionedRESTClientFor(&config) return &DiscoveryClient{restClient: client, LegacyPrefix: "/api"}, err } 复制代码
及 kubernetes/staging/src/k8s.io/client-go/discovery/discovery-cient.go:192
// ServerResourcesForGroupVersion returns the supported resources for a group and version. func (d *DiscoveryClient) ServerResourcesForGroupVersion(groupVersion string) (resources *metav1.APIResourceList, err error) { url := url.URL{} if len(groupVersion) == 0 { return nil, fmt.Errorf("groupVersion shouldn't be empty") } if len(d.LegacyPrefix) > 0 && groupVersion == "v1" { url.Path = d.LegacyPrefix + "/" + groupVersion } else { url.Path = "/apis/" + groupVersion } resources = &metav1.APIResourceList{ GroupVersion: groupVersion, } err = d.restClient.Get().AbsPath(url.String()).Do(context.TODO()).Into(resources) if err != nil { // ignore 403 or 404 error to be compatible with an v1.0 server. if groupVersion == "v1" && (errors.IsNotFound(err) || errors.IsForbidden(err)) { return resources, nil } return nil, err } return resources, nil } 复制代码
的代码逻辑来看,api/v1这样的 api resource list 属于历史遗留问题。
/api/v1
及 /apis/apiextensions.k8s.io/v1
这样的接口返回的都是 APIResourceList
类型的对象。
namespaced的resource基本上都是有这样的 路径规范组成。而 api/v1
属于缺少了group字段的情形。
以/ api/v1/pod
接口为例,分析针对单个资源的接口组成
{ "name": "pods", "singularName": "", "namespaced": true, "kind": "Pod", "verbs": [ "create", "delete", "deletecollection", "get", "list", "patch", "update", "watch" ], "shortNames": [ "po" ], "categories": [ "all" ], "storageVersionHash": "xPOwRZ+Yhw8=" }, ... "name": "pods/status", "singularName": "", "namespaced": true, "kind": "Pod", "verbs": [ "get", "patch", "update" ] }, 复制代码
一个重要的字段就是 namespaced
,当这个字段为 true
时,说明这个资源对象是受namespace隔离的资源,大多数操作都需要指定namespace才可以执行。
针对某一类资源的动作列表可以在 verbs
字段中获取
- create // 创建
- delete // 删除
- deletecollection // 删除集合(?)
- get // 获取单个资源
- list // 获取资源列表
- patch // 以补丁数据形式修改数据
- update // 以覆盖形式修改数据
- watch // 订阅资源列表数据变化
上述针对pods资源的verbs对应在openapi中的描述则为
以及针对单个资源除了 增删改查以外的其他动作可由 `pods/status这样的接口获取。对应的openapi中的描述则为
另外,通常还会有一个跨ns查询的一个列表接口
kubectl
执行 kubectl options
可以看到有一个参数-v,可以设置日志打印的级别
-v, --v=0: number for the log level verbosity
-v 7
可以看到所有请求apiserver的http相关日志,如
I0221 12:40:50.992381 34765 cached_discovery.go:78] skipped caching discovery info due to the server is currently unable to handle the request I0221 12:40:50.993222 34765 round_trippers.go:420] GET http://localhost:8666/cluster/preee/api/v1/namespaces/default/services?limit=500 I0221 12:40:50.993232 34765 round_trippers.go:427] Request Headers: I0221 12:40:50.993236 34765 round_trippers.go:431] Accept: application/json;as=Table;v=v1;g=meta.k8s.io,application/json;as=Table;v=v1beta1;g=meta.k8s.io,application/json I0221 12:40:50.993240 34765 round_trippers.go:431] User-Agent: kubectl/v1.18.2 (darwin/amd64) kubernetes/52c56ce I0221 12:40:51.048394 34765 round_trippers.go:446] Response Status: 200 OK in 55 milliseconds NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kube-user NodePort 192.168.255.192 <none> 443:31524/TCP 324d kubernetes ClusterIP 192.168.255.1 <none> 443/TCP 338d 复制代码
第一次请求或者删除掉 ~/.kube/cache
目录后执行 kubectl get ns
INFO[0014] GET/api INFO[0015] GET/apis INFO[0015] GET/apis/events.k8s.io/v1beta1 INFO[0015] GET/apis/scheduling.k8s.io/v1 INFO[0015] GET/apis/coordination.k8s.io/v1 ... INFO[0015] GET/api/v1 INFO[0015] GET/apis/extensions/v1beta1 ... INFO[0018] GET/api/v1/namespaces 复制代码
第二次执行 kubectl get ns
INFO[0085] GET/api/v1/namespaces 复制代码
可以得出第一个结论, kubectl 在执行第一次请求时,会缓存所有的api-resource 字段以便后续命令执行时进行相关请求拼装等
kubectl get po
/api/v1/namespaces/default/pods 复制代码
kubectl get po -A
/api/v1/pods 复制代码
/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500
INFO[0007] GET/apis/apps/v1/namespaces/kube-system/deployments?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 复制代码
get all 命令会分别获取
- pods
- replicationcontrollers
- services
- daemonsets
- deployments
- replicasets
- statefulsets
- horizontalpodautoscalers
- jobs
- cronjobs
kubectl get all -A -l kubernetes.io/cluster-service=true
INFO[0064] GET/api/v1/pods?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0064] GET/api/v1/replicationcontrollers?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0064] GET/api/v1/services?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0064] GET/apis/apps/v1/daemonsets?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0064] GET/apis/apps/v1/deployments?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0064] GET/apis/apps/v1/replicasets?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0065] GET/apis/apps/v1/statefulsets?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0065] GET/apis/autoscaling/v1/horizontalpodautoscalers?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0065] GET/apis/batch/v1/jobs?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 INFO[0065] GET/apis/batch/v1beta1/cronjobs?labelSelector=kubernetes.io%2Fcluster-service%3Dtrue&limit=500 复制代码
describe deploy
- deployments
- events
- replicasets
kubectl -nkube-system describe deploy coredns
INFO[0651] GET/apis/apps/v1/namespaces/kube-system/deployments/coredns? INFO[0651] GET/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.name%3Dcoredns%2CinvolvedObject.namespace%3Dkube-system%2CinvolvedObject.kind%3DDeployment%2CinvolvedObject.uid%3D1aa51196-7468-4aba-b26f-5908a50c31a7 INFO[0651] GET/apis/apps/v1/namespaces/kube-system/replicasets?labelSelector=k8s-app%3Dkube-dns 复制代码
describe po
- pods
- Events
kubectl -nkube-system describe po coredns coredns-5488fc95f4-5jr6l
INFO[0840] GET/api/v1/namespaces/kube-system/pods/coredns-5488fc95f4-5jr6l? INFO[0840] GET/api/v1/namespaces/kube-system/pods/coredns-5488fc95f4-5jr6l? INFO[0840] GET/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.name%3Dcoredns-5488fc95f4-5jr6l%2CinvolvedObject.namespace%3Dkube-system%2CinvolvedObject.uid%3D3f0b3909-35ae-4c22-bfdc-5f5ddd84a9b8 复制代码
describe svc
- services
- endpoints
- events
kubectl -nkube-system describe svc prometheus
INFO[0886] GET/api/v1/namespaces/kube-system/services/prometheus? INFO[0886] GET/api/v1/namespaces/kube-system/services/prometheus? INFO[0886] GET/api/v1/namespaces/kube-system/endpoints/prometheus? INFO[0886] GET/api/v1/namespaces/kube-system/events?fieldSelector=involvedObject.name%3Dprometheus%2CinvolvedObject.namespace%3Dkube-system%2CinvolvedObject.kind%3DService%2CinvolvedObject.uid%3D1c27a873-4c53-42d6-ab86-f7050a35fd6c 复制代码
describe node
- nodes
- leases
- pods
- events
kubectl -nkube-system describe no vm-2-177-centos
INFO[1055] GET/api/v1/nodes/vm-2-177-centos? INFO[1056] GET/api/v1/nodes/vm-2-177-centos? INFO[1056] GET/apis/coordination.k8s.io/v1/namespaces/kube-node-lease/leases/vm-2-177-centos? INFO[1056] GET/api/v1/pods?fieldSelector=spec.nodeName%3Dvm-2-177-centos%2Cstatus.phase%21%3DFailed%2Cstatus.phase%21%3DSucceeded INFO[1056] GET/api/v1/events?fieldSelector=involvedObject.kind%3DNode%2CinvolvedObject.uid%3Dvm-2-177-centos%2CinvolvedObject.name%3Dvm-2-177-centos%2CinvolvedObject.namespace%3D 复制代码
kubectl auth can-i get po
INFO[1516] GET/api/v1?timeout=32s INFO[1516] POST/apis/authorization.k8s.io/v1/selfsubjectaccessreviews? 复制代码