概述
今天在测试通过 kubeadm 搭建 Kubernetes 1.30.4 版本集群的时候，遇到一个问题，发现 controlPlane 是正常初始化的，但是 Node 却一直无法加到集群里，从 kubeadm 的命令返回报错看，一直在报下面的问题。
Retrying due to error: could not find a JWS signature in the cluster-info ConfigMap
操作回顾
参考下面的日志，可以看到执行 kubeadm join 之后，一直报错，无法加入到集群中。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
|
[root@node2 ~]# kubeadm join 192.168.1.201:6443 --token qm4yvj.asffqcbpxvmclzx2 --discovery-token-ca-cert-hash sha256:07c86b3424595f6b2958e3869b7dda9b7ce77a761b0330431492bb0b30a3439c --v 10
I0830 12:50:45.665098 388661 join.go:417] [preflight] found NodeName empty; using OS hostname as NodeName
I0830 12:50:45.666207 388661 initconfiguration.go:122] detected and using CRI socket: unix:///var/run/containerd/containerd.sock
[preflight] Running pre-flight checks
I0830 12:50:45.666336 388661 preflight.go:93] [preflight] Running general checks
I0830 12:50:45.666410 388661 checks.go:278] validating the existence of file /etc/kubernetes/kubelet.conf
I0830 12:50:45.666439 388661 checks.go:278] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I0830 12:50:45.666458 388661 checks.go:102] validating the container runtime
I0830 12:50:45.699860 388661 checks.go:637] validating whether swap is enabled or not
I0830 12:50:45.699992 388661 checks.go:368] validating the presence of executable crictl
I0830 12:50:45.700055 388661 checks.go:368] validating the presence of executable conntrack
I0830 12:50:45.700083 388661 checks.go:368] validating the presence of executable ip
I0830 12:50:45.700327 388661 checks.go:368] validating the presence of executable iptables
I0830 12:50:45.700363 388661 checks.go:368] validating the presence of executable mount
I0830 12:50:45.700393 388661 checks.go:368] validating the presence of executable nsenter
I0830 12:50:45.700424 388661 checks.go:368] validating the presence of executable ebtables
I0830 12:50:45.700458 388661 checks.go:368] validating the presence of executable ethtool
I0830 12:50:45.700497 388661 checks.go:368] validating the presence of executable socat
I0830 12:50:45.700526 388661 checks.go:368] validating the presence of executable tc
I0830 12:50:45.700550 388661 checks.go:368] validating the presence of executable touch
I0830 12:50:45.700579 388661 checks.go:514] running all checks
I0830 12:50:45.713423 388661 checks.go:399] checking whether the given node name is valid and reachable using net.LookupHost
[WARNING Hostname]: hostname "node2" could not be reached
[WARNING Hostname]: hostname "node2": lookup node2 on 192.168.1.1:53: no such host
I0830 12:50:45.718027 388661 checks.go:603] validating kubelet version
I0830 12:50:45.786298 388661 checks.go:128] validating if the "kubelet" service is enabled and active
I0830 12:50:45.806523 388661 checks.go:201] validating availability of port 10250
I0830 12:50:45.807055 388661 checks.go:278] validating the existence of file /etc/kubernetes/pki/ca.crt
I0830 12:50:45.807092 388661 checks.go:428] validating if the connectivity type is via proxy or direct
I0830 12:50:45.807130 388661 checks.go:327] validating the contents of file /proc/sys/net/ipv4/ip_forward
I0830 12:50:45.807183 388661 join.go:536] [preflight] Discovering cluster-info
I0830 12:50:45.807222 388661 token.go:79] [discovery] Created cluster-info discovery client, requesting info from "192.168.1.201:6443"
I0830 12:50:45.807796 388661 token.go:210] [discovery] Waiting for the cluster-info ConfigMap to receive a JWS signaturefor token ID "qm4yvj"
I0830 12:50:45.807961 388661 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.30.4 (linux/amd64) kubernetes/a51b3b7" 'https://192.168.1.201:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s'
I0830 12:50:45.808548 388661 round_trippers.go:510] HTTP Trace: Dial to tcp:192.168.1.201:6443 succeed
I0830 12:50:45.819374 388661 round_trippers.go:553] GET https://192.168.1.201:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 11 milliseconds
I0830 12:50:45.820071 388661 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 7 ms ServerProcessing 3 ms Duration 11 ms
I0830 12:50:45.820114 388661 round_trippers.go:577] Response Headers:
I0830 12:50:45.820132 388661 round_trippers.go:580] Date: Fri, 30 Aug 2024 04:50:45 GMT
I0830 12:50:45.820146 388661 round_trippers.go:580] Audit-Id: f43ff68b-372b-4323-b4c5-3870e9cfa536
I0830 12:50:45.820212 388661 round_trippers.go:580] Cache-Control: no-cache, private
I0830 12:50:45.820223 388661 round_trippers.go:580] Content-Type: application/json
I0830 12:50:45.820239 388661 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 9bf7f64e-a652-4337-a96a-e37d1d5b4606
I0830 12:50:45.820308 388661 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: e9583c3b-b879-4d24-afe2-275e79c0e311
I0830 12:50:45.820339 388661 round_trippers.go:580] Content-Length: 2105
I0830 12:50:45.820513 388661 request.go:1212] Response Body: {"kind":"ConfigMap","apiVersion":"v1","metadata":{"name":"cluster-info","namespace":"kube-public","uid":"3b013c64-6019-4f33-8b28-4ace06e0848f","resourceVersion":"264","creationTimestamp":"2024-08-30T03:45:10Z","managedFields":[{"manager":"kubeadm","operation":"Update","apiVersion":"v1","time":"2024-08-30T03:45:10Z","fieldsType":"FieldsV1","fieldsV1":{"f:data":{".":{},"f:kubeconfig":{}}}}]},"data":{"kubeconfig":"apiVersion: v1\nclusters:\n- cluster:\n certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJQzFONUdEMCtZdjB3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TkRBNE16QXdNek01TlRSYUZ3MHpOREE0TWpnd016UTBOVFJhTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUUN2dWNFYnlNWGpmUlBUZ3pQa0lSRXdyZGpFTXdMT2hmTU1yTUhxa2pMZE1JbjVjaVVuTHhOQW5EZkIKdUV0b3psTmxNRUxyVWdZTlVIa0pqN0RYV2IrUU9DanBpbk1oTm1NR0VvL3RkRVlOaStDeWtoZ2RFV0VhRzZldAplSFV4SWxiOFRGaE5UVDRQUlhwYTRlVzBVOTRUZEdHQjRTMXd0enVQM3RzT0ViQTk1OGYxUGJtZXBicit6dXB6CjR3VGh2OTc5TjRsbnM1VEIzMDFxRDJXNGcxblF6WVB6cjZlYURMZDRqYUpRakJOYUg4Mis1dkNHa2poOU8rbEIKcHJHc1VMUU83dHptRTk3ZHRsTVUyTmRVaS9VdzZXMnU0dENTNjZrMDY5OWZxVGFJbEVTNS8wQllWOS9pSjRHSwpRTSt6Um9MTTRucVNSd2IxS1E4MktBZjM5eFJuQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJSU3doQ0xVNXZDZUdJbStSdnJsWHh1emYvUjlqQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQWVjVGE4TnI3WgpTM2M3Z25wbjNzYWZkUGpwa1RNbUtoTHFDemtnQzdOdmFmY05PNVNHTndDZndhQWdaTitBMTRBZVcwRnBncTR4CjAveHdNcGZNNVRZZGw1cFFVYUdlbHBybUNxY3R3UndLajR6M1VaNDc1bTBRMlA4cW5sNFM0TmNHS3FuVHUweXcKRURDZFB5aUZOdXgzcmhrOGhXclVLQlh0eXM2OHVwSDdmclFYZ3VjVUtId2JPaEJXN2pnTXYyOHZxT1Jic2VobgpMUVZWc2lPek8yZEMwdDl6ZGNQelpFN2dCdDA1dGcvWjZlWm9Id2ZyMDdFenB0cm5IbHBxdGtMMHJwYjVvK0tCCmxhVllxa0RtT1ZyaHl3aWw1dzBIc285U3E2Y0pHbVVZMjNmdnE5d1JsZVJIb3YxNlIrUEhQVzFBdm9Ga0ZDTC8KQnNjZUJFSWJuVnBuCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K\n server: https://192.168.1.201:6443\n name: \"\"\ncontexts: null\ncurrent-context: \"\"\nkind: Config\npreferences: {}\nusers: null\n"}}
I0830 12:50:45.821070 388661 token.go:228] [discovery] Retrying due to error: could not find a JWS signature in the cluster-info ConfigMap for token ID "qm4yvj"
I0830 12:50:50.812861 388661 round_trippers.go:466] curl -v -XGET -H "Accept: application/json, */*" -H "User-Agent: kubeadm/v1.30.4 (linux/amd64) kubernetes/a51b3b7" 'https://192.168.1.201:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s'
I0830 12:50:50.813828 388661 round_trippers.go:510] HTTP Trace: Dial to tcp:192.168.1.201:6443 succeed
I0830 12:50:50.825078 388661 round_trippers.go:553] GET https://192.168.1.201:6443/api/v1/namespaces/kube-public/configmaps/cluster-info?timeout=10s 200 OK in 12 milliseconds
I0830 12:50:50.825174 388661 round_trippers.go:570] HTTP Statistics: DNSLookup 0 ms Dial 0 ms TLSHandshake 6 ms ServerProcessing 3 ms Duration 12 ms
I0830 12:50:50.825195 388661 round_trippers.go:577] Response Headers:
I0830 12:50:50.825213 388661 round_trippers.go:580] Content-Type: application/json
I0830 12:50:50.825229 388661 round_trippers.go:580] X-Kubernetes-Pf-Flowschema-Uid: 9bf7f64e-a652-4337-a96a-e37d1d5b4606
I0830 12:50:50.825242 388661 round_trippers.go:580] X-Kubernetes-Pf-Prioritylevel-Uid: e9583c3b-b879-4d24-afe2-275e79c0e311
I0830 12:50:50.825255 388661 round_trippers.go:580] Content-Length: 2105
I0830 12:50:50.825268 388661 round_trippers.go:580] Date: Fri, 30 Aug 2024 04:50:50 GMT
I0830 12:50:50.825281 388661 round_trippers.go:580] Audit-Id: bb5c586b-7e34-4be3-a7c9-b473483cc6c1
I0830 12:50:50.825295 388661 round_trippers.go:580] Cache-Control: no-cache, private
|
解决问题
从日志看，应该是权限校验的问题，因为这个集群搭建的时候有太多不规范的操作了，导致像证书那些配置的多少多有点问题，尽快解决就是通过 yes|kubeadm reset 将集群配置重置之后重新安装。
参考资料
- failed to verify JWS signature of received cluster info object, can’t trust this API Server #109233
警告
本文最后更新于 2023年11月22日，文中内容可能已过时，请谨慎参考。