背景
本文基于二进制部署的kubernetes集群
通常内网中kubernetes各个组件与master间可以通过kube-apiserver的非安全端口8080通信,但有时候api server需要对外提供服务或者集群中的某些容器要访问api server来获取集群的某些信息,较为安全的做法就是启用https访问,而kubernetes提供了基于ca签名的双向数字认证来实现安全访问
本文配置ca证书是在二进制安装kubernetes基础上进行
1、master节点CA数字认证
创建一个专门用于临时存储的证书目录
mkdir -p /root/ca && cd ca
kube-apiserver数字证书配置
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=node158" -days 5000 -out ca.crt
openssl genrsa -out server.key 2048
上面的-subj参数值为master主机名
创建用于生成apiserver服务端的根证书及私钥的配置文件
cat master_ssl.cnf
[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints =CA:FALSE
keyUsage = nonRepudiation,digitalSignature,keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = node158
IP.1 = 169.169.0.1
IP.2 = 192.168.0.158
创建服务端证书和私钥
openssl req -new -key server.key -subj "/CN=node158" -config master_ssl.cnf -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt
配置apiserver配置文件
cat /etc/kubernetes/apiserver
KUBE_API_ARGS="--etcd-servers=http://127.0.0.1:2379 --insecure-bind-address=0.0.0.0 --insecure-port=0 --secure-port=6443 --client-ca-file=/root/ca/ca.crt --tls-private-key-file=/root/ca/server.key --tls-cert-file=/root/ca/server.crt --service-cluster-ip-range=169.169.0.0/16 --service-node-port-range=1-65535 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --logtostderr=false --log-dir=/var/log/kubernetes --v=0"
主要有以下两处修改
- 添加了参数--client-ca-file、--secure-port、--tls-cert-file、--tls-private-key-file
- 修改了参数--insecure-port
kube-controller-manager证书配置
openssl genrsa -out cs_client.key 2048
openssl req -new -key cs_client.key -subj "/CN=node158" -out cs_client.csr
openssl x509 -req -in cs_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000
cp cs_client.key cs_client.csr cs_client.crt /var/run/kubernetes/
通过创建kubeconfig来配置客户端证书
cat /etc/kubernetes/kubeconfig
apiVersion: v1
kind: Config
users:
- name: controllermanager
user:
client-certificate: /root/ca/cs_client.crt
client-key: /root/ca/cs_client.key
clusters:
- name: local
cluster:
certificate-authority: /root/ca/ca.crt
server: https://192.168.0.158:6443
contexts:
- context:
cluster: local
user: controllermanager
name: my-context
current-context: my-context
配置controller-manager配置文件
cat /etc/kubernetes/controller-manager
KUBE_CONTROLLER_MANAGER_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --service-account-private-key-file=/root/ca/server.key --root-ca-file=/root/ca/ca.crt --logtostderr=false --log-dir=/var/log/kubernetes --v=0"
注意上面的证书相关参数--service-account-private-key-file在不同的kubernetes版本里可能不一样,本文使用的是v1.19,在v1.14中使用的是--service-account-key-file如果不确定使用kube-controller-manager --help查看
kube-scheduler证书配置
KUBE_SCHEDULER_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --tls-private-key-file=/root/ca/server.key --client-ca-file=/root/ca/ca.crt --tls-cert-file=/root/ca/server.crt --logtostderr=false --log-dir=/var/log/kubernetes --v=0"
2、node节点证书配置
node159 node160上操作
mkdir -p /root/ca && cd ca
kubelet证书配置
复制kube-apiserver上的ca.key和ca.crt到node上
scp ca.crt ca.key node159:/root/ca
scp ca.crt ca.key node160:/root/ca
openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=192.168.0.159" -out kubelet_client.csr
openssl x509 -req -in kubelet_client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000
上面的-subj参数值要与kubelet的配置文件里的参数--hostname-override保持一致
kubeconfig配置客户端证书相关参数
cat /etc/kubernetes/kubeconfig
apiVersion: v1
kind: Config
users:
- name: kubelet
user:
client-certificate: /root/ca/kubelet_client.crt
client-key: /root/ca/kubelet_client.key
clusters:
- name: local
cluster:
certificate-authority: /root/ca/ca.crt
server: https://192.168.0.158:6443
contexts:
- context:
cluster: local
user: kubelet
name: my-context
current-context: my-context
kube-proxy证书配置
复用kubelet证书配置即可,其余参照二进制安装kubernetes相关部分
3、验证
node158操作
kubectl --server=https://192.168.0.158:6443 --certificate-authority=/root/ca/ca.crt --client-certificate=/root/ca/cs_client.crt --client-key=/root/ca/cs_client.key get nodes
返回
192.168.0.159 Ready <none> 3d4h v1.19.0
192.168.0.160 Ready <none> 3d4h v1.19.0
kubectl --server=https://192.168.0.158:6443 --certificate-authority=/root/ca/ca.crt --client-certificate=/root/ca/cs_client.crt --client-key=/root/ca/cs_client.key get cs
返回
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME STATUS MESSAGE ERROR
etcd-0 Healthy {"health":"true"}
scheduler Healthy ok
controller-manager Healthy ok
返回 正常状态,这说明已成功完成了ca双向数字认证