kubernetes

《Kubernetes权威指南》学习笔记第三篇-双向ca数字认证

6 min read

背景

本文基于二进制部署的kubernetes集群

通常内网中kubernetes各个组件与master间可以通过kube-apiserver的非安全端口8080通信，但有时候api server需要对外提供服务或者集群中的某些容器要访问api server来获取集群的某些信息，较为安全的做法就是启用https访问，而kubernetes提供了基于ca签名的双向数字认证来实现安全访问
本文配置ca证书是在二进制安装kubernetes基础上进行

1、master节点CA数字认证

创建一个专门用于临时存储的证书目录
mkdir -p /root/ca && cd ca

kube-apiserver数字证书配置

openssl genrsa -out ca.key 2048   
openssl  req -x509 -new -nodes -key ca.key  -subj "/CN=node161" -days 5000 -out ca.crt
openssl  genrsa -out server.key 2048

上面的-subj参数值为master主机名

创建用于生成kube-apiserver服务端的根证书及私钥的配置文件
cat master_ssl.cnf

[req]
req_extensions = v3_req
distinguished_name = req_distinguished_name
[req_distinguished_name]
[ v3_req ]
basicConstraints =CA:FALSE
keyUsage = nonRepudiation,digitalSignature,keyEncipherment
subjectAltName = @alt_names
[alt_names]
DNS.1 = kubernetes
DNS.2 = kubernetes.default
DNS.3 = kubernetes.default.svc
DNS.4 = kubernetes.default.svc.cluster.local
DNS.5 = node158
IP.1 = 10.0.0.1  ## 与apiserver里配置的参数--cluster-service-ip-range一致
IP.2 = 192.168.0.161

创建服务端证书和私钥

openssl  req -new -key server.key  -subj "/CN=node161" -config master_ssl.cnf  -out server.csr 
openssl  x509 -req -in server.csr  -CA ca.crt -CAkey ca.key  -CAcreateserial -days 5000 -extensions v3_req -extfile master_ssl.cnf -out server.crt

配置kube-apiserver配置文件
cat /etc/kubernetes/apiserver

KUBE_API_ARGS="--etcd-servers=http://192.168.0.161:2379 --insecure-port=0 --secure-port=6443 --client-ca-file=/root/ca/ca.crt --tls-private-key-file=/root/ca/server.key --tls-cert-file=/root/ca/server.crt  --insecure-bind-address=0.0.0.0  --service-cluster-ip-range=10.0.0.0/24 --service-node-port-range=1-65535 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,DefaultStorageClass,DefaultTolerationSeconds,MutatingAdmissionWebhook,ValidatingAdmissionWebhook,ResourceQuota --logtostderr=false --log-dir=/var/log/kubernetes --v=0"

kube-controller-manager证书配置

创建客户端证书

openssl  genrsa -out cs_client.key 2048
openssl req -new -key cs_client.key -subj "/CN=node161" -out cs_client.csr 
openssl  x509 -req -in cs_client.csr  -CA ca.crt -CAkey ca.key -CAcreateserial -out cs_client.crt -days 5000

配置kube-controller-manager配置文件
cat /etc/kubernetes/controller-manager

KUBE_CONTROLLER_MANAGER_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --service-account-private-key-file=/root/ca/server.key --root-ca-file=/root/ca/ca.crt  --logtostderr=false --log-dir=/var/log/kubernetes --v=0"

kube-scheduler证书配置

kube-scheduler复用kube-controller-manager的客户端证书

配置kube-scheduler配置文件
cat /etc/kubernetes/scheduler

KUBE_SCHEDULER_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig   --logtostderr=false --log-dir=/var/log/kubernetes --v=0"

kubelet证书配置

创建客户端证书

openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=node161" -out kubelet_client.csr
openssl  x509 -req -in kubelet_client.csr  -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

配置kubelet配置文件
cat /etc/kubernetes/kubelet

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=192.168.0.161  --logtostderr=false --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   --log-dir=/var/log/kubernetes --v=0"

kube-proxy证书配置

复用kubelet证书即可

配置kube-proxy配置文件
cat /etc/kubernetes/proxy

KUBE_PROXY_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=192.168.0.161  --cluster-cidr=10.0.0.0/24 --proxy-mode=ipvs --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

kubeconfig配置

cat /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: controllermanager
  user:
    client-certificate: /root/ca/cs_client.crt
    client-key: /root/ca/cs_client.key
- name: scheduler
  user:
    client-certificate: /root/ca/cs_client.crt
    client-key: /root/ca/cs_client.key
- name: kubelet
  user: 
    client-certificate: /root/ca/kubelet_client.crt
    client-key: /root/ca/kubelet_client.key
- name: proxy
  user:
    client-certificate: /root/ca/kubelet_client.crt
    client-key: /root/ca/kubelet_client.key
clusters:
- name: local
  cluster:
    certificate-authority: /root/ca/ca.crt
    server: https://192.168.0.161:6443
contexts:
- context:
    cluster: local
    user: controllermanager
  name: my-context1
- context:
    cluster: local
    user: scheduler
  name: my-context2
- context:
    cluster: local
    user: kubelet
  name: my-context3
- context:
    cluster: local
    user: proxy
  name: my-context4
current-context: my-context1
current-context: my-context2
current-context: my-context3
current-context: my-context4

kubectl证书配置

cat /root/.kube/config

apiVersion: v1
kind: Config
users:
- name: kubectl
  user:
    client-certificate: /root/ca/cs_client.crt
    client-key: /root/ca/cs_client.key
clusters:
- name: local
  cluster:
    certificate-authority: /root/ca/ca.crt
    server: https://192.168.0.161:6443
contexts:
- context:
    cluster: local
    user: kubectl
  name: my-context
current-context: my-context

ps：如果不配置为kubectl配置客户端证书，kubectl使用时就需要添加相关证书文件参数，比如
kubectl --server=https://192.168.0.161:6443 --certificate-authority=/root/ca/ca.crt --client-certificate=/root/ca/cs_client.crt --client-key=/root/ca/cs_client.key get nodes 显然这是不利于管理员操作的

2、node节点证书配置

mkdir -p /root/ca && cd ca
复制master上的ca.key和ca.crt到node上
scp ca.crt ca.key node162:/root/ca
scp ca.crt ca.key node163:/root/ca

kubelet证书配置

创建客户端证书

node162

openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=node162" -out kubelet_client.csr
openssl  x509 -req -in kubelet_client.csr  -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

node163

openssl genrsa -out kubelet_client.key 2048
openssl req -new -key kubelet_client.key -subj "/CN=node163" -out kubelet_client.csr
openssl  x509 -req -in kubelet_client.csr  -CA ca.crt -CAkey ca.key -CAcreateserial -out kubelet_client.crt -days 5000

上面的-subj参数值要与kubelet的配置文件里的参数--hostname-override保持一致

创建kubelet配置文件

node162
cat /etc/kubernetes/kubelet

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=192.168.0.162 --logtostderr=false --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   --log-dir=/var/log/kubernetes --v=0"

node163
cat /etc/kubernetes/kubelet

KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=192.168.0.163 --logtostderr=false --pod-infra-container-image=registry.cn-hangzhou.aliyuncs.com/google-containers/pause-amd64:3.0   --log-dir=/var/log/kubernetes --v=0"

kube-proxy证书配置

kube-proxy复用kubelet的客户端证书

创建kube-proxy配置文件

node162
cat /etc/kubernetes/proxy

KUBE_PROXY_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=192.168.0.162  --cluster-cidr=10.0.0.0/24 --proxy-mode=ipvs --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

node163
cat /etc/kubernetes/proxy

KUBE_PROXY_ARGS="--kubeconfig=/etc/kubernetes/kubeconfig --hostname-override=192.168.0.163  --cluster-cidr=10.0.0.0/24 --proxy-mode=ipvs --logtostderr=false --log-dir=/var/log/kubernetes --v=2"

kubeconfig配置

cat /etc/kubernetes/kubeconfig

apiVersion: v1
kind: Config
users:
- name: kubelet
  user: 
    client-certificate: /root/ca/kubelet_client.crt
    client-key: /root/ca/kubelet_client.key
- name: proxy
  user:
    client-certificate: /root/ca/kubelet_client.crt
    client-key: /root/ca/kubelet_client.key
clusters:
- name: local
  cluster:
    certificate-authority: /root/ca/ca.crt
    server: https://192.168.0.161:6443
contexts:
- context:
    cluster: local
    user: kubelet
  name: my-context1
- context:
    cluster: local
    user: proxy
  name: my-context2
current-context: my-context1
current-context: my-context2

3、验证

[root@node161 ~]# kubectl get cs,nodes
Warning: v1 ComponentStatus is deprecated in v1.19+
NAME                                 STATUS    MESSAGE             ERROR
componentstatus/etcd-0               Healthy   {"health":"true"}   
componentstatus/controller-manager   Healthy   ok                  
componentstatus/scheduler            Healthy   ok                  

NAME                 STATUS   ROLES    AGE     VERSION
node/192.168.0.161   Ready    master   2d23h   v1.19.0
node/192.168.0.162   Ready    node     2d22h   v1.19.0
node/192.168.0.163   Ready    node     2d22h   v1.19.0

GitHub Repository

Published in:kubernetes
linuxwt
linuxwt
我是王腾，2016年毕业于武汉工程大学邮电与信息工程学院，专业是通信工程，2016年毕业后在上海AMEC做了一年Helpdesk，自学了linux，2017年初回武汉开始做系统运维方面的工作持续至今，目前就职中国移动设计院有限公司。
Previous Post
kubernetes

《Kubernetes权威指南》学习笔记第二篇-二进制方式安装

Next Post
kubernetes

《Kubernetes权威指南》学习笔记第四篇-token认证和HTTPbase认证