# K8s 亲和力 Affinity
Pod 和节点之间的关系：
- 某些 Pod 优先选择有 ssd=true 标签的节点，如果没有在考虑部署到其它节点；
- 某些 Pod 需要部署在 ssd=true 和 type=physical 的节点上，但是优先部署在 ssd=true 的节点上。
Pod 和 Pod 之间的关系：
- 同一个应用的 Pod 不同的副本或者同一个项目的应用尽量或必须不部署在同一个节点或者符合某个标签的一类节点上或者不同的区域；
- 相互依赖的两个 Pod 尽量或必须部署在同一个节点上或者同一个域内。
# 1. Affinity 分类
# 2. 节点亲和力配置详解
# 2.1 硬亲和力 required
# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy
labels:
app: nginx-deploy
annotations:
app: nginx-deploy
namespace: default
spec:
selector:
matchLabels:
app: nginx-deploy
replicas: 5
template:
metadata:
labels:
app: nginx-deploy
spec:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- k8s-node01
- k8s-node02
restartPolicy: Always
containers:
- name: nginx-deploy
image: nginx:latest
imagePullPolicy: IfNotPresent
resources:
limits:
memory: 1024Mi
cpu: 1
requests:
memory: 128Mi
cpu: 100m
- requiredDuringSchedulingIgnoredDuringExecution：硬亲和力配置
- nodeSelectorTerms：节点选择器配置，可以配置多个 matchExpressions（满足其一即可）
- matchExpressions：matchExpressions 下可以配置多个 key、values（都需要满足），其中 values 可以配置多个（满足其一即可）
- operator：
- IN 相当于 key = value 的形式，NotIn 相当于 key!=value 的形式 (反亲和力)
- Exists: 节点存在 label 的 key 为指定的值即可，不能配置 values 字段
- DoesNotExist: 节点不存在 label 的 key 为指定的值即可，不能配置 values 字段
- Gt：大于 value 指定的值
- Lt：小于 value 指定的值
# 2.2 软亲和力 preferred
# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy
labels:
app: nginx-deploy
namespace: default
spec:
selector:
matchLabels:
app: nginx-deploy
replicas: 6
template:
metadata:
labels:
app: nginx-deploy
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: ssd
operator: In
values:
- 'true'
- weight: 50
preference:
matchExpressions:
- key: kubernetes.io/hostname
operator: In
values:
- k8s-master01
restartPolicy: Always
containers:
- name: nginx-deploy
image: nginx
imagePullPolicy: IfNotPresent
resources:
limits:
memory: 1024Mi
cpu: 1
requests:
memory: 128Mi
cpu: 100m
- preferredDuringSchedulingIgnoredDuringExecution：软亲和力配置
- weight：软亲和力的权重，权重越高优先级越大，范围 1-100
- matchExpressions：matchExpressions 下可以配置多个 key、values（都需要满足），其中 values 可以配置多个（满足其一即可）
- operator：
- IN 相当于 key = value 的形式，NotIn 相当于 key!=value 的形式 (反亲和力)
- Exists: 节点存在 label 的 key 为指定的值即可，不能配置 values 字段
- DoesNotExist: 节点不存在 label 的 key 为指定的值即可，不能配置 values 字段
- Gt：大于 value 指定的值
- Lt：小于 value 指定的值
# 3. Pod 亲和力详解
[root@k8s-master01 ~]# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-deploy
name: nginx-deploy
spec:
replicas: 4
selector:
matchLabels:
app: nginx-deploy
template:
metadata:
labels:
app: nginx-deploy
spec:
affinity:
podAntiAffinity: #pod硬反亲和力
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx-deploy
topologyKey: kubernetes.io/hostname
podAntiAffinity: #pod软反亲和力
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx-deploy
namespaces: #和哪个命名空间的Pod进行匹配，为空为当前命名空间
- default
topologyKey: kubernetes.io/hostname
labelSelector：Pod 选择器配置，可以配置多个
matchExpressions：matchExpressions 下可以配置多个 key、values（都需要满足），其中 values 可以配置多个（满足其一即可）
topologyKey：匹配的拓扑域的 key，也就是节点上 label 的 key，key 和 value 相同的为同一个域，可以用于标注不同的机房和地区
Namespaces: 和哪个命名空间的 Pod 进行匹配，为空为当前命名空间
operator：配置和节点亲和力一致，但是没有 Gt 和 Lt
IN 相当于 key = value 的形式；
Exists: 节点存在 label 的 key 为指定的值即可，不能配置 values 字段；
DoesNotExist: 节点不存在 label 的 key 为指定的值即可，不能配置 values 字段
# 4. 节点亲和力配置示例
Pod 尽量部署在 ssd=true 和 type=physical 的节点上，但是优先部署在 ssd=true 的节点上，不能部署 label 为 gpu=true 的节点。
[root@k8s-master01 ~]# kubectl label nodes k8s-node01 ssd=true
[root@k8s-master01 ~]# kubectl label nodes k8s-master01 ssd=true
[root@k8s-master01 ~]# kubectl label nodes k8s-master01 gpu=true
[root@k8s-master01 ~]# kubectl label nodes k8s-node02 type=physical
[root@k8s-master01 ~]# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy
labels:
app: nginx-deploy
annotations:
app: nginx-deploy
namespace: default
spec:
selector:
matchLabels:
app: nginx-deploy
replicas: 5
template:
metadata:
labels:
app: nginx-deploy
annotations:
app: nginx-deploy
spec:
affinity:
nodeAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
preference:
matchExpressions:
- key: ssd
operator: In
values:
- 'true'
- key: gpu
operator: NotIn
values:
- 'true'
- weight: 50
preference:
matchExpressions:
- key: type
operator: In
values:
- physical
restartPolicy: Always
containers:
- name: nginx-deploy
image: nginx
imagePullPolicy: IfNotPresent
resources:
limits:
memory: 1024Mi
cpu: 1
requests:
memory: 128Mi
cpu: 100m
volumeMounts:
- name: tz-config
mountPath: /usr/share/zoneinfo/Asia/Shanghai
- name: tz-config
mountPath: /etc/localtime
- name: timezone
mountPath: /etc/timezone
volumes:
- name: tz-config
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: ""
- name: timezone
hostPath:
path: /etc/timezone
type: ""
[root@k8s-master01 ~]# kubectl apply -f nginx-deploy.yaml
[root@k8s-master01 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-7d65fbdf-2b4jr 1/1 Running 0 5s 172.16.85.236 k8s-node01 <none> <none>
nginx-deploy-7d65fbdf-jjzwr 1/1 Running 0 5s 172.16.58.251 k8s-node02 <none> <none>
nginx-deploy-7d65fbdf-kx5lm 1/1 Running 0 5s 172.16.85.237 k8s-node01 <none> <none>
nginx-deploy-7d65fbdf-lrmcg 1/1 Running 0 5s 172.16.85.238 k8s-node01 <none> <none>
nginx-deploy-7d65fbdf-n6mlp 1/1 Running 0 5s 172.16.58.250 k8s-node02 <none> <none>
# 5. Pod 亲和力、反亲和力配置示例
# 5.1 Pod 反亲和力 required
同一个应用部署在不同的宿主机
#1.节点存在污点pod无法调度至该节点
# kubectl describe nodes|grep -i taint
Taints: <none>
Taints: <none>
Taints: <none>
Taints: <none>
Taints: <none>
#2.pod反亲和力required
# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy
labels:
app: nginx-deploy
namespace: default
spec:
selector:
matchLabels:
app: nginx-deploy
replicas: 5
template:
metadata:
labels:
app: nginx-deploy
spec:
affinity:
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx-deploy
topologyKey: kubernetes.io/hostname
restartPolicy: Always
containers:
- name: nginx-deploy
image: nginx
imagePullPolicy: IfNotPresent
resources:
limits:
memory: 1024Mi
cpu: 1
requests:
memory: 128Mi
cpu: 100m
volumeMounts:
- name: tz-config
mountPath: /usr/share/zoneinfo/Asia/Shanghai
- name: tz-config
mountPath: /etc/localtime
- name: timezone
mountPath: /etc/timezone
volumes:
- name: tz-config
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: ""
- name: timezone
hostPath:
path: /etc/timezone
type: ""
#3.部署deployment
[root@k8s-master01 ~]# kubectl apply -f nginx-deploy.yaml
[root@k8s-master01 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-5787887b6f-4654b 1/1 Running 0 4s 172.16.85.234 k8s-node01 <none> <none>
nginx-deploy-5787887b6f-8mq7s 1/1 Running 0 4s 172.16.122.152 k8s-master02 <none> <none>
nginx-deploy-5787887b6f-fdkft 1/1 Running 0 4s 172.16.58.247 k8s-node02 <none> <none>
nginx-deploy-5787887b6f-jzcmd 1/1 Running 0 4s 172.16.32.152 k8s-master01 <none> <none>
nginx-deploy-5787887b6f-qdq9g 1/1 Running 0 4s 172.16.195.14 k8s-master03 <none> <none>
#4.将副本扩成6个，由于K8s集群只有5个节点，即5个topologyKey（拓扑域），每个域只能有一个副本，所以有一个pod会pending
[root@k8s-master01 ~]# kubectl scale deploy nginx-deploy --replicas=6
[root@k8s-master01 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-5787887b6f-4654b 1/1 Running 0 4m44s 172.16.85.234 k8s-node01 <none> <none>
nginx-deploy-5787887b6f-8mq7s 1/1 Running 0 4m44s 172.16.122.152 k8s-master02 <none> <none>
nginx-deploy-5787887b6f-fdkft 1/1 Running 0 4m44s 172.16.58.247 k8s-node02 <none> <none>
nginx-deploy-5787887b6f-jzcmd 1/1 Running 0 4m44s 172.16.32.152 k8s-master01 <none> <none>
nginx-deploy-5787887b6f-qdq9g 1/1 Running 0 4m44s 172.16.195.14 k8s-master03 <none> <none>
nginx-deploy-5787887b6f-sztm7 0/1 Pending 0 9s <none> <none> <none> <none>
[root@k8s-master01 ~]# kubectl describe pods nginx-deploy-5787887b6f-sztm7
...
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 102s default-scheduler 0/5 nodes are available: 5 node(s) didn't match pod anti-affinity rules. preemption: 0/5 nodes are available: 5 No preemption victims found for incoming pod.
将副本扩成 6 个，有一个会 pending 状态，原因 K8s 集群只有 5 个节点，即 5 个 topologyKey（拓扑域），每个拓扑域只能有一个副本，所以有一个 pod 会 pending。
topologyKey：匹配的拓扑域的 key，也就是节点上 label 的 key，key 和 value 相同的为同一个域，可以用于标注不同的机房和地区。
# 5.2 Pod 反亲和力 preferred
同一个应用尽量部署在不同的宿主机
#1.节点存在污点pod无法调度至该节点
# kubectl describe nodes|grep -i taint
Taints: <none>
Taints: <none>
Taints: <none>
Taints: <none>
Taints: <none>
#2.pod反亲和力preferred
# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
name: nginx-deploy
labels:
app: nginx-deploy
namespace: default
spec:
selector:
matchLabels:
app: nginx-deploy
replicas: 6
template:
metadata:
labels:
app: nginx-deploy
spec:
affinity:
podAntiAffinity:
preferredDuringSchedulingIgnoredDuringExecution:
- podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx-deploy
topologyKey: kubernetes.io/hostname
weight: 100
restartPolicy: Always
containers:
- name: nginx-deploy
image: nginx
imagePullPolicy: IfNotPresent
resources:
limits:
memory: 1024Mi
cpu: 1
requests:
memory: 128Mi
cpu: 100m
volumeMounts:
- name: tz-config
mountPath: /usr/share/zoneinfo/Asia/Shanghai
- name: tz-config
mountPath: /etc/localtime
- name: timezone
mountPath: /etc/timezone
volumes:
- name: tz-config
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: ""
- name: timezone
hostPath:
path: /etc/timezone
type: ""
#3.部署deployment
[root@k8s-master01 ~]# kubectl apply -f nginx-deploy.yaml
[root@k8s-master01 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-7c47567b79-97qs5 1/1 Running 0 6s 172.16.122.153 k8s-master02 <none> <none>
nginx-deploy-7c47567b79-g49h4 1/1 Running 0 6s 172.16.85.235 k8s-node01 <none> <none>
nginx-deploy-7c47567b79-g5n2s 1/1 Running 0 6s 172.16.58.248 k8s-node02 <none> <none>
nginx-deploy-7c47567b79-g5v5b 1/1 Running 0 6s 172.16.195.15 k8s-master03 <none> <none>
nginx-deploy-7c47567b79-pjwws 1/1 Running 0 6s 172.16.58.249 k8s-node02 <none> <none>
nginx-deploy-7c47567b79-q2hn5 1/1 Running 0 6s 172.16.32.153 k8s-master01 <none> <none>
# 5.3 Pod 亲和力 required
同一个应用必须部署在同一个宿主机
[root@k8s-master01 ~]# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-deploy
name: nginx-deploy
spec:
replicas: 8
selector:
matchLabels:
app: nginx-deploy
template:
metadata:
labels:
app: nginx-deploy
spec:
affinity:
podAffinity: #pod硬亲和力
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx-deploy
topologyKey: kubernetes.io/hostname
containers:
- image: nginx
name: nginx
volumeMounts:
- name: timezone
mountPath: /etc/timezone
- name: tz-config
mountPath: /usr/share/zoneinfo/Asia/Shanghai
- name: tz-config
mountPath: /etc/localtime
volumes:
- name: timezone
hostPath:
path: /etc/timezone
type: File
- name: tz-config
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: File
[root@k8s-master01 ~]# kubectl apply -f nginx-deploy.yaml
[root@k8s-master01 ~]# kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
nginx-deploy-dbcc4d65c-2sthn 1/1 Running 0 12s 172.16.58.255 k8s-node02 <none> <none>
nginx-deploy-dbcc4d65c-78nxf 1/1 Running 0 12s 172.16.58.197 k8s-node02 <none> <none>
nginx-deploy-dbcc4d65c-82ssq 1/1 Running 0 12s 172.16.58.194 k8s-node02 <none> <none>
nginx-deploy-dbcc4d65c-986cb 1/1 Running 0 12s 172.16.58.254 k8s-node02 <none> <none>
nginx-deploy-dbcc4d65c-9rnt7 1/1 Running 0 12s 172.16.58.252 k8s-node02 <none> <none>
nginx-deploy-dbcc4d65c-knm8q 1/1 Running 0 12s 172.16.58.195 k8s-node02 <none> <none>
nginx-deploy-dbcc4d65c-kx56f 1/1 Running 0 12s 172.16.58.253 k8s-node02 <none> <none>
nginx-deploy-dbcc4d65c-sqlhf 1/1 Running 0 12s 172.16.58.198 k8s-node02 <none> <none>
# 5.4 Pod 亲和力 preferre
同一个应用尽量部署在同一个宿主机
# cat nginx-deploy.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: nginx-deploy
name: nginx-deploy
spec:
replicas: 20
selector:
matchLabels:
app: nginx-deploy
template:
metadata:
labels:
app: nginx-deploy
spec:
affinity:
podAffinity: #pod软亲和力
preferredDuringSchedulingIgnoredDuringExecution:
- weight: 100
podAffinityTerm:
labelSelector:
matchExpressions:
- key: app
operator: In
values:
- nginx-deploy
namespaces: #和哪个命名空间的Pod进行匹配，为空为当前命名空间
- default
topologyKey: kubernetes.io/hostname
containers:
- image: nginx
name: nginx
volumeMounts:
- name: timezone
mountPath: /etc/timezone
- name: tz-config
mountPath: /usr/share/zoneinfo/Asia/Shanghai
- name: tz-config
mountPath: /etc/localtime
volumes:
- name: timezone
hostPath:
path: /etc/timezone
type: File
- name: tz-config
hostPath:
path: /usr/share/zoneinfo/Asia/Shanghai
type: File
