-
如果在pod中定义了preStop hook,在停止pod前会被调用。如果在宽限期过后,preStop hook依然在运行,第二步会再增加2秒的宽限期;
-
向Pod中的进程发送TERM信号;
-
跟第三步同时,该Pod将从该service的端点列表中删除,不再是replication controller的一部分。关闭的慢的pod将继续处理load balancer转发的流量;
-
过了宽限期后,将向Pod中依然运行的进程发送SIGKILL信号而杀掉进程。
-
Kublete会在API server中完成Pod的的删除,通过将优雅周期设置为0(立即删除)。Pod在API中消失,并且在客户端也不可见。
删除宽限期默认是30秒。
kubectl delete
命令支持
—grace-period=
<seconds
>
选项,允许用户设置自己的宽限期。如果设置为0将强制删除pod。在kubectl>=1.5版本的命令中,你必须同时使用
--force
和
--grace-period=0
来强制删除pod。
Pod的强制删除是通过在集群和etcd中将其定义为删除状态。当执行强制删除命令时,API server不会等待该pod所运行在节点上的kubelet确认,就会立即将该pod从API server中移除,这时就可以创建跟原pod同名的pod了。这时,在节点上的pod会被立即设置为terminating状态,不过在被强制删除之前依然有一小段优雅删除周期
Container probes
Probe 是在容器上 kubelet 的定期执行的诊断,kubelet 通过调用容器实现的 Handler 来诊断。目前有三种 Handlers :
ExecAction:在容器内部执行指定的命令,如果命令以状态代码 0 退出,则认为诊断成功。
TCPSocketAction:对指定 IP 和端口的容器执行 TCP 检查,如果端口打开,则认为诊断成功。
HTTPGetAction:对指定 IP + port + path路径上的容器的执行 HTTP Get 请求。如果响应的状态代码大于或等于 200 且小于 400,则认为诊断成功
pod生命周期的重要行为:
初始化容器
Init Container在所有容器运行之前执行(run-to-completion),常用来初始化配置。
apiVersion: v1
kind: Pod
metadata:
name: init-demo
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: workdir
mountPath: /usr/share/nginx/html
# These containers are run during pod initialization
initContainers:
- name: install
image: busybox
command:
- wget
- "-O"
- "/work-dir/index.html"
- http://kubernetes.io
volumeMounts:
- name: workdir
mountPath: "/work-dir"
dnsPolicy: Default
volumes:
- name: workdir
emptyDir: {}
容器探测
为了确保容器在部署后确实处在正常运行状态,Kubernetes提供了两种探针(Probe,支持exec、tcp和httpGet方式)来探测容器的状态:
Probe 是在容器上 kubelet 的定期执行的诊断,kubelet 通过调用容器实现的 Handler 来诊断。目前有三种 Handlers :
ExecAction:在容器内部执行指定的命令,如果命令以状态代码 0 退出,则认为诊断成功。
TCPSocketAction:对指定 IP 和端口的容器执行 TCP 检查,如果端口打开,则认为诊断成功。
HTTPGetAction:对指定 IP + port + path路径上的容器的执行 HTTP Get 请求。如果响应的状态代码大于或等于 200 且小于 400,则认为诊断成功
LivenessProbe:探测应用是否处于健康状态,如果不健康则删除重建改容器
ReadinessProbe:探测应用是否启动完成并且处于正常服务状态,如果不正常则更新容器的状态
kubectl explain pods.spec.containers.livenessProbe
vim liveness-exec-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: liveness-exec-pod.yaml
namespace: default
spec:
containers:
- name: liveness-exec-container
image: busybox:latest
imagePullPolicy: IfNotPresent
command: ["/bin/sh","-c","touch /tmp/healthy;sleep 30;rm -rf /tmp/healthy;sleep 3600"]
livenessProbe:
exec:
command: ["test","-e","/tmp/healthy"]
initialDelaySeconds: 1
periodSeconds: 3
kubectl create -f liveness-exec.yaml
[root@k8s-master k8s]# kubectl get pods liveness-exec-pod
NAME READY STATUS RESTARTS AGE
liveness-exec-pod 1/1 Running
3
4m4s
[root@k8s-master k8s]#kubectl describe pods liveness-exec-pod
Name: liveness-exec-pod
Containers:
liveness-exec-container:
Container ID: docker://8d09c9e59b7ab18cc777f7408b7ae889c4b8439d2481a3bb85b0b5dcde166e44
Command:
/bin/sh
-c
touch /tmp/healthy;sleep 30;rm -rf /tmp/healthy;sleep 3600
State: Running
Started: Tue, 23 Apr 2019 18:05:46 +0800
Last State: Terminated
Reason: Error
Exit Code: 137
Started: Tue, 23 Apr 2019 18:04:37 +0800
Finished: Tue, 23 Apr 2019 18:05:46 +0800
Ready: True
Restart Count: 2
Liveness: exec [test -e /tmp/healthy] delay=1s timeout=1s period=3s #success=1 #failure=3
vim liveness-httpget-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: liveness-httpget-pod
namespace: default
spec:
containers:
- name: liveness-httpget-container
image: ikubernetes/myapp:v1
imagePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
livenessProbe:
httpGet:
port: http
path: /index.html
initialDelaySeconds: 1
periodSeconds: 3
kubectl exec -it liveness-httpget-pod -- /bin/sh
/ # rm -rf /usr/share/nginx/html/index.html
[root@k8s-master k8s]# kubectl get pod liveness-httpget-pod
NAME READY STATUS RESTARTS AGE
liveness-httpget-pod 1/1 Running
1
93s
只会重启一次,因为在重启文件恢复了,没有问题了
查看详细的信息
kubectl describe pod liveness-httpget-pod
Name: liveness-httpget-pod
Containers:
liveness-httpget-container:
Last State: Terminated
Ready: True
Restart Count: 1
Liveness: http-get http://:http/index.html delay=1s timeout=1s period=3s #success=1 #failure=3
Normal Killing 93s kubelet, k8s-node1 Killing container with id
docker://liveness-httpget-container:Container failed liveness probe. Container will be killed and recreated.
kubectl explain pods.spec.containers.readinessProbe
vim readiness-httpget-pod
apiVersion: v1
kind: Pod
metadata:
name: readiness-httpget-pod
namespace: default
spec:
containers:
- name: readiness-httpget-container
image: ikubernetes/myapp:v1
magePullPolicy: IfNotPresent
ports:
- name: http
containerPort: 80
readinessProbe:
httpGet:
port: http
path: /index.html
[root@k8s-master k8s]# kubectl get pods readiness-httpget-pod
NAME READY STATUS RESTARTS AGE
readiness-httpget-pod 1/1 Running 0 59s
[root@k8s-master ~]# kubectl exec -it readiness-httpget-pod -- /bin/sh
/ # rm -rf /usr/share/nginx/html/index.html
[root@k8s-master k8s]# kubectl get pods readiness-httpget-pod
NAME READY STATUS RESTARTS AGE
readiness-httpget-pod
0/1
Running 0 60s
[root@k8s-master k8s]# kubectl describe pods readiness-httpget-pod
Name: readiness-httpget-pod
readiness-httpget-container:
Ready: False
Restart Count: 0
Readiness: http-get http://:http/index.html delay=0s timeout=1s period=10s #success=1 #failure=3
Warning Unhealthy 0s (x15 over 10m) kubelet, k8s-node1 Readiness probe failed: HTTP probe failed with statuscode: 404
[root@k8s-master ~]# kubectl exec -it readiness-httpget-pod -- /bin/sh
/ # echo "test" >> /usr/share/nginx/html/index.html
[root@k8s-master k8s]# kubectl get pods readiness-httpget-pod
NAME READY STATUS RESTARTS AGE
readiness-httpget-pod
1/1
Running 0 2m41s
看到恢复了正常的状态
kubectl explain pods.spec.containers.lifecycle
kubectl explain pods.spec.containers.lifecycle.postStart
kubectl explain pods.spec.containers.lifecycle.preStop
容器生命周期钩子(Container Lifecycle Hooks)监听容器生命周期的特定事件,并在事件发生时执行已注册的回调函数。支持两种钩子:
postStart: 容器启动后执行,注意由于是异步执行,它无法保证一定在ENTRYPOINT之后运行。如果失败,容器会被杀死,
并根据RestartPolicy决定是否重启preStop:容器停止前执行,常用于资源清理。如果失败,容器同样也会被杀死
而钩子的回调函数支持两种方式:
exec:在容器内执行命令
httpGet:向指定URL发起GET请求
postStart和preStop钩子示例
:
apiVersion: v1
kind: Pod
metadata:
name: lifecycle-demo
spec:
containers:
- name: lifecycle-demo-container
image: nginx
lifecycle:
postStart:
exec:
command: ["/bin/sh", "-c", "echo Hello from the postStart handler > /usr/share/message"]
preStop:
exec:
command: ["/usr/sbin/nginx","-s","quit"]
vim poststart-pod.yaml
apiVesion: v1
kind: Pod
metadata:
name: poststart-pod
namespace: default
spec:
containers:
- name: busybox-httpd
image: busybox:latest
imagePullPolicy: IfNotPresent
lifecycle:
postStart:
exec:
command: ['mkdir','-p','/data/web/html']
command: ["/bin/httpd"]
args: ["-f","-h /data/web/html"]
[root@k8s-master k8s]# kubectl get pods poststart-pod
NAME READY STATUS RESTARTS AGE
poststart-pod 0/1 PostStartHookError: command 'mkdir -p /data/web/html' exited with 126: 2 18s
[root@k8s-master k8s]# kubectl get pods poststart-pod
NAME READY STATUS RESTARTS AGE
poststart-pod 0/1 PostStartHookError: comman exited with 126: 4 103s
kubectl describe pod poststart-pod
Warning FailedPostStartHook 6s (x4 over 53s)
kubelet, k8s-node1 Exec lifecycle hook ([mkdir -p /data/web/html]) for Container "busybox-httpd" in Pod
"poststart-pod_default(6c37ff00-65bd-11e9-b018-001c42baaf43)" failed - error:
command 'mkdir -p /data/web/html' exited with 126: , message: "cannot exec in a stopped state: unknown\r\n"
vim vim poststart-pod.yaml
apiVersion: v1
kind: Pod
metadata:
name: poststart-pod
namespace: default
spec:
containers:
- name: busybox-httpd
image: busybox:latest
imagePullPolicy: IfNotPresent
lifecycle:
postStart:
exec:
command: ['mkdir','-p','/data/web/html']
command: ['/bin/sh','-c','sleep 3600']
注意:postStart里面的command不能强依赖 pod的command,上面的报错是有误导的,应该是POD command 运行早于postStat
restartPolicy <string>
Always:只要退出就重启
OnFailure:失败退出(exit code不等于0)时重启
Never:只要退出就不再重启 (注意,这里的重启是指在Pod所在Node上面本地重启,并不会调度到其他Node上去)
参考文档:
https://blog.csdn.net/Dou_Hua_Hua/article/details/108164617
https://www.jianshu.com/p/91625e7a8259?utm_source=oschina-app
https://blog.csdn.net/horsefoot/article/details/52324830
https://www.cnblogs.com/linuxk/p/9569618.html