添加链接 注册    登录
link管理
链接快照平台
  • 输入网页链接，自动生成快照
  • 标签化管理网页链接
相关文章推荐
飞奔的企鹅  ·  0026738: WebP ...·  1 月前    · 
闷骚的黑框眼镜  ·  References | ...·  2 月前    · 
正直的手电筒  ·  配置 Rsbuild - Rsbuild·  2 月前    · 
文质彬彬的红豆  ·  HTRA1 x HTRA2 - ...·  7 月前    · 
有腹肌的小熊猫  ·  “欢乐”迪士尼崩了:暴跌2000亿 ...·  1 年前    · 
link管理  ›  K8S的更新、回滚、伸缩_failedgetresourcemetric
cpu参数 edg战队 metrics kubernetes
https://blog.csdn.net/chenleiking/article/details/80197975
机灵的乌冬面
1 月前

一个相对完整的应用可能还包括：volume、env、service、ingress等 

root@master-1:~/kubernetes/example# kubectl apply -f nginx-deployment.yaml 
deployment.apps "nginx-deployment" created
root@master-1:~/kubernetes/example# kubectl get all
NAME                                    READY     STATUS    RESTARTS   AGE
pod/nginx-deployment-85b548b894-ct66d   1/1       Running   0          20s
pod/nginx-deployment-85b548b894-pfmvl   1/1       Running   0          20s
pod/nginx-deployment-85b548b894-v88qq   1/1       Running   0          20s
pod/nginx-deployment-85b548b894-zxml8   1/1       Running   0          20s
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   7d
NAME                               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   4         4         4            4           20s
NAME                                          DESIRED   CURRENT   READY     AGE
replicaset.apps/nginx-deployment-85b548b894   4         4         4         20s

忽略service/kubernetes～～ 

graph TB;
    deployment.apps/nginx-deployment --> replicaset.apps/nginx-deployment-85b548b894
    replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-ct66d
    replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-pfmvl
    replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-v88qq
    replicaset.apps/nginx-deployment-85b548b894 --> pod/nginx-deployment-85b548b894-zxml8
  • 每个deployment会包含一个或者多个replicaset
  • 每个replicaset会可以包含零个或者多个pod
  • 每个replicaset对应deployment的一个revision
  • 每次更新时，pod总是在一个replicaset中创建，然后在现有的replicaset中销毁

2、更新与回滚

只有spec.template.spec.containers.image或者spec.template.metadata.labels发生变化时才会出发更新rolout操作～～ 

root@master-1:~/kubernetes/example# kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
deployment.apps "nginx-deployment" image updated

![deployment-update](/Users/lion/Documents/markdown/2018 - Kubernetes/deployment-update.gif)> gif图片太大，连接： https://pan.baidu.com/s/1cR7l4anB6FrQ27UNvJ1h0g

更新过程如图所示：

  • 首先，因为没有匹配的replicaset，创建一个新的replicaset replicaset.apps/nginx-deployment-767fdf8df6
  • 然后根据spec.strategy.rollingUpdate.maxUnavailable的值（默认25%）终止掉 replicaset.apps/nginx-deployment-85b548b894 中对应数量的pod
  • 最后根据spec.strategy.rollingUpdate.maxSurge的值（默认25%）在 replicaset.apps/nginx-deployment-767fdf8df6 中启动对应数量多pod
  • 最终， replicaset.apps/nginx-deployment-85b548b894 中的pod全部终止， replicaset.apps/nginx-deployment-767fdf8df6 中的pod达到预期值
  • replicaset的名字组成：deploymentName + podTemplate的Hash值
  • maxUnavailable：表示最大不可用pod数量，可以是整数或者百分比。这里的4个pod，25%就是一个，也就是说一次只能停止一个pod
  • maxSurge：表示可超过预期值的pod数量，可以是整数或者百分比。预期值4，25%就是可以超出一个，也就是说同时可以存在5个可用的pod 
graph TB;
    deployment.apps/nginx-deployment --> replicaset.apps/nginx-deployment-85b548b894
    deployment.apps/nginx-deployment --> replicaset.apps/nginx-deployment-767fdf8df6
    replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-5zj8c
    replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-7668l
    replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-s7rpq
    replicaset.apps/nginx-deployment-767fdf8df6 --> pod/nginx-deployment-767fdf8df6-vgm2m
### 查看更新记录 ###
root@master-1:~# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION  CHANGE-CAUSE
1         <none>
2         <none>

默认的部署方式不会记录CHANGE-CAUSE，需要在部署的时候添加 --record 选项～～

spec.revisionHistoryLimit被用来设置保留的历史记录数量，默认为2，设置为0时将不能回滚～～ 

root@master-1:~/kubernetes/example# kubectl delete -f nginx-deployment.yaml 
deployment.apps "nginx-deployment" deleted
root@master-1:~/kubernetes/example# kubectl apply -f nginx-deployment.yaml --record
deployment.apps "nginx-deployment" created
root@master-1:~/kubernetes/example# kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
deployment.apps "nginx-deployment" image updated
root@master-1:~/kubernetes/example# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION  CHANGE-CAUSE
1         kubectl apply --filename=nginx-deployment.yaml --record=true
2         kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
### 回滚到上一个版本 ###
root@master-1:~/kubernetes/example# kubectl rollout undo deployment nginx-deployment
deployment.apps "nginx-deployment" 
root@master-1:~/kubernetes/example# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION  CHANGE-CAUSE
2         kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
3         kubectl apply --filename=nginx-deployment.yaml --record=true

回滚操作的执行流程与更新一样，都是按照spec.strategy.rollingUpdate.maxUnavailable和spec.strategy.rollingUpdate.maxSurge在当前replicaset中终止pod，在 符合条件的replicaset 中启动新的pod～～ 

### 回滚到指定版本 ###
root@master-1:~/kubernetes/example# kubectl rollout undo deployment nginx-deployment --to-revision=2
deployment.apps "nginx-deployment" 
root@master-1:~/kubernetes/example# kubectl rollout history deployment nginx-deployment
deployments "nginx-deployment"
REVISION  CHANGE-CAUSE
3         kubectl apply --filename=nginx-deployment.yaml --record=true
4         kubectl set image deployment nginx-deployment nginx=192.168.101.88:5000/nginx:1.9.1
root@master-1:~/kubernetes/example# kubectl get all
NAME                                    READY     STATUS    RESTARTS   AGE
pod/nginx-deployment-767fdf8df6-56bmx   1/1       Running   0          42s
pod/nginx-deployment-767fdf8df6-bjbx4   1/1       Running   0          42s
pod/nginx-deployment-767fdf8df6-p6xll   1/1       Running   0          45s
pod/nginx-deployment-767fdf8df6-rf5tm   1/1       Running   0          44s
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   7d
NAME                               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   4         4         4            4           6m
NAME                                          DESIRED   CURRENT   READY     AGE
replicaset.apps/nginx-deployment-767fdf8df6   4         4         4         6m
replicaset.apps/nginx-deployment-85b548b894   0         0         0         6m

因为上面的更新与回滚一直在两个版本之间来回修改，deploymentName + podTemplate的Hash值并没有改变，所以没有产生新的replicaset，但是版本号回一直递增～～～

3.1、手动伸缩 

root@master-1:~/kubernetes/example# kubectl scale deployment nginx-deployment --replicas=10
deployment.extensions "nginx-deployment" scaled
root@master-1:~/kubernetes/example# kubectl get all
NAME                                    READY     STATUS    RESTARTS   AGE
pod/nginx-deployment-767fdf8df6-4zz4k   1/1       Running   0          11s
pod/nginx-deployment-767fdf8df6-56bmx   1/1       Running   0          5m
pod/nginx-deployment-767fdf8df6-9rqk6   1/1       Running   0          11s
pod/nginx-deployment-767fdf8df6-bjbx4   1/1       Running   0          5m
pod/nginx-deployment-767fdf8df6-ksrc2   1/1       Running   0          11s
pod/nginx-deployment-767fdf8df6-p6xll   1/1       Running   0          5m
pod/nginx-deployment-767fdf8df6-qcj9q   1/1       Running   0          11s
pod/nginx-deployment-767fdf8df6-rf5tm   1/1       Running   0          5m
pod/nginx-deployment-767fdf8df6-rljpn   1/1       Running   0          11s
pod/nginx-deployment-767fdf8df6-ts8sn   1/1       Running   0          11s
NAME                 TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
service/kubernetes   ClusterIP   10.96.0.1    <none>        443/TCP   7d
NAME                               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   10        10        10           10          11m
NAME                                          DESIRED   CURRENT   READY     AGE
replicaset.apps/nginx-deployment-767fdf8df6   10        10        10        11m
replicaset.apps/nginx-deployment-85b548b894   0         0         0         11m

3.2、自动伸缩（Horizontal Pod Autoscaling）

自动伸缩需要部署 Heapster 和 metrics-server ～ 

root@master-1:~/kubernetes/example# cat nginx-deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  labels:
    app: nginx
spec:
  replicas: 4
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: 192.168.101.88:5000/nginx:1.7.9
        ports:
        - containerPort: 80
        resources:
          requests:
            memory: "64Mi"
            cpu: "25m"
          limits:
            memory: "128Mi"
            cpu: "50m"
kind: Service
apiVersion: v1
metadata:
  labels:
    app: nginx-deployment
  name: nginx-deployment-service
spec:
  ports:
    - port: 80
      targetPort: 80
      name: http
  selector:
    app: nginx
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx-deployment-service-ingress
spec:
  rules:
    - host: nginx-deployment.chenlei.com
      http:
        paths:
          - backend:
              serviceName: nginx-deployment-service
              servicePort: 80
  • 使用HPA需要在部署文件中明确指定资源配置情况 resources ，最少需要配置 resources.requests.cpu
  • 为了测试，向部署文件中增加了service和ingress 
root@master-1:~/kubernetes/example# kubectl apply -f nginx-deployment.yaml 
deployment.apps "nginx-deployment" created
service "nginx-deployment-service" created
ingress.extensions "nginx-deployment-service-ingress" created
root@master-1:~/kubernetes/example# kubectl get all
NAME                                    READY     STATUS    RESTARTS   AGE
pod/nginx-deployment-5f79d4fc87-7k6cn   1/1       Running   0          9s
pod/nginx-deployment-5f79d4fc87-f5v27   1/1       Running   0          9s
pod/nginx-deployment-5f79d4fc87-ls6hz   1/1       Running   0          9s
pod/nginx-deployment-5f79d4fc87-rlwh4   1/1       Running   0          9s
NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/kubernetes                 ClusterIP   10.96.0.1       <none>        443/TCP   8d
service/nginx-deployment-service   ClusterIP   10.97.127.155   <none>        80/TCP    9s
NAME                               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   4         4         4            4           10s
NAME                                          DESIRED   CURRENT   READY     AGE
replicaset.apps/nginx-deployment-5f79d4fc87   4         4         4         9s
root@master-1:~/kubernetes/example# kubectl autoscale deployment nginx-deployment --min=1 --max=10 --cpu-percent=20
deployment.apps "nginx-deployment" autoscaled
root@master-1:~/kubernetes/example# kubectl get all
NAME                                    READY     STATUS        RESTARTS   AGE
pod/nginx-deployment-5f79d4fc87-7k6cn   1/1       Running       0          2m
pod/nginx-deployment-5f79d4fc87-f5v27   0/1       Terminating   0          2m
pod/nginx-deployment-5f79d4fc87-ls6hz   0/1       Terminating   0          2m
pod/nginx-deployment-5f79d4fc87-rlwh4   0/1       Terminating   0          2m
NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/kubernetes                 ClusterIP   10.96.0.1       <none>        443/TCP   8d
service/nginx-deployment-service   ClusterIP   10.97.127.155   <none>        80/TCP    2m
NAME                               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   1         1         1            1           2m
NAME                                          DESIRED   CURRENT   READY     AGE
replicaset.apps/nginx-deployment-5f79d4fc87   1         1         1         2m
NAME                                                   REFERENCE                     TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/nginx-deployment   Deployment/nginx-deployment   0%/20%    1         10        4          34s

HPA创建之后，会在30秒（根据controller-manager的参数配置）内根据CPU的负载情况自动调整POD数量。因为负载为零，所以自动终止了三个pod，仅保留一个～～～

kube-controller-manage的调度日志如下： 

I0504 08:25:48.431218       1 horizontal.go:512] Successful rescale of nginx-deployment, old size: 4, new size: 1, reason: All metrics below target
I0504 08:25:48.431412       1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"adb4b7a9-4f74-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1314846", FieldPath:""}): type: 'Normal' reason: 'SuccessfulRescale' New size: 1; reason: All metrics below target
I0504 08:25:48.475085       1 event.go:218] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"default", Name:"nginx-deployment", UID:"79043755-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314901", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled down replica set nginx-deployment-5f79d4fc87 to 1
I0504 08:25:48.599598       1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314904", FieldPath:""}): type: 'Normal' reason: 'SuccessfulDelete' Deleted pod: nginx-deployment-5f79d4fc87-ls6hz
I0504 08:25:48.599622       1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314904", FieldPath:""}): type: 'Normal' reason: 'SuccessfulDelete' Deleted pod: nginx-deployment-5f79d4fc87-rlwh4
I0504 08:25:48.599634       1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1314904", FieldPath:""}): type: 'Normal' reason: 'SuccessfulDelete' Deleted pod: nginx-deployment-5f79d4fc87-f5v27
W0504 08:27:12.650959       1 reflector.go:341] k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:125: watch of <nil> ended with: very short watch: k8s.io/kubernetes/pkg/controller/garbagecollector/graph_builder.go:125: Unexpected watch close - watch lasted less than a second and no items received

增加负载： 

LiondeMacBook-Pro:~ lion$ for i in {1..10000}; do curl http://nginx-deployment.chenlei.com/ > /dev/null -s; done;

这里打开四个终端分别执行，然后观察pod伸缩情况 

root@master-1:~/kubernetes/example# kubectl get all
NAME                                    READY     STATUS    RESTARTS   AGE
pod/nginx-deployment-5f79d4fc87-7k6cn   1/1       Running   0          12m
pod/nginx-deployment-5f79d4fc87-gvfpr   1/1       Running   0          4s
pod/nginx-deployment-5f79d4fc87-mzd58   1/1       Running   0          4s
NAME                               TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/kubernetes                 ClusterIP   10.96.0.1       <none>        443/TCP   8d
service/nginx-deployment-service   ClusterIP   10.97.127.155   <none>        80/TCP    12m
NAME                               DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx-deployment   3         3         3            3           12m
NAME                                          DESIRED   CURRENT   READY     AGE
replicaset.apps/nginx-deployment-5f79d4fc87   3         3         3         12m
NAME                                                   REFERENCE                     TARGETS   MINPODS   MAXPODS   REPLICAS   AGE
horizontalpodautoscaler.autoscaling/nginx-deployment   Deployment/nginx-deployment   44%/20%   1         10        1          11m

当CPU压力增加时，kubernetes自动创建了两个新的pod来分摊压力，新增pod的数量取决于CPU压力打下

kube-controller-maanage的调度日志： 

I0504 08:36:14.996758       1 horizontal.go:512] Successful rescale of nginx-deployment, old size: 1, new size: 3, reason: cpu resource utilization (percentage of request) above target
I0504 08:36:14.997180       1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"adb4b7a9-4f74-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1315533", FieldPath:""}): type: 'Normal' reason: 'SuccessfulRescale' New size: 3; reason: cpu resource utilization (percentage of request) above target
I0504 08:36:15.061094       1 event.go:218] Event(v1.ObjectReference{Kind:"Deployment", Namespace:"default", Name:"nginx-deployment", UID:"79043755-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1316131", FieldPath:""}): type: 'Normal' reason: 'ScalingReplicaSet' Scaled up replica set nginx-deployment-5f79d4fc87 to 3
I0504 08:36:15.163244       1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1316134", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: nginx-deployment-5f79d4fc87-mzd58
I0504 08:36:15.208366       1 event.go:218] Event(v1.ObjectReference{Kind:"ReplicaSet", Namespace:"default", Name:"nginx-deployment-5f79d4fc87", UID:"79075132-4f74-11e8-9807-0050568e3344", APIVersion:"extensions", ResourceVersion:"1316134", FieldPath:""}): type: 'Normal' reason: 'SuccessfulCreate' Created pod: nginx-deployment-5f79d4fc87-gvfpr

压力较低之后又会消减pod数量

3.3、自定义指标

这一块内容留在下次补充～～～～

3.4、异常记录 

root@master-1:~/kubernetes/example# kubectl get hpa
NAME               REFERENCE                     TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
nginx-deployment   Deployment/nginx-deployment   <unknown>/10%   1         10        4          37s
root@master-1:~/kubernetes/example# kubectl get hpa
NAME               REFERENCE                     TARGETS         MINPODS   MAXPODS   REPLICAS   AGE
nginx-deployment   Deployment/nginx-deployment   <unknown>/10%   1         10        4          37s
root@master-1:~/kubernetes/example# kubectl describe hpa nginx-deployment
Name:                                                  nginx-deployment
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Fri, 04 May 2018 04:46:16 -0400
Reference:                                             Deployment/nginx-deployment
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  <unknown> / 10%
Min replicas:                                          1
Max replicas:                                          10
Conditions:
  Type           Status  Reason                   Message
  ----           ------  ------                   -------
  AbleToScale    True    SucceededGetScale        the HPA controller was able to get the target's current scale
  ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count: unable to get metrics for resource cpu: no metrics returned from resource metrics API
Events:
  Type     Reason                        Age   From                       Message
  ----     ------                        ----  ----                       -------
  Warning  FailedGetResourceMetric       21s   horizontal-pod-autoscaler  unable to get metrics for resource cpu: no metrics returned from resource metrics API
  Warning  FailedComputeMetricsReplicas  21s   horizontal-pod-autoscaler  failed to get cpu utilization: unable to get metrics for resource cpu: no metrics returned from resource metrics API

kube-controller-manage日志： 

I0504 08:47:16.381435       1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"9b7a46ff-4f77-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1317451", FieldPath:""}): type: 'Warning' reason: 'FailedGetResourceMetric' missing request for cpu on container nginx in pod default/nginx-deployment-85b548b894-2lzg8
E0504 08:47:16.391281       1 horizontal.go:189] failed to compute desired number of replicas based on listed metrics for Deployment/default/nginx-deployment: failed to get cpu utilization: missing request for cpu on container nginx in pod default/nginx-deployment-85b548b894-2lzg8
I0504 08:47:16.391308       1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"9b7a46ff-4f77-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1317451", FieldPath:""}): type: 'Warning' reason: 'FailedComputeMetricsReplicas' failed to get cpu utilization: missing request for cpu on container nginx in pod default/nginx-deployment-85b548b894-2lzg8

定义pod的时候，没有设置resources，HPA取不到CPU当前值 

        resources:
          requests:
            memory: "64Mi"
            cpu: "25m"
          limits:
            memory: "128Mi"
            cpu: "50m"
root@master-1:~/kubernetes/example# kubectl autoscale deployment nginx-deployment --min=1 --max=10 --cpu-percent=10
deployment.apps "nginx-deployment" autoscaled
root@master-1:~/kubernetes/example# kubectl describe hpa nginx-deployment
Name:                                                  nginx-deployment
Namespace:                                             default
Labels:                                                <none>
Annotations:                                           <none>
CreationTimestamp:                                     Fri, 04 May 2018 04:53:20 -0400
Reference:                                             Deployment/nginx-deployment
Metrics:                                               ( current / target )
  resource cpu on pods  (as a percentage of request):  <unknown> / 10%
Min replicas:                                          1
Max replicas:                                          10
Conditions:
  Type           Status  Reason                   Message
  ----           ------  ------                   -------
  AbleToScale    True    SucceededGetScale        the HPA controller was able to get the target's current scale
  ScalingActive  False   FailedGetResourceMetric  the HPA was unable to compute the replica count: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
Events:
  Type     Reason                        Age   From                       Message
  ----     ------                        ----  ----                       -------
  Warning  FailedGetResourceMetric       27s   horizontal-pod-autoscaler  unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
  Warning  FailedComputeMetricsReplicas  27s   horizontal-pod-autoscaler  failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
root@master-1:~/kubernetes/example#

kube-controller-manage日志： 

E0504 08:54:20.852828       1 horizontal.go:189] failed to compute desired number of replicas based on listed metrics for Deployment/default/nginx-deployment: failed to get cpu utilization: unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)
I0504 08:54:20.852958       1 event.go:218] Event(v1.ObjectReference{Kind:"HorizontalPodAutoscaler", Namespace:"default", Name:"nginx-deployment", UID:"987a4389-4f78-11e8-9807-0050568e3344", APIVersion:"autoscaling/v2beta1", ResourceVersion:"1318418", FieldPath:""}): type: 'Warning' reason: 'FailedGetResourceMetric' unable to get metrics for resource cpu: unable to fetch metrics from resource metrics API: the server could not find the requested resource (get pods.metrics.k8s.io)

没有部署 metrics-server 服务，部署后任然无法解决的可参考GitHub上的 这个问题 。

4、参考资料

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/

pod的升级 回滚 若pod是通过Deployment创建的,我们可以通过修改对应deployment的spec.template 或镜像名称，并应用到deployment上,系统便可以完成deployment的 更新 。如果 更新 过程中 出现了错误，则还可以通过 回滚 操作恢复pod。 Deployment的升级 以nginx deployment为例: apiVersion: apps/v1 kind: ...   Warning Failed Get Resource Metric 57m (x2401 over 5d)  horizontal-pod-autoscaler  unable to get metric s for resource cpu: unable to fetch metric s from resource metr... Kubernetes 集群版本升级是为了获得最新的功能、增强的安全性和性能改进。然而，升级过程需要谨慎进行，特别是在生产环境中。通常， Kubernetes 集群的版本升级应遵循逐步升级的策略，不建议直接跳过多个版本。 今天我正在专心致志的敲go代码的bug，突然同事问到我们现在的服务数量太多了，同一个服务的实例pod数可能达到了十几个，能不能放在一个pod中，跑多个container的形式运行。当时我想了一下，就回答了：放在一个pod里边，这些container管理起来可能比较复杂一些，pod挂掉，所有的container都卒了，粒度太大。然后正准备继续写bug的时候，突然想到同事这么问的目的可能是性能优化，自然而然的就想到 hpa 了，联想到istio和knative的 hpa 问题还没解决，卒，好吧，来解决问题吧。 k8s 平台实现 HPA 过程中问题排查背景问题描述问题分析解决问题 最近在着手学习 HPA 的过程， HPA 需要 metric -service进行采集数据。在实现过程中，遇到了一些问题，进行解决。 查看当前 HPA 的情况。 (base) root@gz k8s -master1:/home/cyye# kubectl get hpa -A NAMESPACE NAME REFERENCE TARGETS kubectl uncordon node_name ================================================drain 驱逐节点 首先，驱逐node上的pod，其他节点重新创建 Warning Failed Get Resource Metric 12s (x41 over 20m) horizontal-pod-autoscaler unable to get metric s for resource cpu: unable to fetch metric s from API: the server could not find the requested r... 新钛云服已累计为您分享772篇技术干货介绍Kube-downscaler 是一款开源工具，允许用户定义 Kubernetes 中 pod 资源自动缩减的时间。这有助于通过减少非高峰时段的资源使用量来降低基础设施成本。在本文中，我们将详细介绍 kube-downscaler 的功能、安装和配置，以及它的用例和未来前景。kube-downscaler的特点Kube-downscaler 是一款基于调度... 前言弹性 伸缩 介绍 Metric s Server聚合 API安装 HPA （弹性 伸缩 实验）缩放间隙基于内存弹性 伸缩 弹性 伸缩 介绍 在使用中我们使用一条 kubectl scale 命令可以来实现 Pod 的扩缩容功能，但是这个毕竟是完全手动操作的，要应对线上的各种复杂情况，我们需要能够做到自动化去感知业务，来自动进行扩缩容。为此， Kubernetes 也为我们提供了这样的一个资源对象：Horizontal Pod Autoscaling（Pod 水平自动 伸缩 ），简称 HPA ， HPA 通过监控分析一些控制器控制的所有
 
推荐文章
飞奔的企鹅  ·  0026738: WebP attachments are not forced to display inline - MantisBT
1 月前
闷骚的黑框眼镜  ·  References | Artificial Intelligence in Health Professions Education: Proceedings of a Workshop | Th
2 月前
正直的手电筒  ·  配置 Rsbuild - Rsbuild
2 月前
文质彬彬的红豆  ·  HTRA1 x HTRA2 - Drugs, Indications, Patents - Synapse
7 月前
有腹肌的小熊猫  ·  “欢乐”迪士尼崩了:暴跌2000亿 没有童话只有翻包？_手机新浪网
1 年前
Link管理   ·   51好读   ·   Sov5搜索   ·   小百科
link管理 - 链接快照平台