Support TLS termination with AWS NLB · Issue #73297 · kubernetes/kubernetes

link管理

链接快照平台

输入网页链接，自动生成快照
标签化管理网页链接

相关文章推荐

强悍的红烧肉 · 借助云端解决方案加快软件开发 | Qt· 2 天前 ·

发呆的小蝌蚪 · DeleteObjects与 AWS ...· 昨天 ·

憨厚的鼠标 · select标签添加onclick()事件的 ...· 1 年前 ·

想表白的水煮鱼 · perl ...· 2 年前 ·

怕老婆的小蝌蚪 · 在使用uniapp和vue3.0开发支付宝小 ...· 2 年前 ·

瘦瘦的青蛙 · [译] 在 Node.js 和 C++ ...· 2 年前 ·

粗眉毛的牛排 · 使用 useRef 构建 React ...· 2 年前 ·

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What would you like to be added :

AWS has announced TLS termination for network load balancers: https://aws.amazon.com/blogs/aws/new-tls-termination-for-network-load-balancers/

It would be amazing if Kuberentes could support this! The annotations are all defined, we would need to hook up the cloud provisioner.

service.beta.kubernetes.io/aws-load-balancer-type: nlb
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: `arn:...`

Why is this needed :

ACM can only be used with AWS Load balancers, so supporting TLS with NLBs means we will be able to leverage ACM and support http2 and WebSockets without using alb-ingress. This would resolve my need for ALBs in Kubernetes: #30518 . For example it would make running nginx-ingress with http2 and WebSocket support possible with ACM.

All reactions

In addition to certificate ARN on the listener, we need ssl security policy. Also TLS protocol should be applied to the target group. Without this last part, API GW (using PrivateLink) will throw errors.

Perhaps a more complete implementation would look like this:

apiVersion: v1
kind: Service
metadata:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn: ...
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-1-2017-01
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: TLS
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"

All reactions

Hi all :)

I am using this at the moment and its working as expected, except that the annotation service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm is not setting the certificate for the NLB (works for the classic as expected). Just wondering if this is a know issue or am I not configuring it correctly?

apiVersion: v1
kind: Service
metadata:
  name: guestbook
  labels:
    app: guestbook
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <my acm arn>
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
spec:
  ports:
    - name: http
      port: 80
      targetPort: guestbook
    - name: https
      port: 443
      targetPort: guestbook
  selector:
    app: guestbook
  type: LoadBalancer
Hi guys :)
I am using this at the moment and its working as expected, except that the annotation service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm is not setting the certificate for the NLB (works for the classic as expected). Just wondering if this is a know issue or am I not configuring it correctly?

apiVersion: v1
kind: Service
metadata:
  name: guestbook
  labels:
    app: guestbook
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: <my acm arn>
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: https
spec:
  ports:
    - name: http
      port: 80
      targetPort: guestbook
    - name: https
      port: 443
      targetPort: guestbook
  selector:
    app: guestbook
  type: LoadBalancer
Is that creating a listener of port type TLS
          On version 1.15 I'm still getting * spec.ports[1].protocol: Unsupported value: "TLS": supported values: "SCTP", "TCP", "UDP"
ports:
    - name: https
      port: 443
      protocol: TLS
      targetPort: https
          The current values for the annotation are thus (according to the k8s documentation):

service.beta.kubernetes.io/aws-load-balancer-backend-protocol (http|https|ssl|tcp)
I recall in the back-channel conversations for this feature that ssl was going to be used in the annotation, even though the NLB itself uses the value TLS. I recall that there was no desire to create a new value for the annotation and that the specific implementation of NLB would be obscured - i.e. implemented as TLS on the NLB listener even though ssl is used as the k8s annotation value.
Perhaps you can conduct an experiment by setting the annotation to ssl in k8s and checking the NLB to see what the backend protocol is used for the listeners?
          @tettusud The ssl-ports annotation needs to be 443:
annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "<arn>"
          I've tried to have this feature working using a NLB without success. However it does work using a Classic Load Balancer
I am using a EKS cluster, K8s version 1.14
I followed the following steps : AWS doc
As soon as I add the annotation : service.beta.kubernetes.io/aws-load-balancer-type: "nlb"

The NLB listener for port 443 do not get the certificate attached.
Anyone had more luck ?
          @denismoret wait for EKS to release version 1.15
After that happens, upgrade it and it will. For now, we're stuck with manual change in AWS console or some workaround bash scripts :)
PS: I'm in the same situation!
          Just upgraded to 1.15, the following works correctly:
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "${AWS_SSL_CERT}"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-connection-idle-timeout: "3600"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
          @mercantiandrea Try with
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "ssl"
That fixed it for me.

From https://kubernetes.io/docs/concepts/services-networking/service/#ssl-support-on-aws: "TCP and SSL selects layer 4 proxying: the ELB forwards traffic without modifying the headers."
          Running on EKS 1.17 , and somehow i am unable to register a 443 TLS listener for NLB.  Seems like it's defaulted to port 80 TCP listener.
Anybody facing the same issue?
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert:  "${cert_arn}"
    service.beta.kubernetes.io/aws-load-balancer-type: nlb
    service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-ssl-negotiation-policy: ELBSecurityPolicy-TLS-1-1-2017-01
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
          @robincher

This is what I have on EKS 1.17 (recent install)
    service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "${AWS_SSL_CERT}"
    service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
    service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
    service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
Note: Quote the values
          Hi @stevelacy ,thanks i tried that and still getting only the default Listener on port 80.
I suspect that my EKS cluster are having issue reading the acm.
May i know , if you are using AWS certificate manager, do you attached any specific IAM policies to EKS the node or cluster IAM role?
Updated : Fixed. Somehow i just reapply the manifest and it works... thanks all!
          Hey everyone, I was trying to deploy a NLB terminating TLS on it and for myself, I could make it work:
By understanding that there is 2 different usages of the service.beta.kubernetes.io/aws-load-balancer-ssl-ports. Either you can use service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443" if the Port is 443 (it could be any port on which you want to terminate the TLS) OR service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https" BUT https represents the target name so in the below example, I should put service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "https-port" and not https.
Port:                        https-port  443/TCP
TargetPort:                  8200/TCP
NodePort:                    https-port  31264/TCP
Endpoints:                   10.101.0.208:8200
Also I noticed if you start to create the service with the bad config on service.beta.kubernetes.io/aws-load-balancer-ssl-ports, if you modify the service after, it doesn't update correctly the NLB, so you need to recreate the service
At the end end I can have:
annotations:
        service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
        service.beta.kubernetes.io/aws-load-balancer-cross-zone-load-balancing-enabled: "true"
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443" (or the name of the target, it's as you want)
        service.beta.kubernetes.io/aws-load-balancer-ssl-cert: arn:aws:acm:[Region]:[ID]:certificate/[CERT]
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: http
Hope that can help !
For example it would make running nginx-ingress with http2 and WebSocket support possible with ACM.
True but it breaks nginx support for HTTP/1.1, which is unacceptable.
My NLB terminates TLS and sends the request to nginx-ingress on port 8000, which is configured as listen 8000 proxy_protocol http2. HTTP/2 requests work, but HTTP/1.1 requests don't.
curl error: curl: (1) Received HTTP/0.9 when not allowed

node error: Error: Parse Error: Expected HTTP/
Are there any workarounds for this issue?
What would you like to be added:
AWS has announced TLS termination for network load balancers: https://aws.amazon.com/blogs/aws/new-tls-termination-for-network-load-balancers/
It would be amazing if Kuberentes could support this! The annotations are all defined, we would need to hook up the cloud provisioner.
service.beta.kubernetes.io/aws-load-balancer-type: nlb
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: `arn:...`
Why is this needed:
ACM can only be used with AWS Load balancers, so supporting TLS with NLBs means we will be able to leverage ACM and support http2 and WebSockets without using alb-ingress. This would resolve my need for ALBs in Kubernetes: #30518. For example it would make running nginx-ingress with http2 and WebSocket support possible with ACM.
this is NOT true and does not work, these 2 annotations alone do not work

why is everyone giving thumbs up to this when it does NOT work?