https Ingress (ssl 인증서 적용)

refer to

---

https://kubernetes.github.io/ingress-nginx/user-guide/tls/
https://github.com/kubernetes/ingress-nginx/issues/2173
https://kubernetes.github.io/ingress-nginx/examples/multi-tls/

사전 환경

---

아래와 같이 https를 수신하는 두 서비스가 있다. 이를 Ingress tls로 분기할 예정이다.

[root@wyj05_deploy_0 ingress]# kubectl get svc -n https 
NAME          TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
https-test1   NodePort   10.233.44.245   <none>        8443:30004/TCP   169m
https-test2   NodePort   10.233.54.192   <none>        8443:30054/TCP   169m

[root@wyj05_deploy_0 ingress]# curl -k https://10.95.90.20:30004
test1 https server

[root@wyj05_deploy_0 ingress]# curl -k https://10.95.90.20:30054
test2 https server

인증서 생성

---

[root@master1 ~]# openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ingresstls.key -out ingresstls.cert

인증서 기반으로 secret 생성

[root@master1 ~]# kubectl create secret tls httpstls --key ingresstls.key --cert ingresstls.cert --namespace https

Ingress 생성

---

https://kubernetes.github.io/ingress-nginx/user-guide/nginx-configuration/annotations/

계속 400 ERROR가 발생하여 아래 옵션을 추가하여 조치했다.
내가 구성한 서버는 http > https 리다이렉션 설정이 돼있지 않았기 때문에 Nginx가 backend에 있는 Service들과 HTTPS로 바로 통신하도록 설정해 주어야 한다. 안그러면 HTTP 프로토콜로 통신하기에 제대로 동작을 안한다.

원문

Backend Protocol
Using backend-protocol annotations is possible to indicate how NGINX should communicate with the backend service. (Replaces secure-backends in older versions) Valid Values: HTTP, HTTPS, GRPC, GRPCS, AJP and FCGI

Default ) HTTP

---

host를 명시하지 않았을 때

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
#    nginx.ingress.kubernetes.io/secure-backends: "true"
#    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
#    nginx.org/ssl-services: https-test1,https-test2
  name: https-ingress
  namespace: https
spec:
  tls:
  - secretName: httpstls
  rules:
  - http:
      paths:
      - backend:
          serviceName: https-test1
          servicePort: 8443
        path: /https1
      - backend:
          serviceName: https-test2
          servicePort: 8443
        path: /https2

host를 명시 했을 때

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
#    nginx.ingress.kubernetes.io/secure-backends: "true"
#    nginx.ingress.kubernetes.io/ssl-passthrough: "true"
#    nginx.org/ssl-services: https-test1,https-test2
  name: https-ingress
  namespace: https
spec:
  tls:
  - hosts:
    - ingress.example.com
    secretName: httpstls2
  rules:
  - host: ingress.example.com
    http:
      paths:
      - backend:
          serviceName: https-test1
          servicePort: 8443
        path: /https1
      - backend:
          serviceName: https-test2
          servicePort: 8443
        path: /https2
status:
  loadBalancer:
    ingress:
    - ip: 10.95.90.20
    - ip: 10.95.90.21

Multi Domain으로 설정 했을 때

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
  name: https-ingress
  namespace: https
spec:
  tls:
  - secretName: httpstls
  rules:
  - host: foo.bar.com
    http:
      paths:
      - backend:
          serviceName: https-test1
          servicePort: 8443
        path: /https1
  - host: foo.bar2.com
    http:
      paths:
      - backend:
          serviceName: https-test2
          servicePort: 8443
        path: /https2
status:
  loadBalancer:
    ingress:
    - ip: 10.95.90.20
    - ip: 10.95.90.21

테스트

[root@master1 ~]# curl -k https://foo.bar.com:30443/https1
test1 https server
[root@master1 ~]# curl -k https://foo.bar2.com:30443/https1
test2 https server

Multi Domain으로 host 지정 했을 때 (refer 참조)

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: foo-tls
  namespace: default
spec:
  tls:
  - hosts:
    - foo.bar.com
    # This secret must exist beforehand
    # The cert must also contain the subj-name foo.bar.com
    # https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/PREREQUISITES.md#tls-certificates
    secretName: foobar
  - hosts:
    - bar.baz.com
    # This secret must exist beforehand
    # The cert must also contain the subj-name bar.baz.com
    # https://github.com/kubernetes/ingress-nginx/blob/master/docs/examples/PREREQUISITES.md#tls-certificates
    secretName: barbaz
  rules:
  - host: foo.bar.com
    http:
      paths:
      - backend:
          serviceName: http-svc
          servicePort: 80
        path: /
  - host: bar.baz.com
    http:
      paths:
      - backend:
          serviceName: nginx
          servicePort: 80
        path: /