Publicly expose a website via HTTPS with Ingress Nginx, Cert-manager, Let's Encrypt and Kubernetes
by Alex Arica

In this blog post we are going to publicly expose a website via HTTPS with Ingress Nginx controller in a bare-metal Kubernetes cluster. We are going to explain how to integrate Let’s Encrypt with Kubernetes so that SSL certificates are issued and renewed automatically for our domain names.

We assume that you read the blog post about how to expose a website via HTTP using Ingress Nginx. This current page is a continuation of that blog post.

Let's Encrypt

Let’s Encrypt is a certificate authority (CA) which provides HTTPS certificates for domain names without the need to manually create an account with them. The process of requesting certificates and renewing them is fully automated and free.

Install Cert-manager controller

We are going to use Cert-manager which is a controller handling the creating and renewal of SSL certificates in Kubernetes. It integrates well with the Ingress Nginx controller.

Installation command:

kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/[version]/cert-manager.yaml
                    

Replace the variable "[version]" by the latest version available in the official Cert-manager's GitHub page.

Once the installation command above is run, Cert-manager controller creates a namespace "cert-manager" where it installs its custom resources. Let's wait until that all cert-manager pods are running:

kubectl get pods -n cert-manager -w
                    

Create a certificate issuer

Once Cert-manager is ready, we are going to create a "ClusterIssuer" using Let's Encrypt to issue SSL certificates. The idea is to create this set-up once for the entire cluster. And we will be able to reference that resource from any namespace for websites which meed a certificate for their domain names. That's the reason why "ClusterIssuer" resource is used rather than an "Issuer" resource.

A "ClusterIssuer" allows to register to a certificate authority like Let's Encrypt. Then Let's Encrypt can issue certificates. Each time a resource like an Ingress Nginx controller wants to create a SSL certificate for a domain name, it will delegate this operation to a "ClusterIssuer" to request the SSL certificate.

We are going to create a staging issuer so that we can test the process is working. I highly recommend creating a staging issuer before using a prod one, because it will allow to test our installation.

In a new file:

vi lets-encrypt-staging-issuer.yaml
                    

Add:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: lets-encrypt-staging
spec:
  acme:
      email: contact@reactive.tech.io
      server: https://acme-staging-v02.api.letsencrypt.org/directory
      privateKeySecretRef:
         name: lets-encrypt-staging
    solvers:
        - http01:
                 ingress:
                      class: nginx
                    

In the yaml above, we configured the private key to be in the secret "lets-encrypt-staging". Cert-Manager will create it with the key provided by Let's Encrypt. Please replace "email: contact@reactive.tech.io" by your email address. Let's encrypt will us that email to notify about certificate expiry dates, etc...

Deploy it:

kubectl apply -f lets-encrypt-staging-issuer.yaml
                    

Check the issuer is ready:

kubectl get ClusterIssuer -n cert-manager -w
                    

For example:

NAME                   READY   AGE
lets-encrypt-staging   True    5s
                    

In the yaml above, we configured the private key to be in the secret "lets-encrypt-staging". Let's check that Cert-Manager created it:

kubectl get secret lets-encrypt-staging -n cert-manager
                    

Once our ClusterIssuer is ready, we can configure our existing Ingress configuration so that the domain name "www.reactive-tech.io" has an SSL certificate.

Issue a staging certificate for "www.reactive-tech.io"

We are going to create a staging SSL certificate for the domain "www.reactive-tech.io" by referencing the "ClusterIssuer" that we have created.

Modify the existing ingress resource that we have created in the previous blog post, by updating the file:

vi reactive-tech-website-ingress.yaml
                    

And add the configuration marked in bold:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: reactive-tech-website
  namespace: static-websites
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
    cert-manager.io/issuer-kind: "ClusterIssuer"
    cert-manager.io/issuer: "lets-encrypt-staging"
    nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
  tls:
    - hosts:
        - www.reactive-tech.io
      secretName: reactive-tech-website-cert-staging
  rules:
    - host: "www.reactive-tech.io"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: reactive-tech-website
                port:
                  number: 80

                    

The new lines marked in bold above, are instructing Ingress controller to delegate the process of issuing a SSL certificate to a resource of type "ClusterIssuer" with name "lets-encrypt-staging". And the line "nginx.ingress.kubernetes.io/ssl-redirect: false" instructs to NOT force a redirection from HTTP to HTTPS. Otherwise, all requests for the domain www.reactive-tech-io will go through HTTPS (port 443) even if the initial call was via HTTP (port 80). We will enable this option later when we will issue a production SSL certificate.

The configuration "secretName: reactive-tech-website-cert-staging" instruct Cert-Manager to create a secret and store the SSL certificate keys in it. The secret's namespace would be the same as the Ingress configuration. In this case the namespace used is "static-websites".

Apply the new configurations:

kubectl apply -f reactive-tech-website-ingress.yaml
                    

Let's check the secret "reactive-tech-website-cert-staging" was created by Cert-manager:

kubectl get secret reactive-tech-website-cert-staging -n static-websites
                    

Next, we are going to check that the staging certificate was successfully issued by Let's Encrypt:

kubectl get clusterissuer,certificate,certificaterequest,order,challenge -n static-websites
                    

All listed resources should have their column "READY" set to "true". For example:

NAME                                                 READY   AGE
clusterissuer.cert-manager.io/lets-encrypt-staging   True    47s

NAME                                                             READY   SECRET                               AGE
certificate.cert-manager.io/reactive-tech-website-cert-staging   True    reactive-tech-website-cert-staging   46s

NAME                                                                          READY   AGE
certificaterequest.cert-manager.io/reactive-tech-website-cert-staging-f7h2r   True    46s

NAME                                                                            STATE   AGE
order.acme.cert-manager.io/reactive-tech-website-cert-staging-f7h2r-815108175   valid   46s
                    

Once the resources above all are ready, it means that Let's Encrypt assigned a staging SSL certificate for the domain "www.reactive-tech.io". The way it works is the Cert-Manager controller creates an endpoint using Ingress, such as: "www.reactive-tech.io/.well-known/acme-challenge/[token]".
The [token] variable is a long string such as "w5tYb92Vt0GAHm1i5a2FXSk2VHYbmHOFTzTItV2U5T". Then Let's Encrypt calls that endpoint to validate the domain name and then assigns an SSL certificate.

Troubleshooting

If the resources above are NOT ready, that means Let's Encrypt could not issue an SSL certificate. And this could happen if you are using an external load balancer to route the traffics to Ingress Nginx controller. Please make sure to configure your load balancer using TCP option rather than HTTP and HTTPS when defining the routing rules. Otherwise, if HTTP and HTTPS options are used, the load balancer is likely to rewrite the routing rules preventing Let's Encrypt to validate your domain name using the endpoint "www.reactive-tech.io/.well-known/acme-challenge/[token]".

Cert-Manager's doc has a page about troubleshooting.

Call the website via HTTPS using the staging cert

The last step is to call the website "www.reactive-tech.io" via HTTPS. The browser should warn you that the SSL certificate is not signed by a certificate authority. This is normal since it is a staging certificate. Accept the risk and you should be able to access to the website.

The next step is to create a valid production SSL certificate which will be accepted as valid by the browsers.

Issue a production certificate for "www.reactive-tech.io"

First, we need to crete a production ClusterIssuer using Let's Encrypt. In a new file:

vi lets-encrypt-prod-issuer.yaml
                    

Add the configurations below. We marked in bold the differences with the staging ClusterIssuer:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: lets-encrypt-prod
spec:
  acme:
      email: contact@reactive.tech.io
      server: https://acme-v02.api.letsencrypt.org/directory
      privateKeySecretRef:
         name: lets-encrypt-prod
      solvers:
         - http01:
            ingress:
               class: nginx
                    

Deploy it:

kubectl apply -f lets-encrypt-prod-issuer.yaml
                    

Check the issuer is ready:

kubectl get ClusterIssuer -n cert-manager -w
                    

For example:

NAME                READY   AGE
lets-encrypt-prod   True    5s
                    

Once the ClusterIssuer is ready, we can generate a production certificate by modifying the existing Ingress configuration. Open the existing file:

vi reactive-tech-website-ingress.yaml
                    

And modify using the configurations marked in bold:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: reactive-tech-website
  namespace: static-websites
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/from-to-www-redirect: "true"
    cert-manager.io/issuer-kind: "ClusterIssuer"
    cert-manager.io/issuer: "lets-encrypt-prod"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
    - hosts:
        - www.reactive-tech.io
      secretName: reactive-tech-website-cert-prod
  rules:
    - host: "www.reactive-tech.io"
      http:
        paths:
          - pathType: Prefix
            path: "/"
            backend:
              service:
                name: reactive-tech-website
                port:
                  number: 80
                    

Apply the changes:

kubectl apply -f reactive-tech-website-ingress.yaml
                    

Let's check the secret "reactive-tech-website-cert-prod" was created by Cert-manager:

kubectl get secret reactive-tech-website-cert-prod -n static-websites
                    

Next, we are going to check that the production certificate was successfully issued by Let's Encrypt:

kubectl get clusterissuer,certificate,certificaterequest,order,challenge -n static-websites
                    

All listed resources should have their column "READY" set to "true". For example:

NAME                                                READY   AGE
clusterissuer.cert-manager.io/lets-encrypt-prod     True    47s

NAME                                                             READY   SECRET                               AGE
certificate.cert-manager.io/reactive-tech-website-cert-prod      True    reactive-tech-website-cert-staging   46s

NAME                                                                       READY   AGE
certificaterequest.cert-manager.io/reactive-tech-website-cert-prod-f7h2r   True    46s

NAME                                                                         STATE   AGE
order.acme.cert-manager.io/reactive-tech-website-cert-prod-f7h2r-815108175   valid   46s
                    

Call the website via HTTPS using the production cert

The last step is to call the website "www.reactive-tech.io" via HTTPS. The browser should accept the SSL certificate without any warning since it was signed by the certificate authority Let's Encrypt. We should be able to access to the website via HTTPS.

Additional readings

Cert-Manager's doc about How to install the controller in Kubernetes.

Cert-Manager's doc about How to install it with Ingress Nginx and Let's Encrypt.