How we generate and renew SSL certs for arbitrary custom domains using LetsEncrypt!

Sandeep Panda

·

·

6.2K views

Last month we decided to do something interesting and allow Hashnode users to publish articles under a free hashnode.dev subdomain or any custom domain of their choice. It required a bit of work on our part, but the most challenging task was enabling SSL for arbitrary custom domains. Thankfully, we have LetsEncrypt which lets you generate SSL for any domain for free. However, it's still a challenge to generate and serve the certs on demand using nginx. While building hashnode.dev we faced that challenge. In the spirit of blogging, I am going to share the steps we took to overcome this.

For those of you who are unaware, hashnode.dev lets developers start a personal blog for free. All you need is a hashnode.com account. Fun fact: this blog is powered by hashnode.dev. You can find more details here.

So, what options did we really have? Let me outline those here.

Option 1

Our backend is a Node.js server. So, I thought of using something like node-greenlock to generate and serve SSL certs. But that would require us to expose our Node.js server on port 80/443 and we will lose the benefits of nignx. I didn't want that. So, I kept digging more.

Option 2

I kept researching for couple of more days, but every solution was complex and demanded time. Since we were done with all the aspects of hashnode.dev (except SSL part), I decided not to delay the launch of alpha preview. Then I had an idea -- Why not create a Amazon Cloudfront distribution temporarily and use it as a reverse proxy to <username>.hashnode.dev? AWS offers free SSL certificates as long as you want to use them with CF distributions. This was certainly not a long term solution because:

  • Number of cloudfront distributions/certs are limited per account. You have to request AWS to increase the limit periodically.

  • Cost of the CDN usage.

  • The process was complex. Our users had to validate ownership of their domain first which took several hours in some cases. Then they had to update their CNAME to CF distribution URL. But they couldn't use it at root level of their domain because of CNAME flattening issue. Only those who used a registrar that supported CNAME flattening (e.g. Cloudflare) were able add the record at root.

Clearly, I didn't want the above solution. I wanted something that just works with minimal efforts from customers. I anyway went ahead with this solution and decided to keep looking for a better solution.

Option 3

This is the best part. I came across OpenResty which is apparently one of the most used web servers. I felt like I was living under a rock. As I researched further I realized that OpenResty bundles nginx core and lets you embed logic into your server using Lua. 😍

To my surprise I also discovered a module called lua-resty-auto-ssl which generates and renews SSL certs automatically inside OpenResty. I was overjoyed. In the coming few days my teammate Aravind did a small PoC and confirmed that it's indeed working. So, I decided to give this a try.

Step 1

The first step was obviously installing OpenResty. I followed this simple guide from DigitalOcean and successfully installed OpenResty.

I did only one modification though which is enabling http2 during the configuration:

./configure -j2 --with-pcre-jit --with-ipv6 --with-http_v2_module

Step 2

Installed LuaRocks package manager so that I could install lua-resty-auto-ssl package. I am on latest Ubuntu.

sudo apt-get install unzip
wget http://luarocks.org/releases/luarocks-2.0.13.tar.gz
tar -xzvf luarocks-2.0.13.tar.gz
cd luarocks-2.0.13/
./configure --prefix=/usr/local/openresty/luajit \
    --with-lua=/usr/local/openresty/luajit/ \
    --lua-suffix=jit \
    --with-lua-include=/usr/local/openresty/luajit/include/luajit-2.1
make
sudo make install

Then I installed lua-resty-auto-ssl and set up the directory:

sudo luarocks install lua-resty-auto-ssl
sudo mkdir /etc/resty-auto-ssl
sudo chown sandeep /etc/resty-auto-ssl

Step 3

Then I added configuration for our web server. It goes into /usr/local/openresty/nginx/conf/nginx.conf:

user sandeep www;
events {
  worker_connections 1024;
}

http {
  lua_shared_dict auto_ssl 1m;
  lua_shared_dict auto_ssl_settings 64k;
  resolver 8.8.8.8 ipv6=off;

  init_by_lua_block {
    auto_ssl = (require "resty.auto-ssl").new()
    auto_ssl:set("allow_domain", function(domain)
      return true // Allow all the domains
    end)
    auto_ssl:init()
  }

  init_worker_by_lua_block {
    auto_ssl:init_worker()
  }

  server {
    listen 443 ssl http2;
    ssl_certificate_by_lua_block {
      auto_ssl:ssl_certificate()
    }
    ssl_certificate /etc/letsencrypt/live/hashnode.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/hashnode.com/privkey.pem;

    location / {
      proxy_pass http://localhost:3000;
      # Other configs  
    }
  }

  server {
    listen 80;
    location /.well-known/acme-challenge/ {
      content_by_lua_block {
        auto_ssl:challenge_server()
      }
    }
  }

  server {
    listen 127.0.0.1:8999;
    client_body_buffer_size 128k;
    client_max_body_size 128k;

    location / {
      content_by_lua_block {
        auto_ssl:hook_server()
      }
    }
  }
}

SSL cert is provided by ssl_certificate_by_lua_block. We still need ssl_certificate and ssl_certificate_key directives to serve SSL certs for domains *.hashnode.dev and *.hashnode.com. We exclude these two cases and non-whitelisted domains in init_by_lua_block as following:

init_by_lua_block {
    auto_ssl = (require "resty.auto-ssl").new()
    auto_ssl:set("allow_domain", function(domain)

       -- we don't want to generate SSL for hashnode.com
       if domain == "hashnode.com" then
          return false
       end

       -- we don't want to generate SSL for *.hashnode.dev as well
       if string.find(domain, ".hashnode.dev") == nil then 
         local http = require("resty.http")
         local httpc = http.new()

         httpc:set_timeout(3000)

         local uri = "<API_ON_HASHNODE>"

         local res, err = httpc:request_uri(uri, {
           ssl_verify = false,
           method = "GET"
         })

         if not res then
           return false
         end

         if res.status == 200 then
           return true
         end

         if res.status == 404 then
           return false
         end
       else
         return false
       end
    end)
   --auto_ssl:set("ca", "https://acme-staging.api.letsencrypt.org/directory")
   auto_ssl:init()
  }

This simple solution pings our endpoint to check if a domain is whitelisted for SSL. If not, it just skips the process.

Make sure to use auto_ssl:set("ca", "https://acme-staging.api.letsencrypt.org/directory") > during development otherwise your account may hit rate limiting issues.

Step 4

With a few other configurations, I could make this work successfully. Now we simply ask our users to add an A record that points to our IP and everything just works. If SSL cert doesn't exist for a domain, it's generated and served automatically when you request the URL for the first time. Naturally, there is a latency of ~10s for the very first request. Next time onwards it's blazing fast since the certs are cached. Since LE certs are valid for 90 days, renewals happen automatically if the expiry date is < 30 days.

This is the message I sent to our alpha testers after I deployed the change:

Screenshot 2019-03-21 at 1.01.49 AM.png


I hope this guide will help you if you are going to create a multi-tenant SaaS app that relies on SSL. Special thanks to LetsEncrypt, OpenResty and auto-ssl package. It wouldn't have been possible without these tools/services.

If you are a developer and are looking to start a personal blog, please show your interest by visiting hashnode.dev.

10 comments
Add a comment