In previous posts I pointed out why TLS is important, how to configure Apache to score an A+ and how to tune HTTP headers. All this is dependent on getting an SSL cert.
Some jargon explained
Before we delve into a “how to”, some basic jargon should be explained:
SSL/TLS
TLS (“Transport Layer Security”) is the successor to SSL (“Secure Socket Layer”). SSL was created by Netscape in the mid 90s (I remember installing “Netscape Commerce Server” in 1996). When you hear people talk about SSL and TLS they pretty much use the terms interchangeably. You can think of it as a progression SSL2, SSL3, TLS1.0, TLS1.1, TLS1.2 with each version better than the previous.
SSL/TLS is what makes the connection to the server secure.
SSL Certificate
This is a file you place on your server and this basically says “I am server x.y.z”. The server for this website says “I am server www.sweharris.org”. There are other things this certificate can say (”I can also be called sweharris.org”). Different certificates can also be used for other reasons (e.g. signing code), but I won’t go into that here.
The SSL certificate is signed by a trusted party (the “Certificate Authority” - CA), so when your browser goes to a web site it can verify the certificate is valid. Your browser has a list of CAs that the browser creator trusts. If I tried to create a certificate for “google.com” then no CA would sign it, and your browser would say “Nope!”
Domain Validation
In order to get a certificate signed the CA needs to know that you are
who you claim you are. “Domain Validation” (DV) is the weakest and easiest
method of validating ownership. This may be done by sending an email to
a standard address (eg postmaster or hostmaster). It might be done
by requiring you to put a specific file on the web server. Or changing
a DNS entry… the idea is to require the requester to prove they have
access to a controlled resource associated with the domain.
This techique is used elsewhere; e.g. if you put your website into
Google Webmaster Tools then it will require you to put a specific entry
on your server (eg /googleSOMESTRING.html). If you want to use
Google Apps for your mail then it may want a DNS entry (eg googleSOMESTRING IN CNAME google.com.). These are both “domain validation” examples.
Extended Verification
Extended Validation (EV) certs take a deeper approach to validating
the request; they also check the owner claims. So, for example, I
might be able to get a DV cert for chas3.com, but I would not be able
to get an EV cert saying “this is owned by JPMorgan Chase”. An EV
cert is meant to help reduce phishing
attacks by helping the web user spot if an EV cert is in use.
Compare my site to Chase’s:
We both have the green padlock, showing the cert’s are good, but the Chase entry has more details about the owner next to the padlock.
For a site like mine a DV cert is sufficient. You need to decide if you want a DV or EV for your site.
Getting a cheap cert
When I first started out in 1996 getting a cert was a hard tedious process. We had to fax (fax!) details to one of the very few CAs that were around, and it took weeks to get the cert. Painful. It was effectively an EV process, long before such a thing really existed.
As time went on the DV process became easier to get, but they still cost money. It wasn’t really worth it for most people.
In 2012 I noticed that StartCom were providing free DV certs, under the brand StartSSL. This was an Israeli company and their CA was a common one. It made sense.
The StartSSl process was pretty painful. It was totally manual and involved using a web site with cut’n’paste. The DV process expired after 30 days so every renewal required revalidation. But it was free, and I only had 7 certs with them, so I could jump through the hoops every so often.
Unfortunately, it’s not clear if StartSSL is still trustworth, and may now operate from China.
Letsencrypt
Fortunately Letsencrypt started up. Their goal was to automate a lot of the manual steps, while still providing free DV certs. They even provide a client so you can basically say “encrypt my site” and their software will create the necessary cert files, do the DV process (stick a file on the server), retrieve the signed result, reconfigure your webserver daemon and, boom, in seconds your site is now SSL enabled. The certificate is only valid for 90 days, but that’s not a problem; the same software can automatically renew the cert before it expires.
Very impressive.
Unfortuntately it doesn’t work for me. My site is replicated across
multiple providers (one copy at Linode in Texas; one copy at Panix in
New York; one copy at OVH in Canada). I manage content on my home machine
and rsync the results out. I also manage the webserver configurations
via ansible.
If I ran their client on one machine then I’d have to copy the resulting certs to the other two… it’s doable, but messy.
However there was another way of doing the DV certification, and that’s
via DNS. Now my DNS is also replicated (hidden primary server at home;
public primary at the three sites). So this means I could do the cert
DV validation via DNS from my home machine, then push the resulting cert
out to the web servers via ansible.
The original Letsencrypt client couldn’t do DNS validation, but Lukas Schaur created an alternative shell based client called deydrated. This is a much simpler program, but allowed DNS mode to be used, and had a call out so I could modify how the DNS change ran.
How it works
Last week one of my certs needed renewing, so I went through the renew process:
$ cd etc/localnet/Letsencrypt/
$ ./refresh
+ Valid till Dec 6 23:35:00 2016 GMT (Less than 8 days). Renewing!
+ Signing domains...
+ Generating private key...
+ Generating signing request...
+ Requesting challenge for linode.spuddy.org...
+ Responding to challenge for linode.spuddy.org...
+ Challenge is valid!
+ Requesting certificate...
+ Done!
+ Creating fullchain.pem...
/home/sweh/etc/localnet/PlayBooks/Files/Postfix/Per-host/linode
The refresh command calls ./dehydrated -c to do all the hard work
of talking to the Letsencrypt servers. It noticed that the linode
cert needed renewing, and got a new signed cert. My script then
compared all the managed certs and copied any changed ones into the
ansible tree. In this case the single cert for the linode Postfix
server (also used by INN) was updated.
Since a change was done I could then use ansible to push it out:
$ cd ../PlayBooks/
$ ansible-playbook -l linode postfix-inn.yml
PLAY [Set up postfix config on mail servers] **********************************
[...]
TASK: [send machine specific certs] *******************************************
changed: [linode] => (item=privkey.pem)
changed: [linode] => (item=fullchain.pem)
NOTIFIED: [restart postfix] ***************************************************
changed: [linode]
PLAY [Set up INN] *************************************************************
[...]
TASK: [send INN privkey] ******************************************************
changed: [linode]
TASK: [send INN cert chain] ***************************************************
changed: [linode]
[...]
PLAY RECAP ********************************************************************
linode : ok=21 changed=4 unreachable=0 failed=0
And that’s it! I can also verify the new certs are in place and working
$ cd ../Letsencrypt
$ ./check_remote nntp linode.spuddy.org
linode.spuddy.org NNTP: OK notAfter=Feb 27 16:31:00 2017 GMT
$ ./check_remote smtp linode.openvpn linode.spuddy.org
linode.spuddy.org SMTP: OK notAfter=Feb 27 16:31:00 2017 GMT
How to know when to renew a cert
It would be bad to allow a cert to expire, so you need ample warning of
when a cert needs to be refreshed. Because of the tree used by the
dehydrated code is consistent this is very easy to script:
#!/bin/ksh -p
# Warn if cert will expire in this many days
let DAYS=${DAYS:-7}
let warn=DAYS*86400
typeset -R25 h
bad=""
cd /home/sweh/etc/localnet/Letsencrypt/certs || exit
for a in */cert.pem
do
b=$(openssl x509 -checkend $warn -in $a)
if [ "x$b" = "xCertificate will expire" ]
then
b=$(openssl x509 -noout -dates -in $a | sed -n 's/^notAfter=//p')
h=${a%/*}
bad="$bad\nletsencrypt $h: $b"
fi
done
if [ -n "$bad" ]
then
print Following certificate expiration dates:
print "$bad"
fi
So I can now see what certs are due to expire:
$ checkssl
$ DAYS=20 checkssl
$ DAYS=25 checkssl
Following certificate expiration dates:
letsencrypt bofh.spuddy.org: Dec 28 23:46:00 2016 GMT
letsencrypt gallery.spuddy.org: Dec 28 23:47:00 2016 GMT
letsencrypt games.spuddy.org: Dec 28 23:47:00 2016 GMT
letsencrypt home.spuddy.org: Dec 28 23:47:00 2016 GMT
letsencrypt mercury7.spuddy.org: Dec 28 23:48:00 2016 GMT
letsencrypt panix-news.spuddy.org: Dec 28 23:48:00 2016 GMT
This is easy to put into a cron job. Indeed
1 1 * * * /home/sweh/bin/checkssl
In the morning I can refresh any certs necessary while drinking my coffee :-)
Automation
So far I’ve only automated 90% of the job. I still have to run the
refresh and ansible-playbook commands. Partly this is because
it’s so easy, partly because it gives me positive feedback (I know
the work was done and the push successful), but mostly because this
is “good enough” for my use case.
When I started using this process I would occassionally get errors with the DV step; I’m not sure if that was due to DNS propagation issues or caching at the LetsEncrypt side or something else. I haven’t had a problem for quite a while now, but I like to keep my eye on it.
But we can see, quite easily, how a daily “refresh/push” process could be created so the overnight “checkssl” job could become a “check and refresh” job.
Currently I have 12 certificates managed this way, and it takes less than 5 minutes a month to manage.
Summary
Where DV certs are sufficient, a tooling around Letsencrypt is perfectly adequate and sensible. It can even be used for internal web sites, as long as DNS challenge/response can be handled.
There’s little excuse to not use SSL on web sites any more!