CS2911
Lab 10

Resources

Introduction

You will again prepare an individual report for this lab. You are welcome to work with a partner, but prepare independent reports.

In the SMTP lab (Lab 8), we created an SSL/TLS socket using code something like 

context = ssl.create_default_context()
ssl_socket = context.wrap_socket(sock, server_hostname=SMTP_SERVER)

In this lab, we will explore what's going on with this code in a little more detail.

To make things easier, instead of connecting to an SMTP server (which requires us to exchange quite a bit of plain-text with the server before going to TLS), we will connect to an HTTPS server (which uses TLS from the start). Start by copying the python code below into a new script in PyCharm. Every time you make a change to the code, re-run it to get a fresh connection to the server.

import socket
import ssl
import pprint
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('faculty-web.msoe.edu', 443))

(Note that 443 is the default HTTPS port, just like 80 is the default HTTP port)

The command ssl.create_default_context() sets up some default options. You can reproduce the effect of ssl.create_default_context() (at least for Python 3.5.2) by running these commands: (found in the source code for create_default_context())

import ssl

context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)

# SSLv2 considered harmful.
context.options |= ssl.OP_NO_SSLv2

# SSLv3 has problematic security and is only required for really old
# clients such as IE6 on Windows XP
context.options |= ssl.OP_NO_SSLv3

# disable compression to prevent CRIME attacks (OpenSSL 1.0+)
context.options |= ssl.OP_NO_COMPRESSION

# verify certs and host name in client mode
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True

# Let's try to load default system
# root CA certificates for the given purpose. This may fail silently.
context.load_default_certs(ssl.Purpose.SERVER_AUTH)

Let's look through this sample code step-by-step

context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)

# SSLv2 considered harmful.
context.options |= ssl.OP_NO_SSLv2

# SSLv3 has problematic security and is only required for really old
# clients such as IE6 on Windows XP
context.options |= ssl.OP_NO_SSLv3

In the code sample above, we see that the SSL protocol uses SSLv23. This is a version that allows the server and the client to negotiate about which SSL version they will use. Next, it explicitly disallows SSLv2 and SSLv3. These older versions are no longer secure. This leaves open the option to use the newer TLS versions.

To see what standards are available, from the Python console, run the command dir(ssl). This will list all of the variables in the ssl module. In the list, everything that starts with PROTOCOL_ is a version that is available. You may find it helpful to write a short loop (hover here for spoilerfor s in dir(ssl):
  if s.startswith('PROTOCOL_'):
   print(s)) to see these more clearly.

You will see that TLSv1, TLSv1_1, and TLSv1_2 are all available. Our settings will allow us to use whichever ones of these the server has enabled. All of these are newer and more secure than the SSL versions.

Now let's continue through example code:

# disable compression to prevent CRIME attacks (OpenSSL 1.0+)
context.options |= ssl.OP_NO_COMPRESSION

You see that the default context turns off compression with OP_NO_COMPRESSION. It turns out that compressing data can allow an attacker to figure out what some of the encrypted data is — with a chosen-plaintext attack. (You can optinally read the Wikipedia article on the CRIME attack for more details.)

# verify certs and host name in client mode
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True

The example code then turns on two very important options: CERT_REQUIRED (meaning don't make a connection unless you can confirm the certificate of the other end) and check_hostname (meaning don't make a connection unless the certificate is for the hostname we want to connect to.)

# Let's try to load default system
# root CA certificates for the given purpose. This may fail silently.
context.load_default_certs(ssl.Purpose.SERVER_AUTH)

Finally, the default context loads the default root certificates provided by the operating system. For the certificate to be valid, it must be signed by one of the root CAs that the operating system recognizes. These certificates contain the public keys for the main root certificate authorities (CAs) on the internet. If a certificate was signed by the root CA, the public key on our machine (which we trust) should correctly decrypt the certificate's data.

Ok, now let's try some things. Create your own custom default context by pasting all the commands we just looked through into your Python script. Add the line to wrap the socket using this context from the beginning of the lab. Replace SMTP_SERVER with a string holding the domain name of the server you are trying to connect to. Confirm that you are able to make a connection to the server without wrap_socket throwing an exception. (You do not need to send/receive any HTTP requests/responses. We will be looking at the certificates instead. Imagine that you really don't trust your network connection — you would not want to send any data until you knew the connection was secure.)

Once a connection is established, you can view the certificate information by calling ssl_socket.getpeercert(). Since the output is a little long, it can be handy to pass it through a pretty-printing module that will format it nicely:

import pprint # Included in imports above
print(pprint.pformat(ssl_socket.getpeercert()))

Paste the code above into your script. Run the script, and look at the certificate information that prints. In your report, note the issuer's organizationName, the subject's organizationName, and the subject's commonName. What sort of name does the subject's commonName look like to you?

Also look at the dates in the "notBefore" and "notAfter" fields. Write when you think the certificate will expire based on these dates.

Now edit the server_hostname= to be something bogus like "nowhere.com". Try to connect again. Can you still connect? (Hopefully not. No need to write anything about this yet.)

Now edit your code to turn off host checking:

context.check_hostname = False

Now attempt to connect to the server without providing the hostname (or by providing a bogus hostname)

ssl_socket = context.wrap_socket(sock) 
# OR
ssl_socket = context.wrap_socket(sock, server_hostname="nowhere.com")

Are you still able to connect to the server? (You should be able to.) Is this good? (No!)

In your report, on a separate piece of paper, describe how an attacker could take advantage of a turned-off hostname check to make a server that looks like MSOE's email server that we used in Lab 8. Suppose the attacker is able to intercept all your network traffic so that any TCP connection you make to the server actually goes to his own server. (This is called a man-in-the middle attack.) Describe how hostname-checking would block this attack.

Now, let's not even bother with certificates at all. Turn off certificate checking (you will need to leave hostname checking off for this to work):

context.verify_mode = ssl.CERT_NONE

Are you still able to connect to the serer? (You should be able to.) Do you still get a certificate (You may not.) Is this good? (No!)

In your report, describe what an attacker would need to do to appear as the msoe email server in this case. Since we have even fewer checks enabled, the attacker should be able to get away with even more. What additional freedoms do we give the attacker by turning off certificate checks entirely?

By the way, if you don't want security checking, you don't even need the SSL context. You can simply wrap the socket directly:

ssl_socket = ssl.wrap_socket(sock)

(Of course, I do not recommend doing this in practice!)

Now turn certificate checking and hostname checking back on.

So we've seen how hostname checking is important, how any kind of certificate checking is important, and how certificates can expire.

Now let's consider one more scenario. Suppose that an attacker somehow steals the private key for a certificate before it expires. Now suppose the owner of the certificate would like to revoke the key. They can't just go all over the internet and take the key back. The certificate and its public key are out there, and they can be confirmed by checking them against the root CA's public key (which is still perfectly valid and hasn't been compromised).

The normal way for the certificate owner to handle this is to REVOKE the certificate by placing it on a Certificate Revocation list.

Within Python with a default context, try connecting to the domain names and port numbers given in these URLs: (Recall that the port number is the number that follows the : after the domain name. And if no port is given, you should use 443 for HTTPS)

More example certificates can be found at https://badssl.com/

In addition to trying these pages through Python's SSL library, also try them from your browsers to see how they respond.

In your report, note which of these URLs you can connect to from Python and your browser. Note which browser you are using (Chrome, Firefox, ...). Write what you learn about Python's current SSL module from this experiment.

Just for fun, to explore an up-and-coming certificate revocation checking strategy, install opensll (e.g., in Cygwin) and try out these instructions for checking certificates with OCSP. If you you use cygwin, you only need the basic packages plus the openssl package. (I also have rsync, wget, Archive/zip, Archive/unzip, and nc (netcat) installed. These are handy network and utility programs, but I doubt you need them for this lab.) One note from my experience following these instructions: They instruct you to save the certificate in a wikipedia.pem file. Later, they instruct you to save all the certificates in a chain.pem file. Don't include the wikpedia cert in the chain. This will give you hard-to-understand errors. (And as they say, "Number 0 is the certificate for Wikipedia, we already have that.")

(Acknowledgements: This lab written by Dr. Yoder)