CS2911
Lab 9

Resources

Introduction

You will prepare an individual report for this lab. You are welcome to work with a partner, but prepare independent reports.

In Lab 5, the just-for-fun section talked through how to turn your HTTP client into an HTTPS client. While discussing SMTP, we discussed how the same code could be used with SMTP and the STARTTLS command to encrypt the sending of the email. Both examples used this code:

context = ssl.create_default_context()
ssl_socket = context.wrap_socket(sock, server_hostname=SMTP_SERVER)

In this lab, we will explore what's going on with this code in a little more detail. Although all the code is provided in the lab, you will need to piece the pieces together. Please read each piece to understand what it does to facilitate the puzzle-work.

We will connect to an HTTPS server as in the just-for-fun exercise in Lab 5, but not actually send any data through the wrapped socket. (So no need to break out your Lab 5 code until you have completed the required exercises!) Start by copying the python code below into a new script in PyCharm. Every time you make a change to the code, re-run it to get a fresh connection to the server.

import socket
import ssl
import pprint
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('faculty-web.msoe.edu', 443))

(Note that 443 is the default HTTPS port, just like 80 is the default HTTP port)

To this code, add some code to create a wrapped socket. Confirm that the socket can be wrapped succesfully when connected to a secure web-server. There is no need to send or receive a response through the socket. Indeed, we want to reflect on how you can know you can trust the server before sending it any information!

The command ssl.create_default_context() sets up some default options. You can reproduce the effect of ssl.create_default_context() (at least for Python 3.5.2) by running these commands: (found in the source code for create_default_context())

import ssl

context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)

# SSLv2 considered harmful.
context.options |= ssl.OP_NO_SSLv2

# SSLv3 has problematic security and is only required for really old
# clients such as IE6 on Windows XP
context.options |= ssl.OP_NO_SSLv3

# disable compression to prevent CRIME attacks (OpenSSL 1.0+)
context.options |= ssl.OP_NO_COMPRESSION

# verify certs and host name in client mode
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True

# Let's try to load default system
# root CA certificates for the given purpose. This may fail silently.
context.load_default_certs(ssl.Purpose.SERVER_AUTH)

In your Python file, replace the call to ssl.create_default_context() with the code above. Confirm that you can still connect to the server.

Let's look through this sample code step-by-step

context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)

# SSLv2 considered harmful.
context.options |= ssl.OP_NO_SSLv2

# SSLv3 has problematic security and is only required for really old
# clients such as IE6 on Windows XP
context.options |= ssl.OP_NO_SSLv3

In the code sample above, we see that the SSL protocol uses SSLv23. This is a version that allows the server and the client to negotiate about which SSL version they will use. Next, it explicitly disallows SSLv2 and SSLv3. These older versions are no longer secure. This leaves open the option to use the newer TLS versions.

Question Set 1

To see what standards are available, from the Python console, run the command dir(ssl). This will list all of the variables in the ssl module. In the list, everything that starts with PROTOCOL_ is a version that is available. You may find it helpful to write a short loop (hover here for spoilerfor s in dir(ssl):
  if s.startswith('PROTOCOL_'):
   print(s)) to see these more clearly.

You may see that TLSv1, TLSv1_1, and TLSv1_2 are all available. Our settings will allow us to use whichever ones of these the server has enabled. All of these are newer and more secure than the SSL versions.

In your report, describe a security risk that the default SSL version settings avoid. You may write this onto the checklists itself or into a fuller document to print.

Now let's continue through example code:

# disable compression to prevent CRIME attacks (OpenSSL 1.0+)
context.options |= ssl.OP_NO_COMPRESSION

You see that the default context turns off compression with OP_NO_COMPRESSION. It turns out that compressing data can allow an attacker to figure out what some of the encrypted data is — with a chosen-plaintext attack. (You can optinally read the Wikipedia article on the CRIME attack for more details.)

# verify certs and host name in client mode
context.verify_mode = ssl.CERT_REQUIRED
context.check_hostname = True

The example code then turns on two very important options: CERT_REQUIRED (meaning don't make a connection unless you can confirm the certificate of the other end) and check_hostname (meaning don't make a connection unless the certificate is for the hostname we want to connect to.). We will return to these options in a moment.

# Let's try to load default system
# root CA certificates for the given purpose. This may fail silently.
context.load_default_certs(ssl.Purpose.SERVER_AUTH)

Finally, the default context loads the default root certificates provided by the operating system. For the certificate to be valid, it must be signed by one of the root CAs that the operating system recognizes. These certificates contain the public keys for the main root certificate authorities (CAs) on the internet. If a certificate was signed by the root CA, the public key on our machine (which we trust) should correctly decrypt the certificate's data.

Question Set 2

Ok, now let's try some things. If you have not yet done so, create your own custom default context combining all of the code snippets provided so far. Add the line to wrap the socket using this context from the beginning of the lab. Replace SMTP_SERVER with a string holding the domain name of the server you are trying to connect to. Confirm that you are able to make a connection to the server without wrap_socket throwing an exception. (You do not need to send/receive any HTTP requests/responses. We will be looking at the certificates instead. Imagine that you really don't trust your network connection — you would not want to send any data until you knew the connection was secure.)

Once a connection is established, you can view the certificate information by calling ssl_socket.getpeercert(). Since the output is a little long, it can be handy to pass it through a pretty-printing module that will format it nicely:

import pprint # Included in imports above
print(pprint.pformat(ssl_socket.getpeercert()))

Paste the code above into your script. Run the script, and look at the certificate information that prints. In your report, note the issuer's organizationName, the subject's organizationName, and the subject's commonName. What sort of name does the subject's commonName look like to you?

Also look at the dates in the "notBefore" and "notAfter" fields. Write when you think the certificate will expire based on these dates.

Now edit the server_hostname= to be something bogus like "nowhere.com". Try to connect again. Can you still connect? (Hopefully not. No need to write anything about this yet.)

Now edit your code to turn off host checking:

context.check_hostname = False

Now attempt to connect to the server without providing the hostname (or by providing a bogus hostname)

ssl_socket = context.wrap_socket(sock) 
# OR
ssl_socket = context.wrap_socket(sock, server_hostname="nowhere.com")

Are you still able to connect to the server? (You should be able to.) Is this good? (No!)

Question Set 3

In your report, on a separate piece of paper, describe how an attacker could take advantage of a turned-off hostname check (domain name check) to make a server that looks like https://faculty-web.msoe.edu, but is actually https://hackercentral.com. Suppose the attacker is able to intercept all your network traffic so that any TCP connection you make to the server actually goes to his own server. (This is called a man-in-the middle attack.) Describe how hostname-checking would block this attack. To provide more structure to this question, here are some constraints:

  • Assume that RSA cannot be broken -- that a sufficient key length is used to thwart off this attacker.
  • Assume that the certificate authorities can be trusted to do their jobs correctly and to protect their private keys accurately.

Despite these parts of the network being strong, by leaving off hostname checking, we are leaving ourselves WIDE OPEN to a man-in-the-middle attack. What does an attacker need to do in this case for us to successfully establish a connection to them, thinking we are establishing a connection to faculty-web?

Question Set 4

Now, let's not even bother with certificates at all. Turn off certificate checking (you will need to leave hostname checking off for this to work):

context.verify_mode = ssl.CERT_NONE

Are you still able to connect to the serer? (You should be able to.) Do you still get a certificate (You may not.) Is this good? (No!)

In your report, describe

  • What step(s) from Part 3 an attacker can omit and still convince our latest code to successfully establish a connection to them, thinking we are establishing a connection to faculty-web?

By the way, if you don't want certificate checking, you don't even need the SSL context. You can simply wrap the socket directly:

ssl_socket = ssl.wrap_socket(sock)

We do not recommend doing this in practice, because it leaves your code wide open to the man-in-the-middle attacks we are asking you to explore here!

Question Set 5

Now turn certificate checking and hostname checking back on.

We have now seen how hostname checking is important, how any kind of certificate checking is important, and how certificates can expire.

Now let's consider one more scenario. Suppose that an attacker somehow steals the private key for a certificate before it expires. Now suppose the owner of the certificate would like to revoke the key. They can't just go all over the internet and take the key back. The certificate and its public key are out there, and they can be confirmed by checking them against the root CA's public key (which is still perfectly valid and hasn't been compromised).

The normal way for the certificate owner to handle this is to REVOKE the certificate by placing it on a Certificate Revocation list.

Within Python with a default context, try connecting to the domain names and port numbers given in these URLs: (Recall that the port number is the number that follows the : after the domain name. And if no port is given, you should use 443 for HTTPS)

More example certificates (including a self-signed certificate) can be found at https://badssl.com/

In addition to trying these pages through Python's SSL library, also try them from your browsers to see how they respond.

In your report, note which of these URLs you can connect to from Python and your browser. Note which browser you are using (Chrome, Firefox, ...). Write what you learn about Python's current SSL module from this experiment.

Just for fun, to explore an up-and-coming certificate revocation checking strategy, install opensll (e.g., in Cygwin) and try out these instructions for checking certificates with OCSP. If you you use cygwin, you only need the basic packages plus the openssl package. (I, Dr. Yoder, also have rsync, wget, Archive/zip, Archive/unzip, and nc (netcat) installed. These are handy network and utility programs, but I doubt you need them for this lab.) One note from my experience following these instructions: They instruct you to save the certificate in a wikipedia.pem file. Later, they instruct you to save all the certificates in a chain.pem file. Don't include the wikpedia cert in the chain. This will give you hard-to-understand errors. (And as they say, "Number 0 is the certificate for Wikipedia, we already have that.")

(Acknowledgements: This lab written by Dr. Yoder)

Submission Instructions for Dr. Yoder

For Dr. Yoder's section, please use the following checklist:

Again, these instructions apply to Dr. Yoder's students only.