This is an old version of this course, from Fall 2015. A newer version is available here.

Lab Assignment

This is a team assignment; each team should be two members unless a different size is approved by the instructor.

Introduction

The goal of this lab is to write a short Python program, to request and save a web resource, acting as an HTTP client.

You will start from the template httpclient.py.

The program has at least the following functions; you may add others.

main()

The provided main function will perform basic tests. You may add others. No user input is needed.

• This method has no arguments
• This method is invoked with a main() function call at the end of the program.

get_http_resource(url,file_name)

Using HTTP, request a web resource and store the returned data in the specified file.

• Arguments:
  • url: string containing URL (including the "http://" protocol declaration and the domain name) for desired resource
  • file_name: string containing name of file in which to store response data
• Return value:
  • String "OK, requested entity received, nnn bytes in size" or error message
  • None
• Operation:
  • Send an HTTP request to retrieve the resource at the specified url.
    • The client should recognize and interpret both Content-Length and chunked responses
    • Although you do need to implement chunkning, you do not need to implement the chunk extensions described in the RFC (and the videos)
  • If successful, store the response data in a file with the specified file_name.
  • Return the specified status information as a string.

make_http_request(host, port, resource, file_name)

Using HTTP, request a web resource and store the returned data in the specified file.

• Arguments:
  • host: the domain name or IP address of the server (i.e., host) to connect to
  • port: port number to connect to on host
  • resource: path/name of resource to get. This is everything in the URL after the domain name, including the first /.
  • file_name: name of file in which to store the retrieved resource
• Return value:
  • None

Procedure

1. Download the skeleton Python template: httpclient.py
2. Edit the header of the file to include your team members' names.
3. Complete the method to request, receive, and store the designated resource. You should add other helper methods, but do not change the code provided in the template. As a team, design the methods and the data passed between them. Then divide your efforts, with each team-member writing at least one method. Document who writes each methods with a comment starting with Author:
4. Add comments at the end of your Python file, with the following information:
   • A description of the functionality you implemented and the results of your testing.
   • Comments on your experience in completing the lab, including any problems you encountered. Briefly explain what you learned.
   • Questions and suggestions.

If this base functionality turns out to be too easy, you may experiment with adding additional functions, but be sure the basic requirements are still met.

Divide up the primary responsibility for parts of the program in an equitable way.

Include a multiline #-comment (not docstring) at the start of each method, including a description of the method, a descriptions of the parameters (e.g. what type or value is expected), and a description of the value that is returned.

You should include a comment for every method, but you do not need to document every argument. Instead, document as many arguments as you can document non-trivially. For example, if the only thing you can think of to describe a variable num_lines is "The number of lines" then don't document that variable. But if you think it is useful to say, "num_lines - The number of CRLF-terminated newlines in the entity body" then include the comment. As a ballpark estimate, you should be able to comment at least half of your arguments this way.

As the last line of each method's comment, include the string # Author: Phileas Fogg, replacing Phileas Fogg with the name of the primary author of the method. Each member of the team should be the primary author of at least one method, but a more equitable distribution is desirable.

You may use the next_byte() method from Lab3 if you choose. Whether or not you use next_byte(), your program should use recv correctly. It should handle the situtation when recv() returns fewer bytes than expected during normal transmission. (Though it only needs to handle a zero-byte return after a complete HTTP transmission is received.)

You do not need to implement a persistent connection. The server will send only one response unless you explicitly request a persistent connection. Nevertheless, your code should be extensible to the persistent case. In other words, when you are reading the message, the program should not attempt to read any bytes past the end of the message, to avoid reading bytes from the second response.

To receive an A in this and future labs (throughout the curriculum), go beyond the requirements in some way.