Resources

• Dr. Yoder's section: Please see the checklist at the end of the lab.
• Other sections: Please consult with your instructor for submission instructions.
• CS2911 Coding Standard

Please work in teams of two unless approved by the instructor. Please submit only one report per team. (You will comment which part of your team's work you did.)

Introduction

This week, you will submit your solution to the following problem:

You will write a program to receive a message over the network, using a simple next_byte method. The next_byte method's implementation is not provided until next week's lab (only a debug version is provided). You will write the payload of the received message to a file.

Message Format

A message has a four-byte header with a single field. This field contains the number of lines in a text file as a raw binary number. This four-byte field is in the standard network byte order, that is, big-endian.

Then, the lines follow, sent as ASCII text, each terminated by '\n', that is, a new-line character.

For example, to send the text file

Lab 1
Phileas Fogg

This is my lab 1 assignment.

these bytes would be sent: (showing both the hexadecimal shorthand and the ASCII characters, just as Wireshark does)

00 00 00 04 4c 61 62 20   31 0a 50 68 69 6c 65 61   ....Lab  1.Philea
73 20 46 6f 67 67 0a 0a   54 68 69 73 20 69 73 20   s Fogg.. This is
6d 79 20 6c 61 62 20 31   20 61 73 73 69 67 6e 6d   my lab 1  assignm
65 6e 74 2e 0a                                      ent..

In this example, there are four lines of text (including the blank line). In the data, we see two \n's in a row (0a 0a) because one is the end of the line "Phileas Fogg, and the next is the end of the blank line.

In this example, you would save the text file, but not any of the header bytes.

The next_byte method

Suppose you can get one byte (single-byte string) of the message by calling next_byte(). Here's a full comment for next_byte:

def next_byte():
    """
    Read the next byte from the sender, received over the network.
    If the byte has not yet arrived, this method blocks (waits)
      until the byte arrives.
    If the sender is done sending and is waiting for your response, this method blocks indefinitely.
    
    :return: the next byte, as a bytes object with a single byte in it
    """

You do not need to implement the next_byte method. I will provide it in a later lab, and a testing next_byte is provided later on in this lab.

There are no other ways to access the bytes in the message. There is no has_next_byte() method. You should not ask the user to enter anything outside of the next_byte() method because the data will be coming over the network; the sender is on the other side of the network. The message coming over the network will come through next_byte().

Teamwork and Design

Your solution will be Python code, consisting of multiple methods. Put your top-level method first in the file. This should call other helper methods, which may call other helper methods as well.

Follow the parser design steps:

1. Write an example message. I recommend writing a very short message that you can use for debugging.
2. Brainstorm list of steps needed to receive the message
3. Write pseudocode
4. Write Python code

In steps 2-4, focus your efforts on how your program will know when it has read the last byte of the message. This is the biggest challenge of the lab.

In step 4, write the names of the actual python methods together (e.g. def read_something() and decide who will write each method. Then, figure out the place that one person's code will call the other's. (Ideally there will be just one such place.) In the caller's method, write the method call, e.g.

def read_automobile_body():
    bolts = read_bolts(number_of_hex_bolts)

And then write the complete docstring for the method that is called, e.g.,

def read_bolts():
    """
    This method reads the hex-bolts out of the chassis
    :param number_of_bolts: number of hex-bolts needed for the chassis
    :return: a list containing all the hex-bolts objects found
    :author: Phileas Fogg
    """

Now, the caller knows exactly what read_bolts() will do for them, and the callee knows exactly what they need to write. So ideally, the two pieces of code can be written at the same time and then will fit together perfectly for integration. Now complete your methods individually.

Divide the coding effort as evenly as you can. In a comment at the start of each method, include the author's name (using the Sphinx field :author: field) with each team-member being the primary author for at least one method. Once you agree on who will write each method, be sure to agree on the arguments, return value, and purpose of each method. Include these in a comment at the beginning of the method. When you finish writing your code, revise the comments so they match the final version.

Informal rubric

(See the checklist for the formal rubric for Dr. Yoder's section)

You do not need to run the code — it is OK if simple errors cause your code not to run at all. You don't even need to import everything needed by your code. In fact, you really CAN'T run your code to receive a message over the network, since it relies on the next_byte() method, whose true network implementation is not provided. However, you may find it helpful to use the implementation of next_byte() that comes in the next section for testing. You don't even need to import everything needed by your code.

Your instructor may consider these quality factors when grading this week's lab:

• How well does the code break the large task into smaller sub-tasks?
• Is the work divided reasonably among the team members?
• Does the code read in the message in the specified format, without reading past the end of the message?
• Does the code illustrate an understanding of how bytes and numbers are represented and converted in Python?
• Does the code illustrate understanding of Python syntax, types, and binary encodings?
• Does the code save the complete payload message to a file?

Example next_byte methods

Although the real next_byte is not provided, you can use these example methods that have the same interface. You can type in each byte of the message as hexadecimal shorthand. You type one byte for each call to the method, so you need to wait until your program calls next_byte to type the next byte of the message.

def next_byte():
    """
    Enter the byte in hexadecimal shorthand
    e.g. 
      input a byte: e3
    
    :return: the byte as a bytes object holding one byte
    :author: Eric Nowac
    """

    msg = input('input a byte: ')
    return int(msg, 16).to_bytes(1,'big')

The second optional method that you can use to test your program uses a Python generator. Because the next_byte_socket_factory() method includes a yeild keyword, it does not execute when you run it. Instead, it returns a sequence object that executes until it hits yield (or return) every time you call it. This will raise an exception when the end of the stream is reached. Your code should NOT catch that exception -- you should never call next_byte() past the end of the stream. (In a larger program, this would "eat" part of the next message)

def next_byte_socket_factory():
    """
    Enter the byte in hexadecimal shorthand
    e.g.
      input a byte: e3

    :return: the byte as a bytes object holding one byte
    :author: Dr. Yoder
    """

    msg = (b'\x00\x00\x00\x04\x4c\x61\x62\x20'
           b'\x31\x0a\x50\x68\x69\x6c\x65\x61'
           b'\x73\x20\x46\x6f\x67\x67\x0a\x0a'
           b'\x54\x68\x69\x73\x20\x69\x73\x20'
           b'\x6d\x79\x20\x6c\x61\x62\x20\x31'
           b'\x20\x61\x73\x73\x69\x67\x6e\x6d'
           b'\x65\x6e\x74\x2e\x0a')

    for b in msg:
        yield bytes((b,)) # b holds the contents of the byte as an int.
                          # This converts it back to a bytes object
                          # of a single byte.

def next_byte(data_socket):
    return next(data_socket)

To use this second method, you must create the socket at the start of your code, and then pass it through to everywhere it is used

def main():
    data_socket = next_byte_socket_factory()
    read_automobile_body(data_socket)

def read_automobile_body(data_socket):
    bolts = read_bolts(data_socket, number_of_hex_bolts)

def read_bolts(data_socket, number_of_hex_bolts):
    radius_byte = next_byte(data_socket)

# ...

You are not required to use either version of next_byte(), but using a version of next_byte can help you to uncover design issues in your code.

Example code to write to a file

Below is a simple example of writing a text-file in Python, taken from tutorialspoint.com and updated to Python 3.

When opening a file, you can use "b" for binary mode, or leave it off for text mode. In binary mode, you write the contents of bytes objects to a file. In text mode, you write the contents of str objects to the file.

I recommend writing in binary mode, unless you are SURE that your message only contains ASCII text. You must use binary mode if you wish to write raw bytes to a file. If you use text mode, it will automatically encode the text in UTF-8 -- changing what hex values are written. On a Windows machine, if you write in text mode, '\n' is converted to '\r\n', which may not be what you intended.

This will be especially important in the HTTP labs when you write binary image data to files.

#!/usr/bin/python

# Open a file (like try-with-resources in Java)
with open('foo.txt', 'wb') as output_file:

    # We are writing raw bytes (plain ASCII text) to the file
    # Since we opened the file in binary mode,
    # you must write a bytes object, not a str.
    output_file.write(b'Python is a great language.\nYeah it's great!!\n')

    # You can call write multiple times to write more data. You need to explicitly add the newlines yourself, as in the example above.

# The with open syntax guarantees the file will be closed. If you use a different
# syntax, be sure to close it.
# output_file.close()

For (way too much) more information, see the documentation of the Python 3 open method and the Overview of the io module.