Resources
- Lab Checklist — I will provide a paper copy of this.
- CS2911 Coding Standard
You can work the entire lab as a prelab. Please work in teams of two unless approved by the instructor. Please submit only one report per team. (You will comment which part of your team's work you did.)
Introduction
This week, you will submit your solution to the following problem:
You will write a program to receive a message over the network, using a simple next_byte method. The next_byte method's implementation is not provided until next week's lab (only a debug version is provided). You will write the payload of the received message to a file.
Message Format
A message has a four-byte header with a single field. This field contains the number of lines in a text file as a raw binary number. This four-byte field is in the standard network byte order, that is, big-endian.
Then, the lines follow, sent as ASCII text, each terminated by '\n', that is, a new-line character.
For example, to send the text file
Lab 1 Phileas Fogg This is my lab 1 assignment.
these bytes would be sent: (showing both the hexadecimal shorthand and the ASCII characters, just as Wireshark does)
00 00 00 04 4c 61 62 20 31 0a 50 68 69 6c 65 61 ....Lab 1.Philea 73 20 46 6f 67 67 0a 0a 54 68 69 73 20 69 73 20 s Fogg.. This is 6d 79 20 6c 61 62 20 31 20 61 73 73 69 67 6e 6d my lab 1 assignm 65 6e 74 2e 0a ent..
In this example, there are four lines of text (including the blank line). In the data, we see two \n's in a row (0a 0a) because one is the end of the line "Philues Fogg, and the next is the end of the blank line.
In this example, you would save the text file, but not any of the header bytes.
The next_byte method
Suppose you can get one byte (single-byte string) of the message by calling next_byte(). Here's a full comment for next_byte:
def next_byte():
"""
Read the next byte from the sender, received over the network.
If the byte has not yet arrived, this method blocks (waits)
until the byte arrives.
If the sender is done sending and is waiting for your response, this method blocks indefinitely.
:return: the next byte, as a bytes object with a single byte in it
"""
You do not need to implement the next_byte method. I will provide it in a later lab, and a testing next_byte is provided later on in this lab.
There are no other ways to access the bytes in the message. There is no has_next_byte() method. You should not ask the user to enter them yourself (The sender is on the other side of the network.) The message coming over the network will come through next_byte().
Teamwork and Design
Your solution will be Python code, consisting of multiple methods. Put your top-level method first in the file. This should call other helper methods, which may call other helper methods as well.
Follow the parser design steps that were introduced in class:
- Write an example message. I recommend writing a very short message that you can use for debugging.
- Brainstorm steps needed to debug
- Write pseudocode
- Write Python code
In steps 2-4, focus your efforts on how your program will know when it has read the last byte of the message. This is the biggest challenge of the lab.
In step 4, write the names of the actual python methods together (e.g. def read_something() and decide who will write each method. Then, figure out the place that one person's code will call the other's. (Ideally there will be just one such place.) In the caller's method, write the method call, e.g.
def read_automobile_body():
bolts = read_bolts(number_of_hex_bolts)
And then write the complete docstring for the method that is called, e.g.,
def read_bolts():
"""
This method reads the hex-bolts out of the chassis
:param number_of_bolts: number of hex-bolts needed for the chassis
:return: a list containing all the hex-bolts objects found
:author: Phileas Fogg
"""
Now, the caller knows exactly what read_something_smaller will do for them, and the callee knows exactly what they need to write. So ideally, the two pieces of code can be written at the same time and then will fit together perfectly for integration.
Divide the coding effort as evenly as you can. In a comment at the start of each method, include the author's name, with each team-member being the primary author for at least one method. Once you agree on who will write each method, be sure to agree on the arguments, return value, and purpose of each method. Include these in a comment at the beginning of the method. When you finish writing your code, revise the comments so they match the final version.
Informal rubric
(See the checklist for the formal rubric)
You do not need to run the code — it is OK if simple errors cause your code not to run at all. You don't even need to import everything needed by your code. In fact, you really CAN'T run your code to receive a message over the network, since it relies on the next_byte() method, whose true network implementation is not provided. However, you may find it helpful to use the implementation of next_byte() that comes in the next section for testing. You don't even need to import everything needed by your code.
I will consider these quality factors when grading this week's lab:
- How well does the code break the large task into smaller sub-tasks?
- Is the work divided reasonably among the team members?
- Does the code read in the message in the specified format, without reading past the end of the message?
- Does the code illustrate an understanding of how bytes and numbers are represented and converted in Python?
- Does the code illustrate understanding of Python syntax, types, and binary encodings?
- Does the code save the payload message to a file?
Example next_byte method
Although the real next_byte is not provided, you can use this example code to test your method. You can type in each byte of the message as hexadecimal shorthand. You type one byte for each call to the method, so you need to wait until your program calls next_byte to type the next byte of the message.
def next_byte():
"""
Enter the byte in hexadecimal shorthand
e.g.
input a byte: e3
:return: the byte as a bytes object holding one byte
:author: Eric Nowac
"""
msg = input('input a byte: ')
return(int(msg, 16).to_bytes(1,'big'))
Some students write their own version of next_byte. This can be convenient because you don't have to keep typing the bytes over and over again. Please talk to your instructor about the details of doing this.
Example code to write to a file
Below is a simple example of writing a text-file in Python, taken from tutorialspoint.com and updated to Python 3.
When opening a file, you can use "b" for binary mode, or leave it off for text mode. In binary mode, you write the contents of bytes objects to a file. In text mode, you write the contents of str objects to the file.
I recommend writing in binary mode, unless you are SURE that your message only contains ASCII text. You must use binary mode if you wish to write raw bytes to a file. If you use text mode, it will automatically encode the text in UTF-8 -- changing what hex values are written. On a Windows machine, if you write in text mode, '\n' is converted to '\r\n', which may not be what you intended.
This will be especially important in the HTTP labs when you write binary image data to files.
#!/usr/bin/python
# Open a file
output_file = open('foo.txt', 'wb')
# We are writing raw bytes (plain ASCII text) to the file
# Since we opened the file in binary mode,
# you must write a bytes object, not a str.
output_file.write(b'Python is a great language.\nYeah it's great!!\n')
# You can call write multiple times to write more data. You need to explicitly add the newlines yourself, as in the example above.
# Close file
output_file.close()
For (way too much) more information, see the documentation of the Python 3 open method and the Overview of the io module.