This is an old version of this course, from Spring 2015. A newer version is available here.

Introduction

Here it is! The final project! In this lab, you will create a video phone. The project will involve sending video and audio across the network to a second BeagleBone. You will use sockets. You will use real time analysis. And you will have to do some design, as well as verification that your system is working properly.

Because of the increased scope of this lab, and the timing with the end of the quarter, this project may be completed in larger teams of up to 4 people. To ensure maximum learning of all parts of your code, part of your project will be a code review of the code writtten by your teammates. On the final exam, there may be questions related to any portion of the implemented system.

This is also a multiweek lab. The final report will not be due until the end of the quarter.

Initial steps

You will need to install the timing utility on your Beaglebone. To do this, issue the following command:

sudo apt-get install time

This will install a necessary timing utility on your beaglebone.

You may also find it useful to install a virtual keyboard on your Beaglebone. You can do this with the command

sudo apt-get install florence

Basic requirements

The basic requirements of your system are:

• Requirement R1: The video conferencing system shall capture video frames from a web cam.
• Requirement R2: Video frames will be captured from the webcam shall be shown on the local display.
• Requirement R3: Video frames captured from the webcam shall be sent across the network using a socket to a second machine.
• Requirement R4: The system shall capture audio and transmit it across the network to a second device.
• Requirement R5: The second machine shall display the video frames on the LCD panel as well as play back the audio that has been captured.
• Requirement R6: The video and audio playback shall be of reasonable quality, given hardware constraints.
• Requirement R7: The system shall integrate OpenCV and the ALSA libraries as is appropriate.
• Requirement R8: Implemented code shall be fully documented with Doxygen comments explaining the purpose and rational for the implementation.

Part 1: Developing your design

The first part of this project is to sketch your design. Based on the analysis that we have done thusfar this quarter, and the tools you have been shown, sketch a set of UML diagrams showing the various classes that will exist and the main attributes for those classes. For classes that have threads, clearly denote that they have threads.

You may find it helpful to use Qt's threads, slots, and signals as discussed (at a high level) in class. Alternatively, while we have not done this thusfar in class, you may find it advantageous to design a generic "Threadable" class that can serve as a parent class for multithreading. (Remember, C++ supports multiple inheritance **Shudder**, so doing it this way will not be as problematic as it might be in Java.)

Once you have completed your design, as a team, you should do a review of the threads to determine their execution rates as well as the required bandwidth. You may find it advantageous to adjust the frame rates and quality to match your needs. For example, lower frame rates may result in lower bandwidth utilization for images.

With the above step completed, you should be able to perform a basic rate monotonic analysis on the system to determine schedulability of the system. (Although we are not using the Rate Monotonic Scheduling algorithm, this is still a good "sanity check") You may need to go back and use the time method on previous labs to get more detailed information on the execution time for various programs. However, most of the data should be available in at least a high level format from your previous labs.

In working on your design, you should pay close attention to the QT UI design. It is probable that you will need to add some additional signals and slots.

Dividing the work

After you have a plan for the key classes in your project, decide how to divide the implementation work among your team members.

You may want to devise unit tests (these do not have to be automated) for each class, so each person can test their own work before integration testing. You may find it advantageous, at least for the video portion, to start by sending the images to the virtual machine instead of another Beaglebone.

Implementing Your Design

Once you have completed your design, start implementation. It is recommended that the first part you integrate is the audio across the network, as that may be the harder one to get right. We may be talking about prioritization in lecture, and may discuss how to handle thread priorities.

Once this is integrated, completed and working, then you can move on to integrating the transmission of the images from the camera, which is the harder problem to solve.

You may find it advantageous to first start by building the image capture program to run on the Beaglebone without an audio channel. This will allow you to capture and show images on your Beaglebone in pseudo real time.

Your GUI must remain responsive while the video is playing. You may want the video producer to execute in its own thread. There are several ways you can accomplish this. The first is to use good old fashion posix threads. You also can use the QT QThread class, which is extremely similar to how Java handles threads. (See http://qt-project.org/doc/qt-4.8/qthread.html and http://qt-project.org/doc/qt-4.8/thread-basics.html#which-qt-thread-technology-should-you-use)

In doing this testing, you should use a conditional compile that will allow the program to be rebuilt for either the development environment or the target environment by changing a single definition. If using development, the image will be the test image, while if on the target, the image will be the actual image captured from the camera.

You also will need to have a dialog box showing how many frames have been transmitted as well as how many bytes have been sent to the server. On the server, there should be a dialog box showing how many frames have been received and how many bytes have been received.

Part 4: Verification of your system

The final step of this lab is to verify that the operational profiles you defined in the first phase are actually being met. You will need to measure with an oscilloscope how often an image or audio chunk is sent from the client to the server, as well as how long it takes for the display to refresh when a new image is acquired. You'll also need to ensure that the real time performance of the system is met. You'll need to figure out what the processor utilization is for the task which captures the image from the camera, the task which updates the UI, the task(s) which send(s) the images and audio across the network, the task(s) which receive the image and audio and display them from the network, and the task which updates the image on the screen. Using the time utility, you'll also want to measure the overall utilization of the CPU. You will need to verify that the frame rate is being met reliably by the system, or if it is not, indicate how often there is a deviation from the desired performance criteria.

Deliverables / Submission

You may use the optional report template and then save as PDF to upload, or simply ensure that your PDF report contains all of the required information included in the list below.

Each team will be responsible for submitting one report with the following contents:

• Introduction: In three to four sentences, in your own words, start from the broad and work toward the specific, describing what your report/project is about.
• Final Design
  • Describe your class structure and threading choices here. Be sure to reference each figure with something like Fig. 1 somewhere in the text.
  • Rate Monotonic Anaylsis: Describe how you determined each number. (Is it a measurement from a previous lab? Is it computed from measurements? How was it computed?) Insert any tables either in this box, or below it. Is the algorithm guaranteed to be scheduled? Include a table showing the period, execution time, CPU usage, and priority of each task.
  • Describe how the development effort was divided among the various members of the team.
  • Effort Log: Write the number of hours spent by each member of the team. Each member should also write the task that took them the most time, individually
• Testing plan
  • Describe the unit tests each student will perform on their subsystem
  • Write, e.g., a bulleted list of the measurements you plan to make. When measurements in a single line, be sure to clearly specify exactly what parameters you will be varying and what parameters you will be measuring.
• Test Results
  • Describe the results of your tests. Include tables and numeric results, with units, as necessary. Correct any deviations from specification if possible.
  • Include a photo of your working videphone
• Conclusions
  • Demonstrate that requirements have been met, or describe how they were not met
• What we learned
  • What did you learn about real-time systems while working on this project? Two to three sentences are sufficient.
• Comments on the Lab
  • This is required. Enter anything you liked or could be improved about the lab.

In addition to the report, you will upload a SMALL video of your project running and your source code.

If you have any questions, consult your instructor.