Introduction

Here it is! The final project! In this lab, you will create a one-way video phone. The project will involve sending video and audio across the network to a second BeagleBone. You will use sockets. You will use real time analysis. And you will have to do some design, as well as verification that your system is working properly.

Because of the increased scope of this lab, and the timing with the end of the quarter, this project may be completed in larger teams of up to 4 people. To ensure maximum learning of all parts of your code, part of your project will be a code review of the code writtten by your teammates. On the final exam, there may be questions related to any portion of the implemented system.

This is also a multiweek lab. The final report will not be due until the end of the quarter.

Initial steps

You will need to install the timing utility on your Beaglebone. To do this, issue the following command:

sudo apt-get install time

This will install a necessary timing utility on your beaglebone.

You may also find it useful to install a virtual keyboard on your Beaglebone. You can do this with the command

sudo apt-get install florence

Basic requirements

The basic requirements of your system are:

• Requirement R1: The video conferencing system shall capture video frames from a web cam on the sending BeagleBone.
• Requirement R2: Video frames will be captured from the webcam shall be shown on the local display.
• Requirement R3: Video frames captured from the webcam shall be sent across the network using a socket to a second machine (The receiving BeagleBone.
• Requirement R4: The system shall capture audio and transmit it one way across the network to a second device.
• Requirement R5: The second machine shall display the video frames on the LCD panel as well as play back the audio that has been captured.
• Requirement R6: The video and audio playback shall be of reasonable quality, given hardware constraints.
• Requirement R7: The system shall integrate OpenCV and the ALSA libraries as is appropriate.
• Requirement R8: Implemented code shall be fully documented with Doxygen comments explaining the purpose and rational for the implementation.

Developing your design (Due in Week 8 Lab

The first part of this project is to sketch your design. Based on the analysis that we have done thusfar this quarter, and the tools you have been shown, sketch a set of UML diagrams showing the various classes that will exist and the main attributes for those classes. For classes that have threads, clearly denote that they have threads.

Second, draw out a sequence diagram which illustrates the major events in your system -- video capture, audio capture, transmitting and receiving data over the network. Include both sending and receiving BeagleBones in the diagram, as well as any multithreading that occurs.

You may find it helpful to use Qt's threads, slots, and signals as discussed in class. Alternatively, while we have not done this thusfar in class, you may find it advantageous to design a generic "Threadable" class that can serve as a parent class for multithreading. (Remember, C++ supports multiple inheritance **Shudder**, so doing it this way will not be as problematic as it might be in Java.)

Once you have completed your design, as a team, you should do a review of the threads to determine their execution rates as well as the required bandwidth. You may find it advantageous to adjust the frame rates and quality to match your needs. For example, lower frame rates may result in lower bandwidth utilization for images.

With the above step completed, you should be able to perform a basic rate monotonic analysis on the system to determine schedulability of the system. (Although we are not using the Rate Monotonic Scheduling algorithm, this is still a good "sanity check") You may need to go back and use the time method on previous labs to get more detailed information on the execution time for various programs. However, most of the data should be available in at least a high level format from your previous labs.

In working on your design, you should pay close attention to the QT UI design. You will likely need to add some additional signals and slots.

Dividing the work (Due in Week 8 lab)

After you have a plan for the key classes in your project, decide how to divide the implementation work among your team members.

Devise unit or component tests (these do not have to be automated) for each class, so each person can test their own work before integration testing. You may find it advantageous, at least for the video portion, to start by sending the images to the virtual machine instead of another Beaglebone.

Please staple the Week 8 Checklist to your materials. I will grade this in lab and return the graded material to you immediately.

Implementing Your Design

Implementing the modules (Due in the Week 9 Lab)

Once you have completed your design, start implementation.

Each student will receive an individual grade for the Week 9 Lab. Each student will demonstrate their working module during this lab period.

As a team, make sure at one member of the team is responsible for each part of the project, and that no parts get missed. A component of the lab grade will be based on whether the whole team has all parts of the project covered (at least in plan). Every member of the team will receive the same grade for this part.

Rate-Monotonic Analysis (Due in the Week 9 Lab)

As a team, create a rate-monotonic analysis for your project. Include a table showing the period, execution time, CPU usage, and priority of each task. Describe how you determined each number. (Is it a measurement from a previous lab? Is it computed from measurements? How was it computed?) Is the algorithm guaranteed to be scheduled, assuming we did use a rate-monotonic scheduler? Only one report needs to be tured in for this part. Every member of the team will received the same grade for this part.

You only need to do a rate monotonic analysis for one of the two BeagleBones. Choose either the sender or the receiver, whichever you think is more likely to be CPU bound. Make clear which one you chose. (Note that "server" and "client" are not clear — some teams choose to make the sender the server, others a client.

Please staple the Week 9 Checklist to your materials. I will grade this in lab if possible and return the graded material to you immediately.

Integrating the modules (Due in the Week 10 Lab)

If you are getting many underflows, you may want to check that the CPU is set to its maximum frequency. On the BB, run the commands cpufreq-info followed by cpufreq-set -f 1000MHz to get the frequency set correctly. (Thanks, Dr. Schilling.)

To link to OpenCV from Qt, please add these lines at the end of your .pro file:

unix: CONFIG += link_pkgconfig
unix: PKGCONFIG += opencv

It is recommended that the first part you integrate is the audio across the network, as that may be the harder one to get right. We may be talking about prioritization in lecture, and may discuss how to handle thread priorities.

Once this is integrated, completed and working, then you can move on to integrating the transmission of the images from the camera, which is the harder problem to solve.

You may find it advantageous to first start by building the image capture program to run on the Beaglebone without an audio channel. This will allow you to capture and show images on your Beaglebone in pseudo real time.

Your GUI must remain responsive while the video is playing. You may want the video producer to execute in its own thread. There are several ways you can accomplish this. The first is to use good old fashion posix threads. You also can use the QT QThread class, which is extremely similar to how Java handles threads. (As we have seen in class. See also http://qt-project.org/doc/qt-4.8/qthread.html and http://qt-project.org/doc/qt-4.8/thread-basics.html#which-qt-thread-technology-should-you-use)

In doing this testing, you should use a conditional compile that will allow the program to be rebuilt for either the development environment or the target environment by changing a single definition. If using development, the image will be the test image, while if on the target, the image will be the actual image captured from the camera.

You also will need to have a dialog box showing how many frames have been transmitted as well as how many bytes have been sent to the server. On the server, there should be a dialog box showing how many frames have been received and how many bytes have been received.

Verification of your system

Final Report (Due 11pm Friday Weeek 10Due in the Week 10 Lab)

You may use the optional report template and then save as PDF to upload, or simply ensure that your PDF report contains all of the required information included in the list below.

Each team will be responsible for submitting one report with the following contents:

• Introduction: In three to four sentences, in your own words, start from the broad and work toward the specific, describing what your report/project is about.
• Final Design
  • Describe your class structure and threading choices here. Be sure to reference each figure with something like Fig. 1 somewhere in the text.
    • Include your final class diagrams
    • Include your final sequence diagrams
    • Make it clear what code runs on each thread, and which threads are on each BeagleBone
  • Spring 2016: Because I haven’t reviewed your first round of Rate Monotonic Feedback yet, you do not need to include the Rate Monotonic Analysis in your report. Rate Monotonic Anaylsis: Complete this as as in the Week 9 lab, correcting any errors noted in that lab report.
    • Describe how you determined each number. (Is it a measurement from a previous lab? Is it computed from measurements? How was it computed?) Insert any tables either in this box, or below it.
    • Answer the question: Is the algorithm guaranteed to be scheduled (assuming a real-time rate-monotonic scheduler)?
    • Include a table showing the period, execution time, CPU usage, and priority of each task.
  • Describe how the development effort was divided among the various members of the team.
  • Effort Summary: Write the approximate number of hours spent by each member of the team. Each member should also write the task that took them the most time, individually. Do include ALL time spent on the lab, including design. (Spring 2016)You don't need to break this down or record detailed logs.
• Testing plan
  • Describe the tests each student performed on their subsystem (Finalized version based on your original sketches).
  • Write how you measure the frame-rate and audio-rate for your final system.
• Test Results
  • Describe the results of your tests. Include tables and numeric results, with units, as necessary. Correct any deviations from specification if possible.
  • Include a photo of your working videphone
• Conclusions
  • Demonstrate that requirements have been met, or describe how they were not met
• What we learned
  • What did you learn about real-time systems while working on this project? Two to three sentences are sufficient.
• Comments on the Lab
  • This is required. Enter anything you liked or could be improved about the lab.

If you are unable to upload your report, you may upload a SMALL video of your project running.

You must upload your complete souce-code. Include all .cpp and .h files. Include any images or other resources that are required for the program to run. The design portion of your report should match the source code, and the source code should match what is demoed in class or the video.

If you have any questions, consult your instructor.