SE3910
Week 6, Monday 11pm

Introduction

In this lab, we are going to build a simple camera which will take pictures when commanded via a pushbutton. The pictures can then be viewed on the LCD display of the Beaglebone.

Prelab

Before coming to lab, complete these steps:

  • Install required packages on the Beaglebone
    • the gpicview image viewer program
    • the OpenCV development libraries
  • Compile boneCV.cpp on the Beaglebone
  • Install the header files on the VM
  • Develop your code for the lab on the Virtual Machine

If you get stuck working on the Beaglebone, move forward and work on the VM.

Install required packages on the Beaglebone

To begin, log into your Beaglebone as root and connect it to the internet (e.g. ping google.com should work). Issue the following commands in sequence to install appropriate libraries (Use sudo for each command if you didn't log in as root):

apt-get update
apt-get install gpicview
apt-get install libopencv-dev

The update command will update the package listing on your Beaglebone to reflect the most current packages which are available. The install commands will install the gpicview image viewer program and the OpenCV development libraries, which are necessary to access the camera.

Compile boneCV.cpp on the BeagleBone

Download the file boneCV.cpp. This program will use openCV to capture an image from the camera and to filter it into greyscale and using an edge detection algorithm, and write each image to a file. Ultimately, we will only be using the image capture and file writing portions of this in our final project, but this shows how simple it is to add computer vision features to your program if you desire to.

Transfer this cpp file to the Beaglebone. You can use ftp to do this, or the command scp boneCV.cpp root@192.168.7.2: (Include that final colon (:) character.) On the BeagleBone, issue the command

g++ -O2 `pkg-config --cflags --libs opencv` boneCV.cpp -o boneCV

If you type this command, be sure to type a letter O instead of a digit 0 for -O2. Also, use the accute accent quotes `...` instead of single quotes '...'

.

This will compile the program on the Beaglebone. After compilation is completed, execute the program. Assuming the camera is not attached, you should see the message

Started Processing - Capturing Image
Failed to connect to the camera.
Failed to capture an image

Installing header files on the VM

Due to some technical issues with the Beaglebone development environment and the currently available cross compilation libraries, we are not able to link our code in the cross compiled environment. We can, however, use Eclipse to write and compile our code locally, getting a great deal of the benefit of the IDE's error-checking abilities. We will, however, be forced to re-compile and link the code on the Beaglebone to create the working executable.

To start, download the tar file lab5update2.tar.gz. This file contains a script and an install set. Place the tar file somewhere reachable by the VM and, from within the VM, unzip it with the command

tar zxvf lab5update2.tar.gz

Now, from the newly created folder lab5, run the script lab5setup.sh as root:

cd lab5

sudo ./lab5setup.sh

This will install the library header files in the appropriate locations for the tools to work properly.

(You can look into this plain-text file to see what the script does, if you are curious.)

Writing and Compiling your code

Once the libraries are installed, you will be developing a camera application. This application will use openCV to capture images and store them to files. You will be able to vary the size of the image captured. For this exercise, you will be capturing png files, though other files can be captured as well.

The program has these basic requirements

  • Requirement R1: When the user presses a pushbutton, the camera shall take a picture and store it into a png file. The user can repeat this as long as desired.
  • Requirement R3: The program shall turn on an LED while the camera is taking the picture.
  • Requirement R4: The program shall turn on a second LED while the actual file write of the image occurs.
  • Requirement R5: The input and output GPIO port numbers (one input switch and the two outputs for R3 and R4 above), and the height and width of images captures shall be accepted as command line arguments
  • Requirement R6: The program shall automatically increment the number of the image file saved as each picture is taken.
  • Requriement R7: When run without parameters, the program shall print out the arguments in the order it expects them.

Your program may take an arbitrarily large number of pictures. It is OK if the user needs to use Ctrl-C to exit.

Create a new project for your program. Make sure to check that you use the workspace located on the shared folder of your VM. Recall that you need to use the prefix arm-linux-gnueabihf- and the path /usr/bin. I put these in a "sticky note" on my VM's desktop for my convenience.

In addition to the standard setup for cross-compiling, we need to explicitly add an include path. In Eclipse, right-click on your project and select "Properties." Under C/C++ General, select Paths and Symbols. Under Includes, select GNU C++. Click on add and paste /usr/i586-linux-gnu/arm-linux-gnueabihf/include. Hit OK.

If you prefer to compile from the command line instead, include the option -I /usr/i586-linux-gnu/arm-linux-gnueabihf/include in your cross-compiling g++ command.

You will need GPIO.h and GPIO.cpp from Lab 3 or Malloy.

Below is a template UML diagram. Your code does not need to match this exactly, but this is a good starting point for your system design. You will want to perform your setup for the camera once in the constructor, and then simply capture frames and write them out in the takePicture function. Note that the box "main" is not a class. It is a file holding the main method.


The material beyond this point is in DRAFT form. Although I am not making major changes to the lab, some commands may not work exactly in the form given here.


This assignment is a team assignment. Please work in teams of two unless approved by the instructor.

Reconfiguring BeaglebBone to enable touch-screen

To enable the touch-screen on the BeagleBone, we need to uncomment a line in a configuration file. (Without this edit, the screen will simply show as a blank white screen.

Log into the beaglebone as root. Edit the file /boot/uEnv.txt, e.g., with

nano /boot/uEnv.txt

In the file, uncomment the line to read (see the red arrow in the figure below) 

##BeagleBone Black: HDMI (Audio/Video) disabled:
dtb=am335x-boneblack-emmc-overlay.dtb

Now turn off the Beaglebone and proceed to the next step, e.g. with

shutdown now

Or by holding down the power button on the BeagleBone

Connecting the LCD and prototype board

If you plug in anything backward in the next part, it has the potential to damage your BeagleBone or the LCD cape.

Unplug the power to your Beaglebone. Plug the LCD into the breakout board such that the outline on the bottom of the LCD lines up with hole in the breakout cape. (The breakout board is the board with two beaglebone-shaped holes and lots of pins.) Plug the USB camera into the BeagleBone. Plug the beaglebone into either one of the holes in the Breakout cape from underneath. Orient the BeageBone so it lines up with the hole in the cape as well. Push up gently on the BeagleBone. You don't have to push it up all the way; the more gently you put it on the easier it will be to take off without bending the pins. Put the basic protoc cape or the breadboard cape on top of the other hole in the breakout cape, again, aligning it with the hole.

Plug in the Beaglebone through USB and shortly after that, plug in the power adapter. The power adapter is needed because the LCD display draws more power than some USB ports supply. If you don't use the power supply, and your BeagleBone mysteriously reboots, it probably had a brown-out reset. Use the power supply to avoid this.

Once your LCD is plugged in, you can use the power switches on the right side of the LCD to turn off the BeagleBone. Watch out! The Reset switch on the LCD board will also immediately stop the OS without a clean shutdown, just like the one on the main board.

Boot up your Beaglebone. You should automatically see the touchscreen turn on. You may wish to do the calibration, or just let it time out. Then you should see a Linux desktop come up on the LCD screen.

Log into the BeagleBone as root. Issue the command lsusb 

Bus 001 Device 002: ID 046d:082d Logitech, Inc. HD Pro Webcam C920
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

The Logitech device is the webcamera.

(The camera doesn't automatically show up unless you reboot the beaglebone. Feel free to try. I did.)

Run the boneCV program that you compiled in the prelab.

You will see that three images are created. Open them in gpicview using the touchscreen. (When plugged in, the touchscreen automatically opens to /home/debian/Desktop, without needing to enter any login info. Once there, you can easily navitage to any subfolder of the Desktop. Double-click on an image to open it with gpicview.) Examples are shown in the Figure below

Compiling and Linking your code

You will be able to compile your code locally. However, as was stated previously, libraries make it difficult to link your code in a cross compiled environment. To get around this, create a directory on your Beaglebone for your code to reside (e.g. /home/debian/Desktop/camera) and transfer it via ftp or scp to your Beaglebone.

To do this, open up a shell on your Linux Virtual Machine (VM) and issue the command

sftp debian@beaglebone.local

to connect to the Beaglebone. Once connected, issue the command cd camera to change into the remote camera directory and then put * to transfer all of the files from the local machine to the remote machine.

(Alternatively, you can use a command like scp *.cpp *.hpp debian@192.168.7.2:Desktop/camera/ if you have Cygwin installed, or from the VM if you have a network connection from the VM to the Beaglebone.)

Now open up an ssh connection to the Beaglebone. Change into the directory containing your code and issue the command

g++ -O2 `pkg-config --cflags --libs opencv` *.cpp -o camera

to compile and link the program on the Beaglebone.

(If you have .cpp files that you don't wish to compile, you will have to name all the cpp files that you DO want to compile instead of just using *.cpp.)

Measuring how long it takes to snap a photo and how big they are

As a last piece, we want to know how long it takes for the photo to be snapped. Connect your two oscilloscope channels to the two LED pins and measure how much time it takes for a picture to be taken, and how long it takes for the picture to be written to disk. Measure the times for various sizes and plot the relationship between size (i.e. number of pixels) and the time it takes to take the picture, the time it takes to store the picture, and the size of the picture in bytes. Make sure to vary your picture sizes from 320 * 240 to 1920 * 1080.

After you are done with this, trying changing the picture format to jpg and repeat the experiment. Is there a difference?

Deliverables / Submission

Use the Lab 5 Checklist. I will bring one copy for each team.

(Spring 2016) You do not need to print your source code.

Appendix: Taking a snapshot of your virtual machine

Once you snapshot your VM, all further changes will be stored on your C drive. Should your C drive need to be reimaged, you will lose all changes beyond the snapshot. I speak from experience. -- Dr. Yoder

If you take a snapshot of your virtual machine, and you later have problems, you can easily recover the machine back to the previous state. To do this, select the snapshots tab in the upper right and click on the "take snapshot" icon that appears to the left. Provide a meaningful name for the snapshot and a description prior to pressing OK. This will take a few seconds, but now it is possible to revert back to the previous state if something goes wrong.

If you have any questions, consult your instructor.

Acknowledgement: This lab originally developed by Dr. Schilling.

Due: Week 6, Monday, 11pm