SE3910
hopefully already done

This assignment is a team assignment. Please work in teams of two unless approved by the instructor. Spring 2017: I will probably approve, but please ask.

Equipment Needed

  • Your beaglebone black
    • 5 V power supply (with LCD + sound card both drawing extra current, a good idea to use all the time in this lab.)
  • SE3910 Kit (Checkout from Tech Support)
    • USB audio I/O ports (for bone)
    • USB hub (for bone)
    • USB Oscilloscope (Digilent Analog Discovery)
    • Beaglebone Breadboard Cape
    • (optional) Beaglebone Breakout Board
    • Logitech webcam
  • Two male-to-male audio cables
  • (possibly) A few 2 or 2.2 KΩ resistors (Checkout from me if not in kit)
  • (optional) NMap especially zenmap
  • (optional) Two network cables, or more if you want to share

Prelab

I was not able to test this lab this year, due to family needs over break. Although I have reviewed the lab, we may encounter unexpected problems.

There was no opportunity for prelabbing this week.

You may find this tutorial on using Audio with the Analog Discovery helpful. (Tutorial by Dr. Schilling)

Introduction

This week, we want to delve into the raw movement of data to and from the soundcard. By the end of this lab, you will have a good understanding of the processor utilization from sound captures and playback from files.

The data you collect in this lab will be useful for your final project.

Package changes and setup

Before proceeding, make the following changes to your installation.

Virtual Box VM

sudo apt-get install libasound2-dev

Changeroot

apt-get install libasound2-dev

Beaglebone

Make sure you can ping Google, then...

sudo apt-get update

sudo apt-get install time

sudo apt-get install libasound2-dev

Part 1: Getting started

As in past labs, you will need to use the chroot to cross-compile your code for the BBB or compile it on the BBB (Spring 2017: but that probably won't work for Qt programs this year... e.g., in the final project)

To start, you will need an audio source. If you have a headset with a microphone, that will allow you to capture your voice. However, if you do not have a headset, you can use the analog discovery to playback audio wave files as inputs and listen to the output on headphones. If you would like to use a wave file, simply find a wave file and play it back using the instructions in Appendix A. (Note: You can also use clicks.wav if you like.)

For the initial tests, you can simply plug the audio output of the Analog Discovery straight into the audio input of the Beaglebone with the USB addendum. Be sure to plug in the USB audio addendum before booting the bone to ensure it is detected.

The USB audio jacks use the standard colors: pink - mic, lime green - speaker

Capture audio from the sound card and placing it in a file. To do this, download the Provided tar file and copy it to the BeagleBone. (I prefer scp (Secure Copy) for this — see Tips) Change into the Lab7Provided folder and run the ./makeProductsBeagleBone.sh script. (Note: One of the three builds will fail, since the code isn't complete yet, but the other two should work.)

To find the source for the record and playback commands, you can peek into the .sh file you used to compile them. Consider mainRecord.cpp and mainPlayback.cpp and the files they depend on (Look at the #includes and the .cpp files that go along with them). For the code that follows, you may wish to edit mainRecord.cpp and the files it depends on so that you can pass in arguments for the parameters that you will change.

Start with a frame size of 2048 (The provided code may use a different value by default. Be sure to check!) (the compiled-in default) and an audio capture rate of 11025.

(There is a nice discussion of where these rather arbitrary numbers come from in the table in this section of a Wikipedia article., though there is no practical reason at this point that you should need to use these sampling rates that I know of.)

From the BeagleBone, capture 10 seconds of audio using the command

time sudo ./record plughw:1 audioCapture1.bin 10

(The command aplay -l (with a lower-case L) may help to find the right number. A reboot is often needed for the beaglebone to recognize inserted hardware -- it doesn't do plug-and-play.).

This command will capture 10 seconds of audio from the soundcard and store it in raw format in the file audioCapture1.bin. The output of this command will be details about the CPU usage for this program, as shown below.

real    0m10.261s
user    0m0.203s
sys     0m0.203s

Record this information and then playback the file using the command

time sudo ./playback plughw:1 audioCapture1.bin

Again, record the CPU utilization.

With this completed, keep the frame size at 2048 and modify the sampling rates to 8000, 11025, (22050 as above), and 44100 and repeat the same things. What are the CPU utilizations? Once you complete this, keep the sampling rate at 11025 and modify the Frame size to be 512, 2048, 16384, and 32768 16, 32, 64, 128, 256, and the default 512. (This will require a little bit more sleuthing to find in the source.) Plot the results and explain what happens to CPU utilizations the configurations change. (Hint: You might find it easier to use nano and make these changes right on your beaglebone. It also would be wise to work with a partner and log the data into a spreadsheet right as you perform the tests.)

If you see the warning underrun occurred, this means that the program has missed a deadline for delivering audio samples to the speaker. In other words, your code is having trouble keeping up with the audio card. It may help to adjust the parameters to give your program more time between deadlines or less work to do in a given time period.

When performing these tests in Spring 2016 or 2015, I started with a frame size of 16 and a sampling rate of 44100. This gave me the error unable to set hw parameters: Input/output error. I had to restart the beaglebone to get the commands working again. Perhaps I was pushing the sound card too hard?)

Part 2: Creating a loopback system

One of the other things we can do is create a simple loopback application. In this system, your software will take the audio that present on the microphone input, capture it, and then immediately send it out to the headphone output. There will be some delay in this.

Your task, given the code for the capture and playback using the class code, is to implement source code which will do this and then to measure the latency as the frame size is varied from 32 to 1024 occurs. The code is pretty straight forward. You will have two audio interfaces, one of which is a capture device and one of which is a playback device. Your main will read from the capture device and immediately write it into the playback device. Typically this would involve multiple threads. However, this program will just be single threaded.

To measure the latency, you will need to connect up the Analog discovery and use the provided click track. This wave file is a digital metronome set to 120 beats per second with silence in-between. Start this playing in the analog discovery waveform generator and attach the headphone jack of the analog discovery device to the audio input of your capture card (pink).

Connect the channel 1 wire of the oscilloscope to output W1. Connect the audio out of the oscilloscope to the BeagleBone's audio in. (W1 and audio out carry the same signal)

Plug your headphones into the green output of the Beaglebone sound jack and verify that the click track is playing. Then disconnect the headphones and connect a cable from the headphone out jack of the sound card. Plug the other end into an audio jack plugged into your breadboard. Connect this to scope channel 2. Adjust the scope until you can see the click track on the display and note the time difference. This is the latency. Be sure to zoom in a good distance so you can get a good reading, as shown below.

Throughout debugging, you might find it helpful to stick your headphones into the output of the Analog Discovery or of the BeagleBone to see if the signal is coming through.

Things that often go wrong and how to fix them:

  • You see many underrun errors.
    • See Lab8's tip on making sure the frequency of the BBB is 1000MHz (1Ghz).
  • Your latency is huge -- or unpredictable.
    • Make sure that you do NOT buffer 10 seconds of sound. It's OK to use buffers in your program (Indeed, you must), but keep the buffers minimal in size. Don't accumlate 10 seconds of sound and then play them back. Another way of thinking about it: If you enter 1000 as the amount of audio the loopback system should record, it should start playing back immediately, not wait until 1000 seconds are recorded before starting playback. It should seem immediate to the user, even though of course it isn't.
  • Your latency is zero -- the input and output clicks happen at exactly the same time (are perfectly aligned on the scope).
    • This is a problem. Somehow, the audio is getting to the output without going through the Beaglebone. For the lab to work right, audio should be copied into the buffer on the BeagleBone, then copied back to audio output. This takse time. (It takes at LEAST the time to fill the buffer, plus additional time to write the info out of the buffer.) So you aren't really seeing an output of the BeagleBone. You might intead see "cross-talk" a signal jumping from one part of the circut to another, perhaps through ground or through radio waves.
    • Treat this like silence and try some of the other troubleshooting steps here.
  • Your code looks reasonable but no sound is coming out.
    • Are you able to hear the click tracks coming from the Analog Discovery if you plug your headphones into it directly?
    • Are you able to hear the click tracks playing back successfully using the provided playback file?
      • Are you using the same parameters for recording as playback?
      • Does your soundcard work? Try plugging it into your laptop host machine and seeing if you can play sounds with it there.
    • Are you able to hear the clickbacks coming from the BBB when your loopback program is running
    • Are you trying to write and read using the same interface? (Should you be? Read on and reflect.) You can write without error to an input interface, but it won't do anything. Make sure to specify the right constants when declaring your audio interfaces.

With this technique perfected, vary the frame size to the values given previously to 32, 64, 128, 256, 512, and 1024, and in each case, measure the latency. How does it change with frame size?

This figure MIGHT be useful...

Rate Monotonic Analysis

Assume camera capture as in your previous analysis. Select the maximum audio rate that you can achieve while still capturing videos. Complete the Rate Monotonic Analysis, again assuming that there are many tasks running which take 0 time but lower the theoretical CPU usage threshold. Show a Rate Montonic Analysis table including your tasks, the frame rates for video and audio, etc, that you selected, and the total CPU usage... You do not need to do a sample time layout like we usually do after the table on exams.

Add the user and sys times to find the total CPU time for a given process.

Deliverables / Submission

Spring 2017: Lab 7 is due at the start of the Lab 8 Lab Period

Staple the Lab 7 Checklist to the front of your submitted materials. Although you are not required to use the Report Template, you should include all the information required by the sample tables in the report, including the typical real recording time for your experiments.

Appendix A: Playing a wave file in WaveGen

You may find this tutorial on using Audio with the Analog Discovery helpful. (Tutorial by Dr. Schilling)

When you open the Analog discovery, it looks like this.

Select "Play" and then "Import"

Open your click track .wav file.

You should the clicks as horizontal lines. You don't need to edit anything here. Just hit OK.

You will now see random noise in the place of the clicks, but when you run, you should hear the clicks when your headphones are plugged into the audio jack on in the Analog Discovery (on the opposite side from the other cables).

You can keep the default configurations. You do not need to change the frequency to match the sampling rate. In Waveforms (2015), it probably won't match.

The 2015 version of Waveforms does not load the .wav file correctly on Macs or Linux. The current version as of Spring 2017 seems to work. If you are on Mac or Linux, and it is not working, please let me know and find a teammate on Windows who can run the Analog Discovery for you.

Acknowledgement: This lab originally developed by Dr. Schilling.