1

UNIVERSIDAD DE Práctica Nº 5
SAN BUENAVENTURA Diseño de Sistemas Embebidos
CALI Profesor: Jose Daniel Bolaños J.

Audio CODEC

Introduction

The purpose of this exercise is to learn how to use the Audio enCOder/DECoder (CODEC) on an Altera
DEseries Board. We will connect a microphone and speakers to the DE-series board to allow recording and
playback of sounds. Besides, we will generate echo effect on the sound data, as well as increase and decrease
the volume.

Background
Sounds, such as speech and music, are signals that change over time. The amplitude of a signal determines
the volume at which we hear it. The way the signal changes over time determines the type of sounds we hear.
For example, an “ah” sound is represented by a waveform shown in Figure 1.

Figure 1. A waveform for an “ah” sound.

The waveform is an analog signal, which can be stored in a digital form by using a relatively small number of
samples that represent the analog values at certain points in time. The process of producing such digital
signals is called sampling. The points in Figure 2 provide a sampled waveform. All points are spaced equally
in time and they trace the original waveform.
 Figure 2. A sampled waveform for an “ah” sound.

To be able to record and play back sounds we will use the DE-2 board Audio Core. In this case, we also have
to process sound data to perform echo effect, then, we won’t use on-chip memory anymore, because we need
more memory capability. Taking into account the above, we will use a SDRAM memory to store the sound
data and process it after. Figure 3 shows a portion of the Nios II embedded system for audio record and play
back.

Figure 3. Nios II embedded system for audio record and play back. (source: Altera Corporation)
Audio Core
The Audio core supports both audio input and output simultaneously. It provides a link between a program
executed by the Nios II processor and the Audio CODEC connected to the external microphone and speaker
system, as shown in Figure 4. The Audio core contains four FIFO buffers for the In and Out audio data, both
having the right and left audio channels. Each FIFO can store up to 128 words (32-bits each one).

Figure 4: The audio subsystem. (source: Altera Corporation)

Sound can be recorded by the system using a microphone. The microphone converts the sound into an analog
waveform, such as the one shown in Figure 1. The CODEC samples the waveform every 1/48000th of a
second, and stores the sample in the Record FIFOs. The Record FIFOs can then be read by the processor. To
produce an output waveform, the Audio Port accepts a sampled waveform as an input from the processor
and stores it in the Playback FIFOs. The CODEC then converts the data in the FIFOs into an analog waveform
and sends it to the speakers.

Register Map
User can control and communicate with the Audio Core through four 32-bit memory-mapped registers. By
writing or reading these registers, data can be fetched from the CODEC’s Analog-Digital Converter (ADC) or
sent to the Digital-Analog Converter (DAC). Table 1 shows the format of the registers.
The Control register contains six bits that are used to control the Audio Port:

 RE bit enables the Audio core to generate an interrupt request when either of the Record FIFO is filled
75% or more.
 WE bit enables the Audio core to generate an interrupt request when Playback FIFO is less than 25%
full.
 CR and CW bits clear the Record and Playback FIFO buffers, respectively.
 RI Indicates that a read interrupt is pending.
 WI Indicates that a write interrupt is pending.

The fifospace register:

 WSLC and WSRC indicate the number of words available (i.e., the amount of empty space) for
outgoing data in the left and right channel FIFOs, respectively. When the outgoing FIFOs are empty
WSLC = WSRC = 128.
 RALC and RARC indicate the number of words of incoming audio data in the left and right channel
FIFOs, respectively. When the outgoing FIFOs are empty RALC = RARC = 0.

The leftdata and rightdata registers:

 The leftdata and rightdata registers are used either to load data into the Playback FIFOs (left or
right) by writing to the registers, or to retrieve sound from the Record FIFOs by reading these
registers.
 Users should take care to check the state of Record and Playback FIFOs before reading or writing to
leftdata and rightdata registers. If the Record FIFOs are empty, then the contents of the leftdata and
rightdata registers are invalid. Also, if the Playback FIFOs are full, then writes to the leftdata and
rightdata registers will be ignored by the Audio core.

Example:
The program below demonstrates the use of the Audio core. This program reads audio samples from Record FIFOs
and writes the same samples to Playback FIFOs.

#include <stdio.h>
#include "system.h"

int main()
{
volatile int * audio_addr = (int *) AUDIO_BASE;
int fifospace, leftdata, rightdata;

while(1){

fifospace = *(audio_addr + 1);

if ( (fifospace & 0x000000FF) > 0 ) {

leftdata = *(audio_addr + 2);
rightdata = *(audio_addr + 3);
*(audio_addr + 2)=leftdata;
*(audio_addr + 3)=rightdata;
}
}
return 0;
}
Remember, if you want you can access the device registers directly using the Altera-provided HAL device
driver. In order to do the above, you have to include the follow library in your C code:

 #include "system.h"
 #include "altera_up_avalon_audio.h"

These files define the core's register map, providing symbolic constants to access the low-level hardware.

Audio/Video Configuration Core

The audio and video peripherals on the DE-series boards require configuration via 2-wire and 3-wire serial
buses. The Audio/Video Configuration core provides a convenient way for configuring and initializing these
devices. In this exercise we will use the Audio/Video Configuration core which automatically configures and
initializes both video and audio ports whenever the Nios II embedded system is reset.

Part I

Creating Nios II Hardware System using Qsys

In this part, you have to include some blocks in your NIOS II embedded system using Qsys in order to allow
recording and playing back sounds. In this case, you have to reuse the system you designed in lab 3. Taking
in to account the above, you only have to include the following cores.

1. Instantiating the Audio/Video Configuration Core

The Audio/Video Configuration IP core is used to set up the audio and video peripherals on the DE-series
boards. To set up a peripheral device, the core takes information from a program (via the Avalon bus) or the
auto-initialization circuit and sends it via a serial bus. The Audio/Video Configuration should be used to
configure the on-board audio and video chips, as well as the 1.3-megapixel digital camera (TRDB_DC2), the
5-megapixel digital camera (TRDB_D5M) and LCD with touchscreen (TRDB_LTM) daughter-cards.

The Audio/Video Configuration core must be instantiated once for on-board peripherals and once for each
of the daughter cards used. The following configurations are available.

Components

 Audio/Video Device — allows users to specify the target peripherals. Each option represents a distinct
serial bus. The On-Board Peripherals option is for selecting the audio and video chips on the DE-series
boards, while the other options are for selecting the various daughtercards.
 DE Board— allows users to specify the target board.
 Auto Initialize Device(s) — allows users to add hardware to the core that will configure the target
peripherals for the default operation required by other Altera University Program IP cores, which
interact with those peripherals.
Auto Initialization Parameters for Audio
enabled when On-Board Peripherals and Auto Initialize Device(s) are both selected.

 Audio In Path - allows users to choose the microphone or the Line In as the input device for the ADC.
 Audio Out - Enable DAC Output—enables data from the DAC to pass to the Line Out.
 Audio Out - Microphone Bypass—If enabled the signal from the microphone input jack bypasses the ADC
and DAC, and goes directly to the output jack.
 Audio Out - Line In Bypass—If enabled the signal from the Line In input jack bypasses the ADC and DAC,
and goes directly to the output jack.
 Data Format - allows users to choose the data format as DSP Mode, I2S Format, Left Justified or Right
Justified. If using Altera’s UP Audio core, then the Left Justified mode must be selected.
 Bit Length - allows the user to specify the number of bits of each audio sample. Valid values include 16,
20 and 24 bits for all modes, and 32 bits for I2S and Left Justified modes only. Both ADC and DAC operate
on 24-bit data, so zero-padding or stripping of the least-significant bits happens when the bit length is
not 24. Refer to the Device Operation Section in the Audio CODEC Datasheet for details.
 Sampling Rate - allows the user to select the number of audio sample per second. Based on the selected
rate, the audio chip’s Base-Over Sampling Rate and Sample Rate control registers will be set
appropriately, as per Wolfson WM8731 Audio CODEC datasheet.

Auto Initialization Parameters for Video

enabled when On-Board Peripherals, Auto Initialize Device(s) and a DE-series boards are all selected.

 Video Source Format— allows users to specify the video source as either NTSC or PAL.

 See the Wolfson WM8731 Audio CODEC datasheet for more information about the audio chip’s
parameters. It is attached to this guide lab.

 Altera strongly recommends that users configure and initialize the audio and video devices with the
auto-initialize option, and the default settings. If the audio input comes from the microphone instead of
Line In, users should select Microphone to ADC in the Audio-In Path. Also, users may want to adjust the
Line In Bypass or Mic Bypass settings, as well as, the Bit Length.

The selected configuration of this core for this example is shown in Figure 5.

Figure 5. Default settings of the Audio/Video Configuration core on the DE2 board
2. Instantiating the Audio Core

Look for “Audio” in the component library and specify the desired features. In the configuration wizard, you
can choose the mode of the Audio Core by selecting Audio Out and/or Audio In. In addition, the Data Width
per Channel can be specified. Data widths of 16, 20, 24, and 32 bits are supported. Remember to export the
external interface connection to connect the core with the DAC and the ADC. It is recommended to set the
Avalon Type to Memory Mapped if you are working on Qsys. The Audio core’s configuration is shown in
Figure 6.

Figure 6. Audio Core configuration.

 When using the Memory Mapped Avalon Type in Qsys, Altera recommends that the Audio Core be used
with the standard or fast versions of the Altera Nios® II processor, so that a program running on the
processor can keep up with the generation of audio data. If the economic version of the processor is
used, then the program may run too slowly, and the audio may not be clear. In such, cases, it may be
possible to improve the audio clarity by selecting a lower sampling rate in the audio chip.

3. Add the SDRAM Memory

Our system has to be able to record and play back sounds. Besides, it has to process sound data to generate
echo effect, then, we won’t use on-chip memory anymore, because we need more memory capability. Taking
into account the above, we will use a SDRAM memory to store the sound data and process it after.

Look for the “SDRAM Controller” and configure it as is shown in Figure 7. You also have to disable the On-
chip memory and connect the s1 port of the SDRAM memory to both the data_master port and
instruction_master port of the Nios II processor. Then, Double-click the Nios II processor to open the Nios
II Processor parameter editor. Finally, under Reset Vector and Exception Vector, select the SDRAM memory
(See Figure 8).
 Figure 7. SDRAM Controller configuration.

Figure 8. Nios II processor’s Reset Vector and Exception Vector.

4. Add External Clock Signals for DE Board Peripherals

You must also instantiate the External Clocks for DE Board Peripherals core and choose the proper audio
clock and SDRAM clock settings for the Audio Core and SDRAM Controller, respectively. For Wolfson
WM8731 audio CODEC the audio clock has to be set to 12.228 MHz.

Look for “Clock Signals for DE Board Peripherals” core and configure it as is shown in Figure 9. This core
receives two clock signal inputs, then you have to create another clock signal with a frequency of 27Mhz.
Finally, you have to make all the connections and export the respective external conduits as is shown in
Figure 10.
 Figure 9. Clock Signals for DE Board Peripherals configuration.

Figure 10. Nios II Embedded system for Audio Processing

Finally, remember to assign appropriate IRQs, assign base addresses, save the system with the name
System_Audio, and generate the Qsys system. Please, be sure there is not errors in this last step.
5. Integrate the Qsys System into the Quartus II Project and Download
Hardware Design to Target FPGA

Now, you have to create the new inputs in your Verilog HDL file according with the new components that
you added in Qsys system. After you have concluded this step, you should have similar lines like the lines
shown below.

module DE2_Hardware(

// Inputs
CLOCK_50,
CLOCK_27,
//----PIO-----
//Inputs
KEY,
SW,
// Outputs
LEDG,
LEDR,
//---
HEX0, HEX1,
HEX2, HEX3,
HEX4, HEX5,
HEX6, HEX7,
//------------
//---AV Config--------
//Inputs
I2C_SDAT,
//Output
I2C_SCLK,
//--------------------
//---Audio-----
//Inputs
AUD_ADCDAT,
//Bidirectional
AUD_BCLK,
AUD_ADCLRCK,
AUD_DACLRCK,
//Outputs
AUD_XCK,
AUD_DACDAT,
//--------------
//----Memory (SDRAM)----
//Bidirectional
DRAM_DQ,
// Outputs
DRAM_ADDR,
DRAM_BA_1,
DRAM_BA_0,
DRAM_CAS_N,
DRAM_RAS_N,
DRAM_CLK,
DRAM_CKE,
DRAM_CS_N,
DRAM_WE_N,
DRAM_UDQM,
DRAM_LDQM
//------------------

);
/*****************************************************************************
* Port Declarations *
*****************************************************************************/
// CLOCKS
input CLOCK_50;
input CLOCK_27;

//---PIO-------
//Inputs

input [3:0] KEY;

input [17:0] SW;

//Outputs

output [8:0] LEDG;

output [17:0] LEDR;

output [6:0] HEX0;

output [6:0] HEX1;
output [6:0] HEX2;
output [6:0] HEX3;
output [6:0] HEX4;
output [6:0] HEX5;
output [6:0] HEX6;
output [6:0] HEX7;
//-------------------------

//----AV Config----------
//Bidirectional
inout I2C_SDAT;
//Outputs
output I2C_SCLK;
//------------------------

//-----Audio-----
//Inputs
input AUD_ADCDAT;
//Bidirectional
inout AUD_BCLK;
inout AUD_ADCLRCK;
inout AUD_DACLRCK;
//Outputs
output AUD_XCK;
output AUD_DACDAT;
//--------------

//----Memory (SDRAM)----
//Bidirectional
inout [15:0] DRAM_DQ;
// Outputs
output [11:0] DRAM_ADDR;
output DRAM_BA_1;
output DRAM_BA_0;
output DRAM_CAS_N;
output DRAM_RAS_N;
output DRAM_CLK;
output DRAM_CKE;
output DRAM_CS_N;
output DRAM_WE_N;
output DRAM_UDQM;
output DRAM_LDQM;
//------------------
Now, you have to instantiate the module generated by Qsys into the Quartus II project. For this step,
remember that Qsys tool generates a Verilog module that defines the Nios II embedded system. In your
design, this module will have been generated in the System_Audio.v file, which can be found in your project
path. Be sure to connect inputs and outputs of all the system components to the appropriate pins on the FPGA
device. After you have concluded this step, you should have similar lines like the lines shown below.

System_Audio u0 (

.clk_27_clk (CLOCK_27),
.clk_50_clk (CLOCK_50),
.reset_reset_n (KEY[0]),

.pin_hex_3_0_export ({1'b1,HEX3[6:0],1'b1,HEX2[6:0],1'b1,HEX1[6:0],1'b1,HEX0[6:0]}),
.pin_hex_7_4_export ({1'b1,HEX7[6:0],1'b1,HEX6[6:0],1'b1,HEX5[6:0],1'b1,HEX4[6:0]}),
.pin_red_led_export (LEDR),
.pin_green_led_export (LEDG),
.pin_keys_export (KEY[3:1]),
.pin_sw_export (SW[7:0]) ,

.pin_audio_external_ADCDAT (AUD_ADCDAT),
.pin_audio_external_ADCLRCK (AUD_ADCLRCK),
.pin_audio_external_BCLK (AUD_BCLK),
.pin_audio_external_DACDAT (AUD_DACDAT),
.pin_audio_external_DACLRCK (AUD_DACLRCK),
.audio_clk_clk (AUD_XCK),

//-------------------------------
.pin_audiovideo_config_SDAT (I2C_SDAT),
.pin_audiovideo_config_SCLK (I2C_SCLK),
//--------------
.pin_sdram_wire_addr (DRAM_ADDR),
.pin_sdram_wire_ba ({DRAM_BA_1,DRAM_BA_0}),
.pin_sdram_wire_cas_n (DRAM_CAS_N),
.pin_sdram_wire_cke (DRAM_CKE),
.pin_sdram_wire_cs_n (DRAM_CS_N),
.pin_sdram_wire_dq (DRAM_DQ),
.pin_sdram_wire_dqm ({DRAM_UDQM,DRAM_LDQM}),
.pin_sdram_wire_ras_n (DRAM_RAS_N),
.pin_sdram_wire_we_n (DRAM_WE_N),
.sdram_clk_clk (DRAM_CLK)

);

Next, save the Verilog HDL file, then Select Project > Add/Remove Files in Project, and select only the files
related with your project. Click on Apply and finally click OK.

Select Processing > Start > Start Analysis & Synthesis, or click on the icon in order to check your code and
prepare for assigning pin locations. When the analysis and synthesis processes have finished, click OK.

Assign FPGA pin locations to match the pinouts of your board. For this purpose, you must know the pin layout
for the board. To complete this section, you can refer to the documentation for your board, in this case the
DE2 User_Manual, in order to find the appropriate pins for each port.

Finally, compile your project by clicking in the icon and download the hardware design to the target
FPGA by clicking in the icon .
6. Developing Software Using the Nios II SBT for Eclipse

In this section, you have to compile a simple C language program. Use the Hello World template to ensure
that the communication with the hardware system is correct. Then, modify Hello world template to perform
the follow:

Part 1 a)
In this part of the exercise you will create a 5-seconds recording and play it back. To do so, a microphone and
speakers need to be connected to the DE-series board. Speakers should be connected to the green jack,
labeled LINE-OUT, and the microphone should be connected to the pink jack labeled MIC.

You have to write a C program that uses the Audio Port to record and play back sounds. Your program should
read the data from the Audio Port and store it in memory when the user presses the KEY1 pushbutton. The
recording should be played back by writing samples stored in memory to the Audio Port when the user
presses the KEY2 pushbutton.

The playback volume of the recording should also be controlled by slider switch SW0 and the pushbutton
KEY3. The switch SW0 control de direction of volume: when SW0 is 1, a push of KEY3 increases volume;
when SW0 is 0, a push of KEY3 decreases volume. The volume must go from 0 to 9 and it have to be displayed
on a 7-Segment Display (HEX0).

Also, each pushbutton KEY1-3 as well as SW0, must display its state on a LED. LEDG for pushbuttons and
LEDR for the switch. So if the system is recording, a LEDG must be ON throughout. If the system is on
playback, a LEDG must show so. The LEDG must be ON while the routine is running, and it has to maintain
its state until process is finished, in that moment LEDG must turn OFF.

On the HEX1 display you must show the time of the playback or the recording in seconds. If the system is in
playback, the display must count up. If the system is in recording mode, the system must count down.

To perform the above, you can take the next code as an example. Remember it is just an example, you have
to change or add the respective lines in order to reach the asked requirements.

//-------------------------------------------------------------------------------------------------
#define BUF_SIZE 96000 // about 2 seconds (48K samples/sec)

int main(void)
{

volatile int * AUDIO_ptr = (int *) AUDIO_BASE_ADR;

volatile int * GREEN_LEDS_ptr = (int *) GREEN_LEDS_BASE_ADR ;
volatile int * RED_LEDS_ptr = (int *) RED_LEDS_BASE_ADR ;
volatile int * KEYS_ptr = (int *) KEYs_BASE_ADR;

/* used for audio record/playback */

int fifospace, leftdata, rightdata;
int buffer_index = 0;

/* reserve memory for audio buffers */

int left_buffer [BUF_SIZE];
int right_buffer [BUF_SIZE];
/* record and playback audio data */
while(1)
{
if (*(KEYS_ptr + 0x3) & 0x1) {

while (buffer_index < BUF_SIZE)

{
fifospace = *(AUDIO_ptr + 1);

if ( fifospace & 0xFF ){

left_buffer[buffer_index] = *(AUDIO_ptr + 2);

right_buffer[buffer_index] = *(AUDIO_ptr + 3);
++buffer_index;

//Here you have to check Buffer ending and restart buffer_index

//record until buffers are full */
}
}
*(KEYS_ptr+0x3) = 0x0;

} else if(*(KEYS_ptr + 0x3) & 0x2) {

while (buffer_index < BUF_SIZE){

fifospace = *(AUDIO_ptr + 1);// read fifospace register

// Here you have to check WSRC spaces

//Here you have to play back audio samples from buffers

//Here you have to check Buffer ending and restart buffer_index

}
*(KEYS_ptr+0x3) = 0x0;
}

// Here you have to check if KEY3 was pressed and change the volume according with SW0

}
}

//-------------------------------------------------------------------------------------------------

Part 1 b)

Write a C program to create an echo effect with the sound signal. Echo is an effect we often hear in large halls
or caverns, where a sound we make comes back to us with some delay. This effect occurs because the sound
bounces off the walls and returns to us. The reflected sound is somewhat attenuated.

You can achieve this by writing a C program that reads sounds from the microphone and produces the sound
back through the speakers. However, moments later the same sound should repeat, at a lower volume, and
again a few moments after that to simulate the effect of an echo. The echo effect should be superimposed on
(added on top of) any sound being provided through the microphone. For example, if you continuously
record audio samples form the microphone in a circular buffer, you can simulate the echo effect by adding
three samples in the buffers as illustrated in the figure below.
 Figure 11. Illustration of a buffer to support an audio echo effect (source: United States Naval Academy (USNA))

References
Edition of this lab guide was based on the next documents from Altera Corporation:

 Altera, “Laboratory Exercise 8 – Embedded Systems”, 2011

 Altera, “Embedded Peripheral IP User Guide”, June 2011.
 Altera, “Audio/Video Configuration Core for DE-Series Boards”, May 2013.
 Altera, “Clock Signals for Altera DE-Series Boards,”, May 2013
 Altera, “Audio Core for Altera DE-Series Boards”, April 2014.

Other Sources:

 “Lesson 08: Audio Interface”, Microcomputer Interfacing (EC463). United States Naval Academy
(USNA).