Correction of the Fundamental Voice Frequency Using

Tactile and Visual Feedback

Rodrigo Leone Alves

Carlos M. G. S. Lima

Raimundo C. S. Freire

Department of Computing at the

Federal Institute of
Rio Grande do Norte IFRN
+55 83988447385

Department of Industrial
Electronics at the University
of Minho - UMinho
+ 351 929256584

Department of Eletrical Engineering

at the Federal University
of Campina Grande UFCG
+55 996599440

rodrigo.leone@ifrn.edu.br

clima@dei.uminho.pt

rcsfreire@dee.edu.br

ABSTRACT
People who are deaf or hard of hearing require correction of the
fundamental frequency that usually is achieved by help of a
phonodiologist. The work of the phonodiologist can be however
strongly reduced by using appropriate devices that can help the
patient to train the voice positioning. This paper reports the
development of such a system based on FPGAs technology that
providing tactile and visual feedback. Contrary to the feedback
from the phonodiologist that is usually relative, the feedback
provided by a device can be provided in absolute terms. This
feature can improve the learning process that however needs the
initial learning provided by the phonodiologist.

Categories and Subject Descriptors

K4.2 [Social Issues]: Assistive technologies for persons with
disabilities

General Terms
Reliability, Experimentation, Human Factors and Verification.

Keywords
Deafness, speech, Assistive technology, tactile stimulation and
visual stimulation.

1. INTRODUCTION
Tact is being explored as a substitute the other senses, such as
sight and hearing a long time. However, due to the development
of electronic circuits and psychophysical studies only in the early
twentieth century that there was a greater increase in research in
this area [1]. The application of tactile stimuli as a means of
transmitting acoustic information for people with hearing
impairments was investigated by [2] that surveyed the use of
vibration as a means of information transfer. From then, it has
seen many haptic devices that assist the hearing impaired [3] [4].
Deafness causes changes in vocal quality, the level of awareness
of the listeners [5]. Such a disorder in speech is noticeable when
the corresponding acoustic signal is observed. Therefore, they can
be monitored and treated by techniques based on acoustic analysis
of voice signal. In that review of signs of voices hearing impaired,
various methods have been studied and employees. Due to the fact
that the fundamental frequency levels in the hearing impaired are
generally changed, this is the first point of challenge when it is
spoken in improving the voice quality of these individuals, since
the auditory monitoring of these is modified. Thus, some
techniques are based on measures of the fundamental frequency

(frequency of vibration of the vocal folds), as jitter (disturbances

in frequency) and Shimmer (disturbances in amplitude) [6] [7].
This paper proposes a device for the detection and measurement
of the fundamental frequency of the voice using a real-time
system based on FPGA1, for further feedback through tactile and
visual stimulus. The goal is to develop a portable device to help
people with hearing loss achieve real-time fundamental frequency
of correct voice.
Concerning the correction of the fundamental frequency of the
voice through visual and tactile stimulation using the Hardware
Description Language VHDL2, this proposal aims to develop new
ideas and build new knowledge in broad areas and little explored
as reconfigurable computing combined with Assistive
Technology3. Thus check the possibility of joining in a single
device the advantages of the hardware and software in a compact
form.

2. METHODOLOGY
The development was done using the Quartus II Web Edition 13.0
consisting of a programming environment, which can be used in
all phases of the design FPGA's, since it performs the steps
synthesis, placement, and routing logic design to be synthesized.
Incremental compilation, which is a configuration of the process
of transforming the source program into an object program that
serves to preserve the results of blocks that have not changed
during the project was used. This tool allows the description of
the code validation synthesis netlist generating (converted cells
are to be set), mapping and routing as well as generating the
bitstream, which constitutes the processing step of the conceptual
model of a digital circuit ready for is released into the FPGA. The
project was set in the Development Kit kit Mercurio IV, the
ALTERA manufacturer, this one comprises a EP4CE30F23C7
Cyclone IV FPGA Family And this has 30,000 logic elements and
an input clock of 50 MHz.

Field-Programmable Gate Array

It is a language for describing digital systems used universally. VHDL

is from VHSIC Hardware Description Language and consists of a
language with emphasis on Integrated Circuits very high speed.
3

Assistive technology (AT) is a relatively new term used to identify all

arsenals of resources and services that contribute to promote or expand
functional abilities of people with disabilities and thus providing more
independence and social inclusion.

For pitch extraction step was done using algorithms used in

previous works. At this stage the relevant parameters are
calculated for the proper processing, such as LPC, cepstral
coefficients, the fundamental frequency and the storage of
reference data. Such algorithms required for processing were
developed in VHDL. To consolidate the verification of the
functionality of the device volunteers were used. The samples
collected from the voice signals in individuals with hearing
impairments and hearing, by performing a comparative study.

3. RESULTS
After two months of testing, the sentences pronounced by the
patients were recorded. Because it is a subjective assessment of
voice quality improvement of the results were compared with the
same sentences recorded before any use of visual and tactile
feedback system.
Twenty volunteers have rated the sentences pronounced before
and after the test and made a quantitative assessment of improved
voice quality, according to the rating scale of Table 1.
Table 1. Quality Improvement of Quantitative Evaluation
Vocal
Comparison of the pronunciation before and
after test sections

Rating scale

No Change

Little change

Good Change

Much Change

56%

30%
22%
11%

11%

0%
No Change

Li le Change

6. REFERENCES
[1] Geldard, F. A., Some neglected possibilities
communication. Science, 131:15831588, 1960.

of

[2] Gault, R.H. Progress in Experiments on Tactual

Interpretations of Oral Speech. Journal of Abnormality
Society Psychological. V19, 1924.
[3] BERNSTEIN, L. E., The Evaluation of Tactile Aids (chapter
8), in Tactile Aids for The Hearing Impaired, Ians Summer
(ed), Whurr Publishers, London, 1992.
[4] WEISENBERGER, J.M. Communication of the Acoustic
Environment via Tactile Stimuli (chapter 4), in Tactile Aids
for The Hearing Impaired, Ians Summer (ed), Whurr
Publishers, London, 1992.
[5] GODINO-LLORENTE, J. I.; GOMES-VILDA, P.;
BLANCO-VELASCO, M. Dimensionality reduction of a
pathological voice quality assessment system based on
gaussian mixture models and short-term cepstral parameters.
IEEE Transactions on Biomedical Engineering, vol. 53(10),
pp. 19431953, 2006.

[8] RAMOS S. Anlise prosdica da fala do deficiente auditivo:

parmetros de durao e freqncia fundamental. Master
Dissertation. So Paulo: Pontifcia Universidade Catlica de
So Paulo; 2000.

40%

10%

The authors thank all who contributed to the work. In particular

the authors wish to thank the University of Minho and Federal
Institute of Rio Grande do Norte - IFRN.

[7] LEJSKA M. Voice field measurements: a new method of

examination: the influence of hearing on the human voice. J
Voice 2004;18(2):209-15.

60%

20%

5. ACKNOWLEDGEMENTS

[6] ARAUJO, A. M. L. Jogos Computacionais fonoarticulatrios

para Crianas com Deficincia Auditiva. Doctoral Thesis.
Campinas: Universidade Estadual de Campinas; 2000.

The results of the evaluators are shown in Figure 1.

50%

proposed system can be used as a portable device for real-time

correction of the fundamental frequency of the voice in the deaf;
(5) Experiments will be conducted in hearing people who work
with the voice and need to maintain a certain frequency of voice,
such as radio and TV journalists.

Good Change

Much Change

Figure 1. Voice quality assessment after use of the system

4. CONCLUSIONS
From the results, the following conclusions can be drawn: (1) In
the majority of cases there was a change in the perception of voice
quality; (2) In the case where there was no perceptible change the
individual had profound pre-lingual deafness. This specific case
needs more investigation to be concluded about the possibility of
improvement with the use of the frequency correction system; (3)
The tactile feedback together with the visual feedback provides
the user more quickly reaches the target frequency; (4) The

[9] LOPES, D. C. J.; OLIVEIRA, I. B.; C., M. S. . Avaliao da

voz e fala de surdos aps a aplicao de um procedimento
teraputico computadorizado. Distrbios da Comunicao,
20(2): 183-192, So Paulo, agosto, 2008.
[10] BARBACENA, I.L., Melhoria Da Qualidade Da Voz De
Deficientes Auditivos Utilizando-se Correo Da Frequncia
Fundamental. Doctoral Thesis UFCG, 2010
[11] ALVES, R. L., Estudo sobre percepo Ttil Utilizando
Dispositivos de Baixo Custo Para a Melhoria da
Acessibilidade do Deficiente Visual. Dissertao de
Mestrado, Universidade Federal de Campina Grande.
Masters Dissertation UFCG, 2007.
[12] BARROS, A. T., Investigaes sobre Estimulao Ttil
Aplicada ao Ensino/aprendizagem da fala. Doctoral Thesis
UFCG, 2004.