Auditory feedback (AF) is an aid used by humans to control speech production and singing by helping the individual verify whether the current production of speech or singing is in accordance with his acoustic-auditory intention. This process is possible through what is known as the auditory feedback loop, a three-part cycle that allows individuals to first speak, then listen to what they have said, and lastly, correct it when necessary. From the viewpoint of movement sciences and neurosciences, the acoustic-auditory speech signal can be interpreted as the result of movements (skilled actions) of speech articulators (the lower jaw, lips, tongue, etc.). Auditory feedback can hence be inferred as a feedback mechanism controlling skilled actions in the same way that visual feedback controls limb movements (e.g. reaching movements).
Speech
Auditory feedback allows one to monitor their speech and rectify production errors quickly when they identify one, making it an important component of fluent speech productions.[1] The role of auditory feedback on speech motor control is often investigated by exposing participants to frequency-altered feedback. Inducing brief and unpredictable changes in the frequency of their auditory feedback has consistently been shown to induce a "pitch-shift reflex", which suggests that this reflex aids in stabilizing voice frequency around the desired target.[2][3]
However, due to the fact that auditory feedback needs more than 100 milliseconds before a correction occurs at the production level,[4] it is a slow correction mechanism in comparison with the duration (or production time) of speech sounds (vowels or consonants). Thus, auditory feedback is too slow to correct the production of a speech sound in real-time. Nonetheless, it has been shown that auditory feedback is capable of changing speech-sound production over a series of trials (i.e. adaptation by relearning; see e.g. perturbation experiments done with the DIVA model: neurocomputational speech processing). 10 minutes is typically sufficient for a nearly-full adaptation. Research has also shown that auditory linguistic prompts resulted in greater correction to acoustic perturbations than non-linguistic prompts, reflecting the decrease in accepted variance for intended speech when external linguistic templates are available to the speaker.[5]
Speech Acquisition and Development
Auditory feedback is an important aid during speech acquisition by toddlers, by providing the child with information about speech outcomes that are used to pick-up and eventually hone speech motor planning processes. Auditory inputs are typically produced by a communication partner (e.g. caretaker) and heard by the toddler, who subsequently tries to imitate them.[6][7] Children as young as the age of four have demonstrated the ability to adapt speech motor patterns to perceived changes in vowel auditory feedback, which enables them to maintain the accuracy of their speech output.[8] However, children's speech motor adaption abilities are not fully optimised due to their limited auditory perceptual skills. Thus, improvements in children's ability to perceive relevant acoustic property will usually be followed by an improvement in their speech adaption performance.[9]
Individuals who are born deaf often fail to acquire fluent speech, further reinforcing how auditory feedback plays a crucial role in speech acquisition and development.[10]
Delayed auditory feedback experiments indicate that auditory feedback is important during speech production, even in adults. It has been shown that severe disfluencies in speech occur when the timing of voice feedback is delayed for a normal speaker.[11][12] Individuals who become deaf post-lingually and are unable to receive vocal feedback anymore also typically experience a deterioration in speech quality,[13][14] highlighting the importance of auditory feedback in speech formation throughout one's lifetime.
Impacts on speech disorders
Stuttering
Stuttering is said to be due to ineffective monitoring of auditory feedback, mainly caused by a deficit in the cortical auditory system modulation during speech planning.[15] When fluent speakers detect a sudden irregularity in a specific acoustic parameter of their auditory feedback, they are able to instantly correct the error in their speech production. Individuals who stutter, on the other hand, are found to have weaker-than-normal abilities to correct such errors.[16] Individuals that stutter hence demonstrate ineffective auditory comparisons of desired speech movements, as compared to fluent speakers.[17]
Delayed auditory feedback has been found to be an effective treatment for some individuals who stutter,[18] since extending the time between speech and auditory perception allows for more time to process and correct errors.
Apraxia of speech
It is posited that individuals with apraxia of speech have weak feedforward programs, which results in the disfluencies of their speech.[19] These individuals hence develop a heavy reliance on auditory feedback to minimize and repair speech errors[20] even in later stages of their lives, whilst fluent speakers easily transitions from feedback dependent to feedforward-dominant.[21] This is not ideal since heavy reliance on mostly auditory feedback is said to be inefficient for the production of rapid and accurate speech.[22]
Auditory masking has been found to decrease disfluency duration and increase vocal intensity as well as syllable rate in some individuals with apraxia of speech.[23] Since apraxia of speech is said to be due to weak feedforward programs and high dependence on auditory feedback, auditory masking can be reasoned to increase fluency by decreasing the frequency of a speaker attending auditorily to speech errors, and hence reducing the likelihood of disfluency-generating corrections.
Impacts on Visually Impaired individuals
Enhanced auditory processing can be observed in individuals with visual impairment, who partially compensate for their lack of vision with greater sensitivity in their other sensories.[24] Their increased sensitivity to auditory feedback allows them to demonstrate impressive spatial awareness despite their lack of sight.[25][26][27]
Desktop Assistance
Studies have shown that when vision is no longer the primary source for obtaining information, focus shifts from vision to hearing in the desktop environment.[28] Currently, there are assistive technologies such as screen readers, which aids visually impaired individuals in obtaining information on their desktop screens via auditory feedback (E.g. JAWS[29]). The assistance can come in the form of either speech based auditory feedback or non-speech based auditory feedback. Speech based interfaces are based on human speech, whilst non-speech based interfaces are based on environmental sounds such as music or artificial sound effects.
For the visually impaired, sole reliance on speech based auditory feedback imposes a heavier cognitive load which is irritating for users.[30] In contrast, non-speech auditory feedback is pleasant and conveys information more quickly, but lacks detailed information in their conveyance and training is required to understand the cues. Hence, the most ideal interface currently is adaptive auditory feedback, which automatically transitions between speech and non-speech cues based on the user state. Such an interface has been found to be more comfortable and generates higher satisfaction amongst visually impaired users.[31]
Impacts on other disorders
Graphomotor learning in writing disorders
A trial was conducted to explore whether auditory feedback had an influence on learning how to write. It was found that in adults, auditory feedback enabled the writer to better discern their writing motions. This resulted in an increase in flow and quickness of writing when using sounds to learn the writing of new characters.[32] Subsequent studies then tested the use of auditory feedback as an aid for children with dysgraphia to learn how to write. It was found that after multiple sittings of using auditory feedback while writing, children could write more smoothly, rapidly and clearly.[33]
Products based on auditory feedback principles have been invented to aid individuals with such writing disorders. Children with speech disorders can also benefit from such products. For example, a headphone called Forbrain[34] uses a bone conductor and a series of dynamic filters to correct the perception of one's own voice. This improves concentration, attention, speech, coordination, and other sensory functions. It was awarded by the BETT Show[35] in 2015 in the category "ICT Special Educational Needs Solutions".
Motor learning in movement disorders
Patients with cerebral palsy have little walking capability, due to limitations of their nervous system.[36] Auditory feedback in the form of periodic audio signals was found to have a significant improvement on the gait of patients, with several explanations proposed. One model argues that auditory feedback acts as an additional information channel for the motor systems, thereby decreasing the onset of motor faults and refining the gait of patients.[37] Another model posits that audio signals influence the gait of patients by directing motion patterns, such as heel strike timings. By wearing a device that provides immediate auditory feedback on the quality of one's gait, children with cerebral palsy learned to set down their feet in proper ways that avoided the sounds created when negative gaits were detected.[36]
Social interaction and motor coordination learning in behavioural disorders
The use of an auditory feedback-based treatment is found to have improved on the social interaction, mimicking and coordination skills of children with autism spectrum disorder.[38] This is achieved through a software which uses sensors to track the body motions of children. Each gesture made will activate a voice recording articulating pieces of sentences.[38] Children then have to reorder these sentence pieces to form a storyline. Different indicators of coordination such as motion quantity and speed were also recorded to keep track of the child's improvement through these auditory cues.[38]
Impacts on Music Performance
Instrument Performance
Auditory feedback is important in the picking up of a new musical piece. By exposing beginner piano players to irregular auditory feedback, they make more mistakes as compared to those who are given logical and anticipatable auditory feedback.[39] Learning in the presence of auditory feedback also improved one's recollection of the musical piece.[40]
However, multiple studies have shown that even without auditory feedback, there is not much disturbance to the performance of seasoned musicians.[41][42] In the absence or delay of auditory feedback, musicians turn to auditory imagery to direct their performance.[42] Other forms of feedback can also be used in compensation instead, such as visual feedback where musicians look at their hands to lead their performance.[43] Major disturbances were only seen in the area of pedaling, where results have shown that pianists were prone to stepping the pedal less often in the absence of auditory feedback.[42]
Singing
The importance of auditory feedback in the case of human singing is reviewed by Howell.[44] In the context of singing, it is important for singers to maintain pitch accuracy, even when they are drowned out by orchestral accompaniment or by fellow singers. Many studies have looked into the effects of both external auditory feedback and proprioception (also known as internal feedback) on pitch control. It has been found that external auditory feedback is crucial in maintaining pitch accuracy, especially for adults without voice training.[45][46] This is further supported by recent research which revealed how non-professional singers show lower pitch accuracy when they receive lesser auditory feedback. However, the research also highlighted how the pitch of professional singers remains almost unaffected by auditory feedback since they are able to rely on their internal feedback after years of training.[47]
Bird songs
The role of auditory feedback in the learning and production of bird-song has been studied in several research papers. It has been found that songbirds rely on auditory feedback to compare the sounds that they make with inborn tunes or songs that they memorize from others.[48] Numerous studies have shown that without the ability to hear themselves, songbirds develop erratic songs or show a deterioration in the songs that they sing after experiencing hearing loss.[49][50] Several scientific models have been put forward to explain the worsening of birdsongs after the loss of hearing. (E.g. see Brainard and Doupe's (2000) error adjustment channel in the anterior forebrain: auditory feedback in birdsong learning).[49]
However, the decline of birdsong quality can vary greatly between different demographics. For example, other studies have found the songs of older songbirds remained consistent, or had a slower rate of deterioration after going deaf.[51] Some researchers have attributed to songbirds learning how to use other forms of non-auditory feedback such as sensory information to maintain the quality of their songs.[50] This process is called sensory-motor coupling. Others have argued that older songbirds have a longer access to auditory feedback to learn their songs, which results in more practice and thus more stable production of songs even after deafening.[51]
See also
References
- ↑ Guenther, F.H (2006). "Cortical interactions underlying the production of speech sounds". Journal of Communication Disorders. 39 (5): 350–365. doi:10.1016/j.jcomdis.2006.06.013. PMID 16887139.
- ↑ Hawco, C.S; Jones, J.A (2009). "Control of vocalization at utterance onset and mid-utterance: different mechanisms for different goals". Brain Res. 1276: 131–139. doi:10.1016/j.brainres.2009.04.033. PMC 2696260. PMID 19394315.
- ↑ Baur, J.J; Mittal, J.; Larson, C.R; Hain, T.C (2006). "Vocal responses to unanticipated perturbations in voice loudness feedback: an automatic mechanism for stabilizing voice amplitude". J. Acoust. Soc. Am. 119 (2): 2363–2371. doi:10.1121/1.1592161. PMC 1698961. PMID 12942983.
- ↑ Burnett, T. A.; Freedland, M. B.; Larson, C. R.; Hain; T. C. (June 1998). "Voice F0 responses to manipulations in pitch feedback". Journal of the Acoustical Society of America. 103 (6): 3153–61. Bibcode:1998ASAJ..103.3153B. doi:10.1121/1.423073. PMID 9637026.
- ↑ Alsius, Agnes; Mitsuya, Takashi; Latif, Nida; Munhall, K.G (2017). "Linguistic initiation signals increase auditory feedback error correction". The Journal of the Acoustical Society of America. 142 (2): 838–845. Bibcode:2017ASAJ..142..838A. doi:10.1121/1.4997193. PMID 28863596.
- ↑ Perkell, J; Mathies, M; Harlan, L; Guenther, F; Wilhelms-Tricarico, R; Wozniak, J; Guiod, P (1997). "Speech motor control: Acoustic goals, saturation effects, auditory feedback and internal models". Speech Communication. 22 (2–3): 227–250. doi:10.1016/S0167-6393(97)00026-5.
- ↑ Callan, DE; Kent, RD; Guenther, FH; Vorperian, HK (2000). "An auditory-feedback-based neural network model of speech production that is robust to developmental changes in the size and shape of the articulatory system". Journal of Speech, Language, and Hearing Research. 43 (3): 721–836. doi:10.1044/jslhr.4303.721. PMID 10877441.
- ↑ Macdonald, E.N; Johnson, E.K; Forsythe, J.; Plante, P; Munhall, K.G (2012). "Children's development of self-regulation in speech production". Curr. Biol. 22 (2): 113–7. doi:10.1016/j.cub.2011.11.052. PMC 3267900. PMID 22197241.
- ↑ Shiller, D.M; Rochon, M.L (2014). "Auditory-Perceptual Learning Improves Speech Motor Adaptation in Children". Journal of Experimental Psychology: Human Perception and Performance. 40 (4): 1308–1315. doi:10.1037/a0036660. PMC 4433313. PMID 24842067.
- ↑ Svirsky, M.A; Teo, S.W; Nueburger, H. (2004). "Development of language and speech perception in congenitally, profoundly deaf children as a function of age at cochlear implantation". Audiol. Neurotol. 9 (4): 224–233. doi:10.1159/000078392. PMID 15205550. S2CID 26092160.
- ↑ Lee, BS (1950). "Some effects of side-tone delay". Journal of the Acoustical Society of America. 22 (5): 639–640. Bibcode:1950ASAJ...22..639L. doi:10.1121/1.1906665.
- ↑ Fairbanks, G (1955). "Selective vocal effects of delayed auditory feedback". Journal of Speech and Hearing Disorders. 20 (4): 333–46. doi:10.1044/jshd.2004.333. PMID 13272227.
- ↑ Cowie, R.; Douglas-Cowie, E.; Kerr, A.G (1982). "A study of speech deterioration in post-lingually deafened adults". J. Laryngol. Otol. 96 (2): 101–112. doi:10.1017/S002221510009229X. PMID 7057081. S2CID 40288349.
- ↑ Goehl, H.; Kaufman, D.K (1984). "Do the effects of adventitious deafness include disordered speech?". J. Speech Hear. Disord. 49 (1): 58–64. doi:10.1044/jshd.4901.58. PMID 6700203.
- ↑ Ringman, A; Max, L (2015). "Modulation of auditory processing during speech movement planning is limited in adults who stutter". Brain Language. 143: 59–68. doi:10.1016/j.bandl.2015.03.002. PMC 4380808. PMID 25796060.
- ↑ Cai, S; Beal, S; Ghosh, S; Tiede, MK; Guenther, FH; Perkell, JS (2012). "Weak responses to auditory feedback perturbation during articulation in persons who stutter: evidence for abnormal auditory-motor transformation". PLOS ONE. 7 (7): 1–13. Bibcode:2012PLoSO...741830C. doi:10.1371/journal.pone.0041830. PMC 3402433. PMID 22911857.
- ↑ Hudock, D; Dayalu, VN; Saltuklaroglu, T; Stuart, A; Zhang, J; Kalinowski, J (2010). "Stuttering inhibition via visual feedback at normal and fast speech rates". International Journal of Language and Communication Disorders. 46 (2): 169–178. doi:10.3109/13682822.2010.490574. PMID 21401815.
- ↑ Borsel, J.V; Reunes, G.; Bergh, N.V.D (2003). "Delayed auditory feedback in the treatment of stuttering: clients as consumers". Intl. Journal of Commun Disord. 38 (2): 119–29. doi:10.1080/1368282021000042902. PMID 12745932.
- ↑ Terband, H.; Maasen, B. (2010). "Speech motor development in childhood apraxia of speech: Generating testable hypotheses by neurocomputational modeling". Folia Phoniatrica et Logopaedica. 62 (3): 134–142. doi:10.1159/000287212. PMID 20424469. S2CID 10289830.
- ↑ Terband, H.; Maassen, B.; Guenther, F.H; Brumberg, J. (2009). "Computational neural modeling of speech motor control in childhood apraxia of speech (CAS)". Journal of Speech, Language, and Hearing Research. 52 (6): 1595–1609. doi:10.1044/1092-4388(2009/07-0283). PMC 2959199. PMID 19951927.
- ↑ Guenther, F.H (2006). "Cortical interactions underlying the production of speech sounds". Journal of Communication Disorders. 39 (5): 350–365. doi:10.1016/j.jcomdis.2006.06.013. PMID 16887139.
- ↑ Houde, J.F; Nagarajan, S.S (2011). "Speech production as state feedback control". Frontiers in Human Neuroscience. 5: 1–14. doi:10.3389/fnhum.2011.00082. PMC 3200525. PMID 22046152.
- ↑ Jacks, A.; Haley, K.L (2015). "Auditory Masking Effects on Speech Fluency in Apraxia of Speech and Aphasia: Comparison to Altered Auditory Feedback". J Speech Lang Hear Res. 58 (6): 1670–1686. doi:10.1044/2015_JSLHR-S-14-0277. PMC 4987030. PMID 26363508.
- ↑ Boas, L.V; Muniz, L.; Neto, S.D.S.C; Gouveia, M.D.C.L.G (2011). "Auditory processing performance in blind people". Brazilian Journal of Otorhinolaryngology. 77 (4): 504–509. doi:10.1590/S1808-86942011000400015. PMC 9450742. PMID 21860978.
- ↑ Thaler, L.; Arnott, S.R; Goodale, M.A (2011). "Neural correlates of natural human echolocation in early and late blind echolocation experts". PLOS ONE. 6 (5): e20162. Bibcode:2011PLoSO...620162T. doi:10.1371/journal.pone.0020162. PMC 3102086. PMID 21633496.
- ↑ Collignon, O; Renier, L.; Bruyer, R.; Tranduy, D.; Veraart, C. (2006). "Improved selective and divided spatial attention in early blind subjects". Brain Res. 1075 (1): 175–182. doi:10.1016/j.brainres.2005.12.079. PMID 16460716. S2CID 22551164.
- ↑ Kolarik, A.J; Cirstea, S.; Pardhan, S.; Moore, B.C (2014). "A summary of research investigating echolocation abilities of blind and sighted humans" (PDF). Hear. Res. 310: 60–68. doi:10.1016/j.heares.2014.01.010. PMID 24524865. S2CID 21785505.
- ↑ Monticelli, C.; Heidrich, R.D.O; Rodriguez, R.; Capellatti, E.; Goulart, R.; Oliveira, R.; Velho, E. (2018). "Text vocalizing desktop scanner for visually impaired people". International Conference on Human–Computer Interaction. Springer: 62=67.
- ↑ "Job Access With Speech". Freedom Scientific.
- ↑ Hussain, I.; Chen, L.; Mirza, H.T; Chen, G.; Hassan, S. (2015). "Right mix of speech and non-speech: hybrid auditory feedback in mobility assistance of the visually impaired". Univ. Access Inf. Soc. 14 (4): 527–536. doi:10.1007/s10209-014-0350-7. S2CID 14365566.
- ↑ Shoaib, M.; Hussain, I.; Mirza, H.T (2020). "Automatic switching between speech and non-speech: adaptive auditory feedback in desktop assistance for the visually impaired". Univ Access Inf Soc. 19 (4): 813–823. doi:10.1007/s10209-019-00696-5. S2CID 204707401.
- ↑ Danna, J.; Fontaine, M.; Paz-Villagran, V.; Gondre, C.; Thoret, E.; Aramaki, M.; Kronland-Martinet, R.; Ystad, S.; Velay, J. (2015). "The effect of real-time auditory feedback on learning new characters" (PDF). Human Movement Science. 43 (1): 216–228. doi:10.1016/j.humov.2014.12.002. PMID 25533208. S2CID 19168413.
- ↑ Danna, J.; Fontaine, M.; Paz-Villagran, V.; Gondre, C.; Thoret, E.; Aramaki, M.; Kronland-Martinet, R.; Ystad, S.; Velay, J. (5 December 2014). Movement Sonification for the Diagnosis and the Rehabilitation of Graphomotor Disorders. International Symposium on Computer Music Multidisciplinary Research. France: Springer. pp. 246–255. doi:10.1007/978-3-319-12976-1_16.
- ↑ Forbrain
- ↑ BETT Show
- 1 2 Pitale, J.T.; Bolte, J.H.IV (2018). "A heel-strike real-time auditory feedback device to promote motor learning in children who have cerebral palsy:a pilot study to test device accuracy and feasibility to use a music and dance-based learning paradigm". Pilot and Feasibility Studies. 4 (42): 1–7. doi:10.1186/s40814-018-0229-0. PMC 5789741. PMID 29423260.
- ↑ Ghai, Shashank; Ghai, Ishan; Effenberg, A.O. (2018). "Effect of rhythmic auditory cueing on gait in cerebral palsy: a systematic review and meta-analysis". Neuropsychiatric Disease and Treatment. 14: 43–59. doi:10.2147/NDT.S148053. PMC 5746070. PMID 29339922.
- 1 2 3 Magrini, M.; Carboni, A.; Salvetti, O.; Curzio, O. (14 November 2017). An Auditory Feedback Based System for Treating Autism Spectrum Disorder. International Workshop on ICTs for Improving Patients Rehabilitation Research Techniques. Springer. pp. 46–58. doi:10.1007/978-3-319-69694-2_5.
- ↑ Lappe, C.; Lappe, M.; Keller, P.K (2018). "The influence of pitch feedback on learning of motor -timing and sequencing: A piano study with novices". PLOS ONE. 13 (11): e0207462. Bibcode:2018PLoSO..1307462L. doi:10.1371/journal.pone.0207462. PMC 6261582. PMID 30485336.
- ↑ Finney, S.; Palmer, C. (2003). "Auditory feedback and memory for music performance: Sound evidence for an encoding effect". Memory & Cognition. 31 (1): 51–64. doi:10.3758/BF03196082. PMID 12699143. S2CID 27525135.
- ↑ Finney, S.A (1997). "Auditory Feedback and Musical Keyboard Performance". Music Perception. 15 (2): 153–174. doi:10.2307/40285747. JSTOR 40285747.
- 1 2 3 Repp, B.H (1999). "Effects of Auditory Feedback Deprivation on Expressive Piano Performance". Music Perception. 16 (4): 409–438. doi:10.2307/40285802. JSTOR 40285802.
- ↑ Banton, L.J (1995). "The Role of Visual and Auditory Feedback during the Sight-Reading of Music". Psychology of Music. 23 (1): 3–16. doi:10.1177/0305735695231001. S2CID 145095050.
- ↑ Howell, P. (1985). "Auditory Feedback of the Voice in Singing". In West, Robert; Howell, Peter; Cross, Ian (eds.). Musical Structure and Cognition. London: Orlando Academic Press. pp. 259–286. ISBN 978-0-12-357170-0.
- ↑ Elliot, L.; Niemoeller, A. (1970). "The role of hearing in controlling voice fundamental frequency". International Journal of Audiology. Taylor & Francis. 9 (1): 47–52. doi:10.3109/05384917009071993.
- ↑ schultz-Coulon, H.J (1987). "The Neuromuscular Phonatory Control System and Vocal Function". Acta Oto-Laryngologica. 86 (1–6): 142–53. doi:10.3109/00016487809124731. PMID 696292.
- ↑ Bottalico, P.; Graetzer, S.; Hunter, E.J (2017). "Effect of Training and Level of External Auditory Feedback on the Singing Voice: Pitch Inaccuracy". Journal of Voice. Elsevier. 31 (1): 122.e9–122.e16. doi:10.1016/j.jvoice.2016.01.012. PMC 5010534. PMID 26948385.
- ↑ Leonardo, A.; Konishi, M. (1999). "Decrystallization of adult birdsong by perturbation of auditory feedback". Nature. 399 (1): 466–470. Bibcode:1999Natur.399..466L. doi:10.1038/20933. PMID 11252766. S2CID 4403659.
- 1 2 Brainard, M. S.; Doupe, A. J. (2000). "Auditory Feedback In Learning And Maintenance Of Vocal Behaviour". Nature Reviews Neuroscience. 1 (1): 31–40. doi:10.1038/35036205. PMID 11252766. S2CID 5133196.
- 1 2 Konishi, M. (2004). "The Role of Auditory Feedback in Birdsong". Annals of the New York Academy of Sciences. 1016 (1): 463–475. Bibcode:2004NYASA1016..463K. doi:10.1196/annals.1298.010. PMID 15313790. S2CID 34284685.
- 1 2 Lombardino, A. J.; Nottebohm, F. (2000). "Age at Deafening Affects the Stability of Learned Song in Adult Male Zebra Finches". Journal of Neuroscience. 20 (13): 5054–5064. doi:10.1523/JNEUROSCI.20-13-05054.2000. PMC 6772266. PMID 10864963.