The Handbook of Language and Speech Disorders

In a recent review on emotional prosody processing, Jiam, Caldwell, Deroche, Chatterjee, and Limb (2017) concluded that CI users have considerable difficulties perceiving and producing emotional prosody. For example, in Gilbers et al. (2015) on a four‐way emotion identification test using nonsense words, CI users scored around 45% correct, NH listeners using vocoders around 70%, and NH listeners using unprocessed speech over 90% (chance level was 25%). Another emotional prosody review (Picou, Singh, Goy, Russo, Hickson, et al., 2018) attributes the deficits to CI recipients’ poor pitch processing capabilities. They rely more on other cues (like intensity and duration) than NH listeners, but despite their poor pitch discrimination abilities, CI listeners do still rely to some extent on pitch cues. This would explain their difficulties in emotional prosody perception, where pitch information can be very important. Another study by Picou et al. (2018) found no relationship between the degree of hearing loss, as measured by pure‐tone average thresholds, or pitch, loudness and gap detection sensitivity, on the one hand, and the recognition of emotions with which semantically neutral sentences were pronounced on the other. This literature shows that deficits in CI recipients’ emotional prosody perception seem more difficult to explain in terms of lower‐level psychophysical abilities than linguistic prosody perception. One speculative explanation is that emotion processing operates relatively independently to that of processing linguistic structure because perception of the emotional content of speech is more basic and universal (Scherer, Banse, & Wallbott, 2001) and less domain‐specific; for instance, affective emotions can also be conveyed by visual cues.

It has been demonstrated that problems with emotional prosody perception have repercussions for CI recipients’ social development, and this is, of course, a problem that is more pronounced in prelingually deafened CI recipients who have developed their receptive communication exclusively through their implants. Wiefferink, Rieffe, Ketelaar, De Raeve, and Frijns (2013) found that Dutch CI children, between 2.5 and 5 years old, performed worse on facial and situational emotion understanding and general expressive and receptive language development. The language test scores correlated with emotion tests that required verbal processing, suggesting that linguistic development can predict emotional development to some extent. Mancini et al. (2016) confirmed the connection between linguistic and emotional development for a group of 72 children aged between 4 and 11 years. However, 79% of their cohort showed no deviant understanding of emotion, which may be due to the fact that their participants were older and that a larger percentage of them used oral language exclusively.

In summary, CI listeners may to some extent, in their daily communication with the device, miss out on many if not most aspects of prosody. In prelingually deafened CI children this seems to be due to their more basic problems with pitch and spectral perception, although this causality is less clear for postlingually deafened CI recipients and the perception of emotional prosody by all CI recipients. Emotional prosody perception deficits in CI children have been shown to have consequences for more general socio‐emotional development.

It is a plausible assumption that proper speech perception is required for proper speech production. For instance, people with congenital (Osberger & McGarr, 1982) and acquired (Waldstein, 1990) hearing loss tend to produce abnormal speech. The mechanism behind this link has been described in the Directions Into Velocity of Articulators (DIVA) model (Guenther, 2006). In short, during preparation for articulation, an abstract acoustic representation of the target is projected. This projection is compared with the acoustics of the actual projection. Deviations can inform corrections in production online and can also update the representations. The acoustic representations are learned on the basis of speech input accumulated during the speaker’s life. It is therefore to be expected that aberrant input, as in the case of individuals with hearing loss including those who have received a CI, results in aberrant representations and therefore abnormal speech output.

This expectation is corroborated by older as well as more recent work. For instance, in the area of segmental phonetics, CI recipients, in comparison to NH speakers, have been found to have inaccurate consonant (Sundarrajan, Tobey, Nicholas, & Geers, 2019) and vowel (Menard et al., 2007; Verhoeven, Hide, De Maeyer, Gillis, & Gillis, 2016) production; however, see also Baudonck, Van Lierde, Dhooge, and Corthals (2011). Sundarrajan et al. (2019) measured Percentage of Consonants Correct, a metric from Shriberg, Austin, Lewis, McSweeny, and Wilson (1997), and consonant diversity in the spontaneous speech of 129 CI children of around 4 years of age. At the age of 3.5 years, the implanted children’s inventory consisted of 14 consonants and that of the controls of 19, with 17 and 19 for the CI and NH groups, respectively, 1 year later. The CI recipients’ consonant accuracy improved from 71% at 3.5 years to 83% 1 year later, whereas it was 94 and 95% for the controls, at the same ages. Verhoeven et al. (2016) measured the vowel space and dispersion (overlap) in imitations of monosyllables containing the 12 monophthongs of Belgian, by children with conventional HA, children with CIs, and NH children. The HA and CI groups had a more centralized (more so for the second formant, F2, than for the first formant, F1) vowel space, and also more overlap between vowel categories than the control participants. These studies show that the diversity of segmental production in CI users on average is reduced relative to the norm.

Research on prosody production has shown that CI users, among other issues, have a higher F 0than controls (Poissant, Peters, & Robb, 2006; Szyfter et al., 1996), abnormal realization and usage of intonation (Chin, Bergeson, & Phan, 2012; Holt et al., 2017; Lee & Sim, 2020), a low speech rate (Evans & Deliyski, 2007; Uchanski & Geers, 2003), and a deviant voice quality (Baudonck et al., 2011; Monini, Banci, Barbara, Argiro, & Filipo, 1997; Ubrig et al., 2011). As an example of a study on linguistic prosody, in Lee and Sim (2020), final contours of imitated declarative and interrogative sentences were measured acoustically and evaluated for appropriateness by naïve raters. Children with CIs were found to distinguish the sentence types less clearly. Duration of implant usage was a significant predictor of these scores, showing that over time, CI users improve their prosodic control. In a comparable study (Peng, Tomblin, & Turner, 2008), such sentence‐type ratings were 74% correct when pronounced by CI users, and they achieved an appropriateness rating of 3.1 on a 5‐point scale, whereas with NH speakers the sentence type rating was 94% with an appropriateness of 4.5.

Research on emotional prosody production is sparser. A review by Jiam et al. (2017) points to studies on production of happy and sad vocal emotions by children and adults showing a lower accuracy in imitations (Nakata, Trehub, & Kanda, 2012) as well as in read sentences (Chatterjee et al., 2016). Comparable results have been reported by Bergeson and Chin (2008) and Wang, Trehub, Volkova, and van Lieshout (2013). Chatterjee et al. (2019), the preliminary results of which were presented in Chatterjee et al. (2016), longitudinally measured F 0mean and variability and mean intensity, duration and spectral centroid of happy and sad renderings of simple utterances in prelingually deafened children with CIs, postlingually deafened adults with CIs, and NH peers for both of those groups. The children with CIs showed the least amount of acoustic distinction between the emotions, mainly for F 0‐related measures. The group of adult CI recipients were better able to contrast the emotions, which suggests, as the authors argue, that the acoustic input that they received prior to implantation supports normal vocal emotion expression. Moreover, the fact that they maintained the contrast after years of degraded (i.e., electric) hearing indicates that the sound representations that are developed and stored prior to implantation are very robust.