Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion

Narjes Bozorg; Michael T. Johnson; Mohammad Soleymanpour

doi:10.1109/SpeD53181.2021.9587350

Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion

Narjes Bozorg, Michael T. Johnson, Mohammad Soleymanpour

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this paper we introduce a new speaker independent method for Acoustic-to-Articulatory Inversion. The proposed architecture, Speaker Independent-Articulatory WaveNet (SI-AWN), models the relationship between acoustic and articulatory features by conditioning the articulatory trajectories on acoustic features and then utilizes the structure for unseen target speakers. We evaluate the proposed SI-AWN on the Electro Magnetic Articulography corpus of Mandarin Accented English (EMA-MAE), using the pool of acoustic-articulatory information from 35 reference speakers and testing on target speakers that include male, female, native and non-native speakers. The results suggest that SI-AWN improves the performance of the acoustic-to-articulatory inversion process compared to the baseline Maximum Likelihood Regression-Parallel Reference Speaker Weighting (MLLR-PRSW) method by 21 percent. To the best of our knowledge, this is the first application of a WaveNet-like synthesis approach to the problem of Speaker Independent Acoustic-to-Articulatory Inversion, and results are comparable to or better than the best currently published systems.

Original language	English
Title of host publication	2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021
Pages	156-161
Number of pages	6
ISBN (Electronic)	9781665427869
DOIs	https://doi.org/10.1109/SpeD53181.2021.9587350
State	Published - 2021
Event	11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021 - Virtual, Bucharest, Romania Duration: Oct 13 2021 → Oct 15 2021

Publication series

Name	2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021

Conference

Conference	11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021
Country/Territory	Romania
City	Virtual, Bucharest
Period	10/13/21 → 10/15/21

Bibliographical note

Publisher Copyright:
© 2021 IEEE.

Keywords

Acoustic-to-articulatory inversion
Deep autoregressive model
Speaker-independent
WaveNet

ASJC Scopus subject areas

Language and Linguistics
Human-Computer Interaction
Signal Processing
Safety, Risk, Reliability and Quality
Communication

Access to Document

10.1109/SpeD53181.2021.9587350

Cite this

Bozorg, N., Johnson, M. T., & Soleymanpour, M. (2021). Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion. In 2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021 (pp. 156-161). (2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021). https://doi.org/10.1109/SpeD53181.2021.9587350

Bozorg, Narjes ; Johnson, Michael T. ; Soleymanpour, Mohammad. / Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion. 2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021. 2021. pp. 156-161 (2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021).

@inproceedings{6f3b00c91a9549bb97171c9368ebe7b1,

title = "Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion",

abstract = "In this paper we introduce a new speaker independent method for Acoustic-to-Articulatory Inversion. The proposed architecture, Speaker Independent-Articulatory WaveNet (SI-AWN), models the relationship between acoustic and articulatory features by conditioning the articulatory trajectories on acoustic features and then utilizes the structure for unseen target speakers. We evaluate the proposed SI-AWN on the Electro Magnetic Articulography corpus of Mandarin Accented English (EMA-MAE), using the pool of acoustic-articulatory information from 35 reference speakers and testing on target speakers that include male, female, native and non-native speakers. The results suggest that SI-AWN improves the performance of the acoustic-to-articulatory inversion process compared to the baseline Maximum Likelihood Regression-Parallel Reference Speaker Weighting (MLLR-PRSW) method by 21 percent. To the best of our knowledge, this is the first application of a WaveNet-like synthesis approach to the problem of Speaker Independent Acoustic-to-Articulatory Inversion, and results are comparable to or better than the best currently published systems.",

keywords = "Acoustic-to-articulatory inversion, Deep autoregressive model, Speaker-independent, WaveNet",

author = "Narjes Bozorg and Johnson, {Michael T.} and Mohammad Soleymanpour",

note = "Publisher Copyright: {\textcopyright} 2021 IEEE.; 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021 ; Conference date: 13-10-2021 Through 15-10-2021",

year = "2021",

doi = "10.1109/SpeD53181.2021.9587350",

language = "English",

series = "2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021",

pages = "156--161",

booktitle = "2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021",

}

Bozorg, N, Johnson, MT & Soleymanpour, M 2021, Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion. in 2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021. 2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021, pp. 156-161, 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021, Virtual, Bucharest, Romania, 10/13/21. https://doi.org/10.1109/SpeD53181.2021.9587350

Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion. / Bozorg, Narjes; Johnson, Michael T.; Soleymanpour, Mohammad.
2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021. 2021. p. 156-161 (2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion

AU - Bozorg, Narjes

AU - Johnson, Michael T.

AU - Soleymanpour, Mohammad

PY - 2021

Y1 - 2021

N2 - In this paper we introduce a new speaker independent method for Acoustic-to-Articulatory Inversion. The proposed architecture, Speaker Independent-Articulatory WaveNet (SI-AWN), models the relationship between acoustic and articulatory features by conditioning the articulatory trajectories on acoustic features and then utilizes the structure for unseen target speakers. We evaluate the proposed SI-AWN on the Electro Magnetic Articulography corpus of Mandarin Accented English (EMA-MAE), using the pool of acoustic-articulatory information from 35 reference speakers and testing on target speakers that include male, female, native and non-native speakers. The results suggest that SI-AWN improves the performance of the acoustic-to-articulatory inversion process compared to the baseline Maximum Likelihood Regression-Parallel Reference Speaker Weighting (MLLR-PRSW) method by 21 percent. To the best of our knowledge, this is the first application of a WaveNet-like synthesis approach to the problem of Speaker Independent Acoustic-to-Articulatory Inversion, and results are comparable to or better than the best currently published systems.

AB - In this paper we introduce a new speaker independent method for Acoustic-to-Articulatory Inversion. The proposed architecture, Speaker Independent-Articulatory WaveNet (SI-AWN), models the relationship between acoustic and articulatory features by conditioning the articulatory trajectories on acoustic features and then utilizes the structure for unseen target speakers. We evaluate the proposed SI-AWN on the Electro Magnetic Articulography corpus of Mandarin Accented English (EMA-MAE), using the pool of acoustic-articulatory information from 35 reference speakers and testing on target speakers that include male, female, native and non-native speakers. The results suggest that SI-AWN improves the performance of the acoustic-to-articulatory inversion process compared to the baseline Maximum Likelihood Regression-Parallel Reference Speaker Weighting (MLLR-PRSW) method by 21 percent. To the best of our knowledge, this is the first application of a WaveNet-like synthesis approach to the problem of Speaker Independent Acoustic-to-Articulatory Inversion, and results are comparable to or better than the best currently published systems.

KW - Acoustic-to-articulatory inversion

KW - Deep autoregressive model

KW - Speaker-independent

KW - WaveNet

UR - http://www.scopus.com/inward/record.url?scp=85125652610&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85125652610&partnerID=8YFLogxK

U2 - 10.1109/SpeD53181.2021.9587350

DO - 10.1109/SpeD53181.2021.9587350

M3 - Conference contribution

AN - SCOPUS:85125652610

T3 - 2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021

SP - 156

EP - 161

BT - 2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021

T2 - 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021

Y2 - 13 October 2021 through 15 October 2021

ER -

Bozorg N, Johnson MT, Soleymanpour M. Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion. In 2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021. 2021. p. 156-161. (2021 11th International Conference on Speech Technology and Human-Computer Dialogue, SpeD 2021). doi: 10.1109/SpeD53181.2021.9587350

Autoregressive articulatory wavenet flow for speaker-independent acoustic-to-articulatory inversion

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this