Dynamic temporal residual learning for speech recognition

Jiaqi Xie; Ruijie Yan; Shanyu Xiao; Liangrui Peng; Michael T. Johnson; Wei Qiang Zhang

doi:10.1109/ICASSP40776.2020.9054653

Dynamic temporal residual learning for speech recognition

Jiaqi Xie, Ruijie Yan, Shanyu Xiao, Liangrui Peng, Michael T. Johnson, Wei Qiang Zhang

Electrical and Computer Engineering

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

4 Scopus citations

Abstract

Long short-term memory (LSTM) networks have been widely used in automatic speech recognition (ASR). This paper proposes a novel dynamic temporal residual learning mechanism for LSTM networks to better explore temporal dependencies in sequential data. The temporal residual learning mechanism is implemented by applying shortcut connections with dynamic weights to temporally adjacent LSTM outputs. Two types of dynamic weight generation methods are proposed: using a secondary network and using a random weight generator. Experimental results on Wall Street Journal (WSJ) speech recognition dataset reveal that our proposed methods have surpassed the baseline LSTM network.

Original language	English
Title of host publication	2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings
Pages	7709-7713
Number of pages	5
ISBN (Electronic)	9781509066315
DOIs	https://doi.org/10.1109/ICASSP40776.2020.9054653
State	Published - May 2020
Event	2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Barcelona, Spain Duration: May 4 2020 → May 8 2020

Publication series

Name	ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings
Volume	2020-May
ISSN (Print)	1520-6149

Conference

Conference	2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020
Country/Territory	Spain
City	Barcelona
Period	5/4/20 → 5/8/20

Bibliographical note

Publisher Copyright:
© 2020 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.

Keywords

ASR
Dynamic residual learning
LSTM

ASJC Scopus subject areas

Software
Signal Processing
Electrical and Electronic Engineering

Access to Document

10.1109/ICASSP40776.2020.9054653

Cite this

Xie, J., Yan, R., Xiao, S., Peng, L., Johnson, M. T., & Zhang, W. Q. (2020). Dynamic temporal residual learning for speech recognition. In 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings (pp. 7709-7713). Article 9054653 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2020-May). https://doi.org/10.1109/ICASSP40776.2020.9054653

@inproceedings{b89b5329caad4a8bb93bcc53b0465d92,

title = "Dynamic temporal residual learning for speech recognition",

abstract = "Long short-term memory (LSTM) networks have been widely used in automatic speech recognition (ASR). This paper proposes a novel dynamic temporal residual learning mechanism for LSTM networks to better explore temporal dependencies in sequential data. The temporal residual learning mechanism is implemented by applying shortcut connections with dynamic weights to temporally adjacent LSTM outputs. Two types of dynamic weight generation methods are proposed: using a secondary network and using a random weight generator. Experimental results on Wall Street Journal (WSJ) speech recognition dataset reveal that our proposed methods have surpassed the baseline LSTM network.",

keywords = "ASR, Dynamic residual learning, LSTM",

author = "Jiaqi Xie and Ruijie Yan and Shanyu Xiao and Liangrui Peng and Johnson, {Michael T.} and Zhang, {Wei Qiang}",

note = "Publisher Copyright: {\textcopyright} 2020 Institute of Electrical and Electronics Engineers Inc.. All rights reserved.; 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 ; Conference date: 04-05-2020 Through 08-05-2020",

year = "2020",

month = may,

doi = "10.1109/ICASSP40776.2020.9054653",

language = "English",

series = "ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings",

pages = "7709--7713",

booktitle = "2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings",

}

Xie, J, Yan, R, Xiao, S, Peng, L, Johnson, MT & Zhang, WQ 2020, Dynamic temporal residual learning for speech recognition. in 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings., 9054653, ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings, vol. 2020-May, pp. 7709-7713, 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020, Barcelona, Spain, 5/4/20. https://doi.org/10.1109/ICASSP40776.2020.9054653

Dynamic temporal residual learning for speech recognition. / Xie, Jiaqi; Yan, Ruijie; Xiao, Shanyu et al.
2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings. 2020. p. 7709-7713 9054653 (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings; Vol. 2020-May).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Dynamic temporal residual learning for speech recognition

AU - Xie, Jiaqi

AU - Yan, Ruijie

AU - Xiao, Shanyu

AU - Peng, Liangrui

AU - Johnson, Michael T.

AU - Zhang, Wei Qiang

PY - 2020/5

Y1 - 2020/5

N2 - Long short-term memory (LSTM) networks have been widely used in automatic speech recognition (ASR). This paper proposes a novel dynamic temporal residual learning mechanism for LSTM networks to better explore temporal dependencies in sequential data. The temporal residual learning mechanism is implemented by applying shortcut connections with dynamic weights to temporally adjacent LSTM outputs. Two types of dynamic weight generation methods are proposed: using a secondary network and using a random weight generator. Experimental results on Wall Street Journal (WSJ) speech recognition dataset reveal that our proposed methods have surpassed the baseline LSTM network.

AB - Long short-term memory (LSTM) networks have been widely used in automatic speech recognition (ASR). This paper proposes a novel dynamic temporal residual learning mechanism for LSTM networks to better explore temporal dependencies in sequential data. The temporal residual learning mechanism is implemented by applying shortcut connections with dynamic weights to temporally adjacent LSTM outputs. Two types of dynamic weight generation methods are proposed: using a secondary network and using a random weight generator. Experimental results on Wall Street Journal (WSJ) speech recognition dataset reveal that our proposed methods have surpassed the baseline LSTM network.

KW - ASR

KW - Dynamic residual learning

KW - LSTM

UR - http://www.scopus.com/inward/record.url?scp=85091278012&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85091278012&partnerID=8YFLogxK

U2 - 10.1109/ICASSP40776.2020.9054653

DO - 10.1109/ICASSP40776.2020.9054653

M3 - Conference contribution

AN - SCOPUS:85091278012

T3 - ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings

SP - 7709

EP - 7713

BT - 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings

T2 - 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020

Y2 - 4 May 2020 through 8 May 2020

ER -

Xie J, Yan R, Xiao S, Peng L, Johnson MT, Zhang WQ. Dynamic temporal residual learning for speech recognition. In 2020 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP 2020 - Proceedings. 2020. p. 7709-7713. 9054653. (ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings). doi: 10.1109/ICASSP40776.2020.9054653

Dynamic temporal residual learning for speech recognition

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this