Speech Enhancement Algorithm Based on a Convolutional Neural Network Reconstruction of the Temporal Envelope of Speech in Noisy Environments

Rahim Soleymanpour, Mohammad Soleymanpour, Anthony J. Brammer, Michael T. Johnson, Insoo Kim

Research output: Contribution to journalArticlepeer-review

10 Scopus citations

Abstract

Temporal modulation processing is a promising technique for improving the intelligibility and quality of speech in noise. We propose a speech enhancement algorithm that constructs the temporal envelope (TEV) in the time-frequency domain by means of an embedded convolutional neural network (CNN). To accomplish this, the input speech signals are divided into sixteen parallel frequency bands (subbands) with bandwidths approximating 1.5 times that of auditory filters. The corrupted TEVs in each subband are extracted and then fed to the 1-dimensional CNN (1-D CNN) model to restore the TEVs distorted by noise. The method is evaluated using 2,700 words from nine different talkers, which are mixed with speech-spectrum shaped random noise (SSN), and babble noise, at different signal-to-noise ratios. The Short-time Objective Intelligibility (STOI) and Perceptual Evaluation of Speech Quality (PESQ) metrics are used to evaluate the performance of the 1-D CNN algorithm. Results suggest that the 1-D CNN model improves STOI scores on average by 27% and 34% for SSN and babble noise, respectively, and PESQ scores on average by 19% and 18%, respectively, compared to unprocessed speech. The 1-D CNN model is also shown to outperform a conventional TEV-based speech enhancement algorithm.

Original languageEnglish
Pages (from-to)5328-5336
Number of pages9
JournalIEEE Access
Volume11
DOIs
StatePublished - 2023

Bibliographical note

Publisher Copyright:
© 2013 IEEE.

Keywords

  • Speech enhancement
  • convolution neural network (CNN)
  • temporal envelope (TEV)

ASJC Scopus subject areas

  • General Engineering
  • General Computer Science
  • General Materials Science

Fingerprint

Dive into the research topics of 'Speech Enhancement Algorithm Based on a Convolutional Neural Network Reconstruction of the Temporal Envelope of Speech in Noisy Environments'. Together they form a unique fingerprint.

Cite this