Lattice Based Transcription Loss for End-to-End Speech Recognition

Jian Kang, Wei Qiang Zhang, Wei Wei Liu, Jia Liu, Michael T. Johnson

Research output: Contribution to journalArticlepeer-review

3 Scopus citations


End-to-end speech recognition systems have been successfully implemented and have become competitive replacements for hybrid systems. A common loss function to train end-to-end systems is connectionist temporal classification (CTC). This method maximizes the log likelihood between the feature sequence and the associated transcription sequence. However there are some weaknesses with CTC training. The main weakness is that the training criterion is different from the test criterion, since the training criterion is log likelihood, while the test criterion is word error rate. In this work, we introduce a new lattice based transcription loss function to address this deficiency of CTC training. Compared to the CTC function, our new method optimizes the model directly using the transcription loss. We evaluate this new algorithm in both a small speech recognition task, the Wall Street Journal (WSJ) dataset, a large vocabulary speech recognition task, the Switchboard dataset and a low resource speech recognition task, OpenKWS16. Results demonstrate that our algorithm outperforms a traditional CTC criterion, and achieves 7% WER relative reduction. In addition, we compare our new algorithm to some discriminative training algorithms, such as state-level minimum Bayes risk (SMBR) and minimum word error (MWE), with results supporting the benefits of the new algorithm.

Original languageEnglish
Pages (from-to)1013-1023
Number of pages11
JournalJournal of Signal Processing Systems
Issue number7
StatePublished - Jul 1 2018

Bibliographical note

Publisher Copyright:
© 2017, Springer Science+Business Media, LLC.


  • Connectionist temporal classification
  • End-to-end system
  • Lattice
  • Transcription loss

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Theoretical Computer Science
  • Signal Processing
  • Information Systems
  • Modeling and Simulation
  • Hardware and Architecture


Dive into the research topics of 'Lattice Based Transcription Loss for End-to-End Speech Recognition'. Together they form a unique fingerprint.

Cite this