Linguistically-enriched and context-awarezero-shot slot filling

A. B. Siddique, Fuad Jamour, Vagelis Hristidis

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

19 Scopus citations


Slot filling is identifying contiguous spans of words in an utterance that correspond to certain parameters (i.e., slots) of a user request/query. Slot filling is one of the most important challenges in modern task-oriented dialog systems. Supervised approaches have proven effective at tackling this challenge, but they need a significant amount of labeled training data in a given domain. However, new domains (i.e., unseen in training) may emerge after deployment. Thus, it is imperative that these models seamlessly adapt and fill slots from both seen and unseen domains - unseen domains contain unseen slot types with no training data, and even seen slots in unseen domains are typically presented in different contexts. This setting is commonly referred to as zero-shot slot filling. Little work has focused on this setting, with limited experimental evaluation. Existing models that mainly rely on context-independent embedding-based similarity measures fail to detect slot values in unseen domains or do so only partially. We propose a new zero-shot slot filling neural model, , which works in three steps. Step one acquires domain-oblivious, context-aware representations of utterance words by exploiting (a) linguistic features such as part-of-speech tags; (b) named entity recognition cues; and (c) contextual embeddings from pre-trained language models. Step two fine-tunes these rich representations and produces slot-independent tags for each word. Step three exploits generalizable context-aware utterance-slot similarity features at the word level, uses slot-independent tags, and contextualizes them to produce slot-specific predictions for each word. Our thorough evaluation on four diverse public datasets demonstrates that our approach consistently outperforms state-of-the-art models by 17.52%, 22.15%, 17.42%, and 17.95% on average for unseen domains on SNIPS, ATIS, MultiWOZ, and SGD datasets, respectively.

Original languageEnglish
Title of host publicationThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021
Number of pages12
ISBN (Electronic)9781450383127
StatePublished - Apr 19 2021
Event2021 World Wide Web Conference, WWW 2021 - Ljubljana, Slovenia
Duration: Apr 19 2021Apr 23 2021

Publication series

NameThe Web Conference 2021 - Proceedings of the World Wide Web Conference, WWW 2021


Conference2021 World Wide Web Conference, WWW 2021

Bibliographical note

Funding Information:
This work is supported in part by the National Science Foundation (NSF) under grants IIS-1838222 and IIS-1901379.

Publisher Copyright:
© 2021 ACM.


  • Cross-domain zero-shot slot filling
  • Natural language processing.
  • Natural language understanding
  • Slot filling
  • Zero-shot learning

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Software


Dive into the research topics of 'Linguistically-enriched and context-awarezero-shot slot filling'. Together they form a unique fingerprint.

Cite this