Reproducibility of Survey Results: A New Method to Quantify Similarity of Human Subject Pools

Atieh R. Khamesi, Riccardo Musmeci, Simone Silvestri, D. A. Baker

Research output: Contribution to journalConference articlepeer-review


Smart Connected Communities (SCCs) is a novel paradigm that brings together multiple disciplines, including social-sciences, computer science, and engineering. Large-scale surveys are a fundamental tool to understand the needs and impact of new technologies to human populations, necessary to realize the SCC paradigm. However, there is a growing debate regarding the reproducibility of survey results. As an example, it has been shown that surveys may easily provide contradictory results, even if the subject populations are statistically equivalent from a demographic perspective. In this paper, we take the initial steps towards addressing the problem of reproducibility of survey results by providing formal methods to quantitatively justify apparently inconsistent results. Specifically, we define a new dissimilarity metric between two populations based on the users answers to non-demographic questions. To this purpose, we propose two algorithms based on submodular optimization and information theory, respectively, to select the most representative questions in a survey. Results show that our method effectively identifies and quantifies differences that are not evident from a purely demographic point of view.

Original languageEnglish
Article number9348076
JournalProceedings - IEEE Global Communications Conference, GLOBECOM
StatePublished - Dec 2020
Event2020 IEEE Global Communications Conference, GLOBECOM 2020 - Virtual, Taipei, Taiwan, Province of China
Duration: Dec 7 2020Dec 11 2020

Bibliographical note

Publisher Copyright:
© 2020 IEEE.


  • Dissimilarity Metrics
  • Reproducibility
  • Surveys

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Networks and Communications
  • Hardware and Architecture
  • Signal Processing


Dive into the research topics of 'Reproducibility of Survey Results: A New Method to Quantify Similarity of Human Subject Pools'. Together they form a unique fingerprint.

Cite this