MEBook: Multimedia social greetings intervention for children with autism spectrum disorders

Nkiruka Uzuegbunam, Wing Hang Wong, Sen Ching Samson Cheung, Lisa Ruble

Research output: Contribution to journalArticlepeer-review

18 Scopus citations


Autism spectrum disorder (ASD) is a developmental disorder that impairs the development of social and communication skills. There is evidence that children with ASD prefer images of self over others[1]. These studies may explain the effectiveness of video self-modeling (VSM), an evidence-based ASD intervention in which one learns by watching oneself on video performing a target behavior. VSM content is difficult to create as target behaviors are sporadic. In this paper, we propose the MEBook system which uses a Kinect sensor to inject self-images into a gesture-based social narrative to teach children with ASD proper greeting behaviors. MEBook consists of two components. The first component is a social narrative. A social narrative is an animated story about the main character meeting and greeting different cartoon characters in a particular setting. Self-modeling is achieved by first replacing the main characters face with an image of the learner, and then animating the learner's body and voice to match the narration. The second component is a positive reinforcement practice session in which the subject is prompted to greet different cartoon characters. Through depth-based body posture tracking, proper greeting behaviors are recognized and immediately reinforced with praise and visual confetti. Novel computational multimedia tools are proposed to turn video recordings of successful attempts into VSM content, thereby alleviating the tedious production process. A multiple-baseline single subject pilot study has been conducted and the preliminary results show that MEBook is potentially effective in teaching greeting behaviors to children with ASD.

Original languageEnglish
Article number8103797
Pages (from-to)520-535
Number of pages16
JournalIEEE Transactions on Learning Technologies
Issue number4
StatePublished - Oct 1 2018

Bibliographical note

Publisher Copyright:
© 2008-2011 IEEE.


  • Kinect
  • Self-modeling
  • autism spectrum disorder
  • behavioral intervention
  • motion-based
  • social skill learning
  • video self-modeling

ASJC Scopus subject areas

  • Education
  • General Engineering
  • Computer Science Applications


Dive into the research topics of 'MEBook: Multimedia social greetings intervention for children with autism spectrum disorders'. Together they form a unique fingerprint.

Cite this