MEBook: Kinect-based self-modeling intervention for children with autism

Nkiruka Uzuegbunam, Wing Hang Wong, Sen Ching Samson Cheung, Lisa Ruble

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

23 Scopus citations


Autism spectrum disorder (ASD) is a chronic developmental disorder that impairs the development of social and communication skills. Multiple studies have shown that children with ASD prefer images of self over others. These studies explain the effectiveness of video self-modeling (VSM), an evidence-based autism intervention in which one learns by watching oneself performing a target behavior in video. VSM content is difficult to create as target behaviors are sporadic, but advances in sensing and graphics enable synthesis of such behaviors. In this paper, we propose the MEBook system which uses Kinect sensor to inject self-images into a social narrative game to teach students with ASD proper greeting behaviors. The social narrative is an animated story about the main character meeting and greeting different cartoon characters in a clinic. Self-modeling is achieved by first replacing the main characters face with an image of the subject, and then animating the subject to match the narration. The second component is a positive reinforcement game in which the subject is prompted to greet different cartoon characters. Through depth-based body posture tracking, proper greeting behaviors are recognized and immediately rewarded with praises and visual confetti. A multiple-baseline single-subject study has been conducted and the preliminary results show that MEBook is effective in teaching greeting behaviors to children with ASD.

Original languageEnglish
Title of host publication2015 IEEE International Conference on Multimedia and Expo, ICME 2015
ISBN (Electronic)9781479970827
StatePublished - Aug 4 2015
EventIEEE International Conference on Multimedia and Expo, ICME 2015 - Turin, Italy
Duration: Jun 29 2015Jul 3 2015

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X


ConferenceIEEE International Conference on Multimedia and Expo, ICME 2015

Bibliographical note

Publisher Copyright:
© 2015 IEEE.


  • Autism
  • Gamification
  • Kinect
  • Social narrative
  • Virtual Reality

ASJC Scopus subject areas

  • Computer Networks and Communications
  • Computer Science Applications


Dive into the research topics of 'MEBook: Kinect-based self-modeling intervention for children with autism'. Together they form a unique fingerprint.

Cite this