MEBook: Kinect-based self-modeling intervention for children with autism

Autism spectrum disorder (ASD) is a chronic developmental disorder that impairs the development of social and communication skills. Multiple studies have shown that children with ASD prefer images of self over others. These studies explain the effectiveness of video self-modeling (VSM), an evidence-based autism intervention in which one learns by watching oneself performing a target behavior in video. VSM content is difficult to create as target behaviors are sporadic, but advances in sensing and graphics enable synthesis of such behaviors. In this paper, we propose the MEBook system which uses Kinect sensor to inject self-images into a social narrative game to teach students with ASD proper greeting behaviors. The social narrative is an animated story about the main character meeting and greeting different cartoon characters in a clinic. Self-modeling is achieved by first replacing the main characters face with an image of the subject, and then animating the subject to match the narration. The second component is a positive reinforcement game in which the subject is prompted to greet different cartoon characters. Through depth-based body posture tracking, proper greeting behaviors are recognized and immediately rewarded with praises and visual confetti. A multiple-baseline single-subject study has been conducted and the preliminary results show that MEBook is effective in teaching greeting behaviors to children with ASD.