MEBook: Multimedia social greetings intervention for children with autism spectrum disorders

Autism spectrum disorder (ASD) is a developmental disorder that impairs the development of social and communication skills. There is evidence that children with ASD prefer images of self over others[1]. These studies may explain the effectiveness of video self-modeling (VSM), an evidence-based ASD intervention in which one learns by watching oneself on video performing a target behavior. VSM content is difficult to create as target behaviors are sporadic. In this paper, we propose the MEBook system which uses a Kinect sensor to inject self-images into a gesture-based social narrative to teach children with ASD proper greeting behaviors. MEBook consists of two components. The first component is a social narrative. A social narrative is an animated story about the main character meeting and greeting different cartoon characters in a particular setting. Self-modeling is achieved by first replacing the main characters face with an image of the learner, and then animating the learner's body and voice to match the narration. The second component is a positive reinforcement practice session in which the subject is prompted to greet different cartoon characters. Through depth-based body posture tracking, proper greeting behaviors are recognized and immediately reinforced with praise and visual confetti. Novel computational multimedia tools are proposed to turn video recordings of successful attempts into VSM content, thereby alleviating the tedious production process. A multiple-baseline single subject pilot study has been conducted and the preliminary results show that MEBook is potentially effective in teaching greeting behaviors to children with ASD.