From Magic mirror to Autism: using computational multimedia for video self-modeling therapy

From Magic mirror to Autism: using computational multimedia for video self-modeling therapy                                                        
Sen-ching Samson Cheung  
University of Kentucky 

Davis Marksbury Theater, 4:15 -- 5:15 PM, Wednesday, September 19th                                                                                 
If you have ever been on a diet, you would know that having an old picture of a thinner "you" on the bathroom mirror or the refrigerator can be a powerful motivating tool. This is the idea behind the psychological theory of self-efficacy -- you can learn  or model to perform certain tasks because you see yourself doing  it. Even more powerful is to use modified imagery to trick your brain into believing a positive personal experience that has not occurred before. Perhaps the most famous example is the mirror box therapy for amputees -- seeing the mirror reflection of an intact arm conjures up an illusion of the presence of the amputated limb and provides an effective mean to alleviate the phantom limb pain.  Another example is video self-modeling (VSM) therapy in which the   patient acts as his/her own model by taping several hours of  behavior in video and splicing together a chain of clips depicting  the new skill for instruction. Mirror and VSM therapies are widely used in the treatments of many neurological disorders. In practice,  these therapies are challenging to administer due to the difficulties in constructing the mirror device or preparing the appropriate video content. This is where multimedia engineers can help -- by creating fictitious multimedia contents that requires little manual input and are perceptually indistinguishable from those captured by sensors. In this talk, I will discuss a number of  on-going projects in my lab that use computational multimedia approaches for therapy. I will demonstrate our SpeakToMe system for  creating VSM content for voice-disorder patients. The system replaces the coarse voice of a patient with a perceptually-similar but healthy voice, and then renders a new video sequence with automated lip-synchronization. I will also discuss the MagicMirror system for helping children with autism spectrum disorders in their behavioral therapy. The MagicMirror system is a large-size computer display that uses a network of depth and color cameras to render a  view-dependent mirror image of a virtual mirror. By injecting new behaviors into the mirror images, we are hoping to provide the instant visual feedback that is crucial to learning but completely lacking in the current form of VSM.                                                
Sen-ching (Samson) Cheung is an associate professor from the Department of Electrical and Computer Engineering of the University of Kentucky (UKY). He also has a joint appointment with the UKY Center of Visualization and Virtual Environments. Before joining UKY in 2004, he was a computer scientist in the Scientific Data Mining group at Lawrence Livermore National Laboratory. Samson got  his Ph.D. from University of California, Berkeley in 2002. His work spans a number of different areas in multimedia including video  copy detection, data mining, video surveillance, privacy protection, encrypted-domain signal processing, and computational  multimedia for therapy. He is an associated editor of IEEE Transactions of Multimedia, Signal Processing: Image  Communications, Statistical Analysis and Data Mining, and EURASIP Journal on Information Security. He is a senior member of both IEEE  and ACM.