Overview
The main goal for Qooco is to provide motivation and practice in helping Chinese students learn to speak English.  The games are always in conjunction with a language learning center with trained instructors.  The games are analogous to drill exercises.  Motivating students to do more drills can sometimes get them to a place where they have made progress and can then be more self-motivated. The goal is a hybrid of fun and education.  Qooco has about 60 employees.

Types of Games
There are about a dozen different game types. The games need to be rather constrained in order for speech recognition to work well. An example of the games used is Jeopardy style, in which a question is asked or answered or a blank filled in. This constrains the speech recognition enough that it can give feedback on the correctness of the answer and the goodness of the pronunciation.  Sometimes the games are ‘shooting’ games, but the shooting is at words to select them for sentences. There are also some cooperative games in which students play as a team and help each other. Some of the questions have more than one correct answer, but usually there is just one.  In addition to language games, there is also a math game --- so that language instruction helps support math instruction and vice versa. Typically the games are used as a reward for making a certain amount of progress in a lesson.  (Points are very motivating).

The Scoring
The threshold for scoring has 3 levels (easy, medium and strict). When the student misses the same questions three times in a row, it is noted, but the game moves on to prevent frustration.  For some languages, tone patterns as well as pronunciation needs to be scored.

Languages and Audience
Most of the company’s work is devoted to teaching English in China, though they are also teaching Mandarin. Most of the students are 6 – 12 years old.

The Technology
The technologies used include automatic speech recognition and tone recognition.  Qooco is working with 3 technology providers: Nuance, IBM, and ATT.  There is a rich client that can be downloaded to a desktop, but the internet is required to play. Subtitles can be toggled on and off to assist in adjusting the difficulty level.  The audio provided by the students is reused to create better acoustic models.  Qooco is also offering a client-less service for a mobile platform, using the video conferencing capabilities of 3G.  The current tools are web-based but not the consumer level.

Pedagogy
Lessons typically start with vocabulary exercises, then listening, then repeating.  The approach is basically communicative but includes a portfolio of methods.  Since much of English teaching in China has no native speakers of English, the focus is oral communication.

Evaluation
The software system plus center has been evaluated against the normal teacher-only situation.  The software assistance enables reaching English proficiency levels about five times more quickly. Of course with the software compared to the control, the student typically has more time on task and more access to native speakers.   The kids in the test school are now about one year ahead of the kids in the control group.

The Business
The initial approach was a purely online approach, but that did not work well with kids. Parents will pay for their children to learn English, but live instruction needs to be a part of what they want to pay for. The model is now a franchise model – there is some online instruction in conjunction with a language school. There are learning centers and with VOIP the students have access to the software in evenings and on weekends. It’s not clear where all the centers will go:  Perhaps near housing areas, or near shopping areas, or both.  When the parents go shopping the kids could be dropped off at a center.  Qooco has received some inquiries from mobile carriers in India and there might be growth in rural areas.  There is already a clientless version running on a social network in China, and this could be a model.  It runs in real time and takes seconds to set up.  Of interest is the Jay Walker TED talk on ‘English Mania’ http://www.ted.com/talks
/jay_walker_on_the_world_s_english_mania.html

Qooco has been one of the sponsors for the Scripps spelling bee, and for the last 2 years, the Chinese winner was a student who uses Qooco.
http://business.globaltimes.cn/comment/2010-03/513219.html

NOTES:
Background on David Topolewski:
David Topolewski is the CEO of Qooco, which provides online spoken English and Mandarin language training on the Internet and mobile phone networks.  At Qooco, he has responsibility for strategy, finance, service development, and business development.  He has over ten years of experience in the English as a Foreign Language (EFL) field, complementing his decade of experience in investment banking (Smith Barney, HLHZ), software development, and international business, including positions as general manager and CFO.  Through various software businesses, Mr. Topolewski has produced dozens of software programs, on several different platforms, ranging from Anpanman for Bandai in Japan, Jackie Chan's Stuntmaster for the PlayStation, TuneLand, Monty' Python's Complete Waste of Time, and numerous EFL titles.  He has taken several companies public, both as an investment banker and as principal, recruited top talent in sales, entertainment, technology, education, and management.  Mr. Topolewski  is a member of the  Yale University Digital Committee established by Yale's president  to advise on digitizing Yale's intellectual property.  He earned his MBA at The Wharton School at the University of Pennsylvania and his BA in Economics at Yale University.

Speaker:
David Topolewski, CEO and Founder, Qooco

Discussants:
Margaret Boothroyd
Jason Brenier
Mike Cohen
Farzad Ehsani
Yoon Kim
Nikki Mirghafori
Patti Price
Fuliang Weng
Jing Zheng

Tessa Ten Tusscher's talk:
Tessa's background is as a geriatric psychologist. The old are often divided into the old old (>85) and the younger old (>65). Her company's goal is to help people age the way they want to age, and 89% of them would like to stay in their own homes. Usually there is some triggering event that causes them to move into assisted living. The home environment can be made more supportive for older people, even after these triggering events. The current population of seniors is not on average very comfortable with technology, so the focus has been providing high touch solutions rather than high tech solutions. However, technology is of interest in that it can help to bring down costs. The approach is holistic, considering the individual capacities, recent changes, desires and current living situation.The brain goes through a number of sensory changes as we age. And many technology experts who design user interfaces do not pay sufficient attention to these. However:

• a significant number of people over age 80 start to have signs of some dementia.
• hearing degrades (there is both loss in higher frequencies, as well as difficulties in hearing and understanding faster speech and speech in background noise).
• cognition (it becomes harder to think about more complex things) and reaction time degrades
  
• vision degrades
  
• aphasia may also result from a stroke

On the other hand, the younger old who have grown up with technology may be suddenly surprised to find that things they were used to finding online are no longer available once they turn 65,since many institutions have decided that the elderly do not use the internet, particularly with Medicare.  Young and old tend to give up in the face of consistent failure, and as we age, things do get harder (since hearing, vision, reaction time, etc. are degrading -- and there may also be arthritis, tremors and other factors that make things more difficult).

LivingWelllAtHome visits each potential client, in their home to perform a series of cognitive tests, measure attention, short term memory, etc.Depending on general health there may be more sophisticated testing. Compare to any recent tests, diagnoses, medical history, medications, recent events (e.g., stroke --- these are usually precipitators for considering assisted living).What are strengths and weaknesses. How to accommodate.Check for depression (which sometimes masquerades as mental loss).There is a tension between too much accommodation (which could lead to further degradation of function) and too little (which is frustrating and limiting).But the attempt is to try to figure out what is possible from a functional point of view and to try to build up to that point, while accommodating function loss that is not recoverable.(analogously, a lost leg may require a crutch, while a sprained ankle needs rest and then rehabilitation to bring back function).

The technology currently being used by LivingWellAtHome is largely industrial technology (e.g., site monitoring technology) adapted to a home environment.The solution is a combination of modifying the home and service provision. Clearly costs are reduced if there is a good technology solution to service costs.

Deborah Dahl's Talk:

1. Why design for seniors (variety of social reasons... and we all tend to aspire to become old).
2. How can speech technology help?
   a. Some are applications that anyone can use, but accommodate the senior demographic
   b. Assistive applications, that help seniors in daily life
    c. Rehabilitative applications that help improve function (e.g., aphasia therapy -- which can help the young as well as the older aphasics).
3. Changes that occur with aging: physical, psychological, motor, etc., but physical (hearing, seeing, dexterity...) and cognitive (memory, attention, language processing... ) are the most noticeable. All of these changes should affect how we design applications.

General Discussion

WHAT DEGRADES AS WE AGE? WHAT MIGHT IMPROVE?
Although seniors have degrading cognition, sensory input, etc., some things may get better.Perhaps they have learned to be more patient?Although it seems that patience may be in some areas but not necessarily all. Ability to focus and to find things (on a computer screen for example) seems to degrade. Seniors today tend to have older, smaller screens with lower resolution and larger fonts, so less fits there and things ARE harder to find. There is a tendency to focus on developing NEW technology to assist seniors in various ways.But we must also remember to adapt current technology to make it easier to use as people age.

WHY MULTIMODALITY?
Multimodality is important for many reasons (for seniors in particular, but for general population as well):

• One modality can be substituted for another (e.g., spoken outputs for low vision individuals, and visual information displays for those hard of hearing)
  
• More often both vision and hearing are a bit impaired, and the two together allow comprehension.
  
• Having the information presented in more than one modality seems to aid memory and understanding generally

WHY CONVERSATIONAL SYSTEMS?
Although many of today's seniors prefer high touch to high tech, this may not be as true of tomorrow's seniors as it is of today's cohort of seniors.And it is not entirely true of seniors today. Some older people seem to prefer the high tech to the high touch, if they feel it works well and gets their job done without bothering or annoying anyone.But, perhaps because of the degradation of various physical and cognitive skills, it may be useful for seniors to break down interaction into smaller steps, so that there are many simple steps rather than a few more complex ones.This, too, may be true of the more general population.For example, Wallace Chafe's work at UCSB shows that intonational units tend to have 1 new idea and last about 1.2 seconds.  Wallace Chafe, Discourse, Consciousness, and Time.

The function of interactivity in conversational interfaces therefore goes beyond just making them more human-like and chatty.A crucial aspect of both human-human and human-machine interactions is to break a task into small, easily confirmable components. Many seniors may be highly motivated to avoid 'bothering' a human for information, and are perfectly capable of finding information themselves on the web.However, if the information goes against their expectations, they may prefer to have a human being say it, to be sure of it. The simple rule of many small one-piece-of-information units may be broken if for any reason the interaction is very slow (e.g., talking via teletype or via an interpreter).In this case, if there are 5 - 10 second delays, the preference for smaller units changes to longer, more complex ones.

HOW IS TECHNOLOGY CURRENTLY USED WITH CARE FOR SENIORS?HOW MIGHT IT BE USED?
Since most seniors say they would prefer to stay in their own homes as they age and since this often is a lot less costly than alternatives to individuals and to society, we should be motivated to find ways to enable that outcome.One way to keep down costs is to use technology where it is as effective as humans but less costly. Here are some ways technology is currently used:

• Medication management: primitive medication reminders, and request to input information about having taken medication. However, hearing an alarm at 3 pm may not be sufficient to let the person know to 'take that long blue pill'.
  
• Home security: use voice activation to open a door.Using biometrics such as voice identification for the senior as well as caretakers would be very useful, to help control who enters the home.Most seniors do not want to wear those pendants that help alert others to their distress. But a more general system that could, e.g., recognize screams would be useful.
  
• GPS might also help track who is where (for the senior and for care givers).
  
• There are some photo frames with a prerecorded message from, e.g., a granddaughter that are not so useful for most seniors, however, a device that could do this as well as other easy messaging tasks might be useful.
  
• Reminder systems often seem more appreciated by family members who feel they are doing something but they don't seem to get a lot of use by seniors.
  
• Small RFID tags on, e.g., glasses, medicines, etc. could be able to alert someone if the medecines have been touched, and make them easier to find.
  
• There are some wireless heart monitors in common use now, and many other systems for monitoring body functions in clinical trials.
  
• Wii and other games are used a lot in senior centers and can help with physical fitness and reaction time and coordination.
  
• Posit Science has a program for brain fitness but it focuses on a very narrow set of skills and is so boring that most people do not do them (just as they don't usually keep up gym membership).More generally --- to keep your brain fit: just do new things that are hard for you, trying to get better.(this is also how you keep your body fit).

WHAT DATA DO WE HAVE?
There is not a lot of data on voices as we age.SRI and ICSI have collected some data from seniors, from elderly care centers, and in their homes.The database consists of conversations between the elderly and another person, assessing activities, cognitive status (e.g., read a story and ask them to retell).There are also some older papers on the voices of seniors, particularly from ATT and Dragon. These data seem to indicate that speech recognition does degrade significantly for older speakers -- even when test and training are both using only older speakers. There are issues with false teeth, more variability (perhaps related to motor control), and lower amplitude levels -- sometimes their voices are very hard for humans to understand.

Speakers:
Tessa Ten Tusscher, CEO and Founder, Living Well at Home
Deborah Dahl (by phone), head of Conversational Technologies. She has been exploring speech and language technology for seniorsrecently made a presentation at SpeechTek on the topic of Designing Speechand Multimodal Applications for Senior Users

Discussants:
Jared Bernstein, founder Ordinate Corporation and Consulting Scientist at Pearson Knowledge Technologies.
Dilek Hakkan-Tur, Senior Researcher at ICSI
Mary Constance Parks, Senior Voice User Interface Designer at Nuance, Inc.
Horacio Franco, Chief Scientist, Speech Technology and Research Lab, SRI International
Marikka Rypa, an Instructional Design Professional
Mike Cohen, Manager, Speech Technology, Google
Patti Price, consultant in speech and language technology.  Recent article for Speech Strategy News on Speech-Enabled Memory Assistants

Not able to attend but with useful information on this topic to share:

Francois Andry, Software Architect at IntercomponentWare, Inc.blogs on technology in health care
Michael, a social worker from the New York City area, who also made a SpeechTek presentation on Senior Friendly Design

Recently, I was invited to participate to a meeting of AVIOS (Applied Voice Input Output Society) local chapter in Menlo Park, California to discuss the use of speech technology in HealthCare and Medicine.

Speech technology is not widely used in healthcare at the moment. The healthcare industry is still focusing on efficiency and cost saving by solving systems integration, re-wamping legacy systems and making paper based processes digital.

Dictation and transcription for EMRs

The most successful type of applications is probably dictation/transcription for electronic medical records (EMR).

Speech recognition, enhancing productivity and reducing medical costs as a result. In this domain, Nuance is one of the market leader.

The products and services of Nuance Healthcare are used at more than 5,000 hospitals and by more than 400,000 providers.

Nuance’s on-premise Dictaphone Enterprise Speech System (ESS) components support recording the dictation audio, managing the audio and the resulting transcription process, transcription document distribution, and online reporting.

Dragon medical is another product from Nuance with which physicians dictate in real-time into their EMR in their own words letting them instantly review, sign, and make their notes available for other clinicians. This include Medical Vocabularies, covering nearly 80 specialties and subspecialties and regional accent support.

Potential use for Care and Disease Management

Besides dictation/transcription there are some additional potential uses of speech technology in other domains related to heathcare. Care and Disease Managament is one of them.

Organizations/companies like Lifemasters have call centers where nurses help to help patients with chronic disease achieve optimal health by closing the gaps between recommended and actual care (evidence-based medicine) and encouraging patients to adopt a healthy lifestyle and reducing the overall cost of healthcare. Speech technology could be used to complement and optimize this type of services: virtual operators, voice activation services, automatic text-to-speech combined with voice-over-IP reminder and notifications (e.g. to take drugs, schedule an exam, do certain types of exercices etc ...). Companies like requall and Jott already provide this type of notification infrastructure that can be delivered on mobile phones for example.

Other Areas

Additional research areas in speech technology and healthcare include:

• Using speech/sound recognition to monitor eating behavior
• disease assessment (e.g. parkinson over the phone, although this disease is very complex and voice is probably not the most important factor)
• using speech recognition for medical transcript anonymization to be used as corpora