Speech Recognition in the browser

Three Strategies

• Speech manually captured and processed in the browser (Firefox and Chrome, using pocketsphinx.js)

• Speech automatically captured by the browser and processed in the cloud (Chrome WebSpeech API)

• Speech manually captured by the browser and processed in the cloud (wit.ai, api.ai,IBM, Nuance NDEV)

 

pocketsphinx.js

• local processing in the browser

• based on CMU Pocketsphinx

• available in browsers that support Web Audio

• grammar only (JSGF format) no dictation

• English and Mandarin

​

Web Speech API

• WebSpeech API in Chrome, Since Chrome version 31

• most Chrome platforms are supported now

• dictation only (no grammars)

• find out about support at Can I Use?

Some resources:

• docs

• tutorial

• TTS tutorial

• dictation demo

• captioning demo

• conformance tests

English and Mandarin

​

Manual Server-based Speech Recognition

“Manual” means that the developer is responsible for capturing the audio and sending it to a server for processing

• Microsoft Project Oxford Speech Recognition

• IBM Watson

• Nuance NDEV

• AT&T Speech API

 

To capture speech in the browser:

• HTML5: (getUserMedia())

• WebRTCis a free, open project that provides browsers and mobile applications with Real-Time Communications (RTC) capabilities via simple APIs.

 

Text-to-Speech (TTS)

TTS in the browser

• SpeechSynthesis API

• implemented in Chrome and Safari

​

Server-based TTS

• Microsoft Project Oxford Text to Speech

• Neospeech

• AT&T Speech API

• IBM Watson Text to Speech

​

Natural language understanding

• wit.ai (Facebook)

• api.ai

• Linguasys

• Wolfram Alpha

• Language Understanding Intelligent Service (LUIS) Note: Private beta. By invitation only

 

Combinations

• Speech recognition and natural language understanding

• API.ai

• wit.ai (Facebook) provides a Javascript library

​

​

Native (non-browser) Mobile Speech Recognition and TTS

Many of the browser-based tools described above also include versions for native OS’s

• MIT App Inventor (Android, very easy to use)

• Android Speech Recognition

• Nuance NDEV

• iSpeech (for Android and iOS)

• CMU PocketSphinx (open source)

 

Dialog Processing

OpenDial

• OpenDial in Github

• Open Dial

• Pandorabots: Build your own Chatbot

 

CMU Ravenclaw

• CMU Ravenclaw

• Ravenclaw wiki

• api.ai

 

Speech Recognition Development

Language Modeling Toolkits

• CMU LM Toolkit

• Statistical Language Modeling using the CMU-Cambridge Toolkit

• Typcial Use

SRI Toolkit

• SRILM Toolkit

• SRILM Tutorial

• SRILM at Sixteen: Update and Outlook Stokcke, et. al

 

Open Source Speech Recognizers

KALDI
Kaldi is a toolkit for speech recognition written in C++ and licensed under the Apache License v2.0. Kaldi is intended for use by speech recognition researchers.

• Kaldi ASR

• Kaldi on Github

​

CMU Sphinx
CMUSphinx represents over 20 years of CMU research, with state of art speech recognition algorithms for efficient speech recognition. CMUSphinx tools are designed specifically for low-resource platforms

• CMU Sphinx D

• More about speech at CMU

 

Evaluting speech recognition

NIST SCLite
Asclite is multi-dimensional extension of the Dynamic Programming solution to Levenshtein Edit Distance calculations capable of evaluating STT and SASTT systems during periods of overlapping, simultaneous speech

• Speech Recognition Scoring Toolkit (SCTK) Version 2.4.0

• Documentation

 

Other tools

Sox: The “Swiss Army Knife” of audio processing

• Download Sox

​

LDC’s original Transcriber

• Download Transcriber

• Note: Only runs on Windows