Speech controlled games – lessons learnt
In making Speakeasy, I had great dreams for a fast paced arcade game using speech as the input system but ended up with a slow paced game with word recognition issues. I worked with speech recognition systems before (Annosoft’s), but only to detect phonemes in real-time. I just assumed word recognition was almost as fast and painless – I was wrong – The freely available Microsoft recognizer was wild inconsistently. It’s poor word recognition severely hampered gameplay mechanics causing me to change my original game design.
For example: The Rap A-long mode (aka “Parappa the Rapper”) asks the player to rap along with the Master Chop Chop song. But unlike the PS1 game with a joystick, the speech system can not be relied on to report a word within a finite amount of time. And the recognizer also incorrectly guesses words if forced into an early decision. So the gameplay was changed to allow a large recognition window making it less challenging and slowing the pace quite considerably.
In building a speech-controlled game, I realized:
What can be relied on:
- Recognition for a limited vocabulary
- 80-90% recognition rate.
- Pre-installed on all Windows XP systems
What can’t be relied on:
- Fast AND accurate recognition
- Recognition of multiple words slurred together – like “Kick Punch Block” spoken quickly.
- 100% accuracy. Even at the highest recognition settinga and clearly spoken words, the system will report false positives.
Because of these limitation, no speech controlled game can be built with the assumption of “fast reliable input” like that of a haptic input device like a mouse or keyboard or joystick. Still, the lure of a speech controlled game is too great for me. It’s weird. It’s funky. It’s different…. and I’m all about different. For future speech controlled game designers, I have a list of pros and cons before you dive in and create a speech-based game:
Pros:
- Word/sentence/language oriented
- Wide input spectrum (i.e infinite words vs. 6 buttons)
- Unique selling point: Not a mouse or keyboard or joystick
- “Telling” a computer what to do and having it do it is a great feeling
- Players who love their sound of their voice, will love speech games
Cons:
- Latency issues
- Accuracy issues
- Microphones not as common as mouse and keyboard
- Noisy backgrounds cause poor recognition
- Accents cause poor recognition
- Game become language specific
Anyhow, I’m onto my next speech-controlled game: ‘You sunk my Battleship!”
Add comment May 16th, 2007