Fri, 11 Nov 2005
Hacking The Knows Your Name Elmo HOWTO
People have been asking for a HOWTO on messing with Elmo. This is all a work in progress, but I will still share with you my findings so far. There are a couple of things that are unknown at the moment, so if you figure out something new, please be sure to contact me at the email address above.
Elmo Knows Your Name is a semi-interactive audio player with several user inputs and audio output. It features a USB port for programming, and shows up as a HID (Human Interface Device).
I have not dared to crack open this device (its tomorrow's present for god's sake!), so I do not know what exactly is on board. I would expect a very basic processor or programmable logic chip, coupled with some sort of memory, and of course, a USB controller.
After you install the software, I suggest you do the initial configuration. You will get an idea on how this device operates. Ultimately, you will use this to push out your tweaks.
Files of Interest
There are two directories, "names" contains samples of about 15,000 names. "sounds" contains elmo spoken phrases in audio files. The audio extension is "rbf". I have not been able to determine this file format. I have tried to import it as a raw format using a-Law, u-Law, etc. but no luck. There are strings of zeros in the audio files, so I suspect that if compression is actually used, its some sort of ADPCM variant. Without being able to convert these files from a usable format, we cannot create new files just yet -- but keep practicing those Elmo voices, i'm sure its format discovery is just around the corner.
Another file of interest is temp.inf. This contains a sort of scripting file that defines what audio files are played in what order. It also has variables, various groupings to handle games and songs, and a memory map at the bottom of the file.
There are also some files that appear that contain both a human-readable and numerical representation of the variables you configured in the software. These are not too interesting.
doll_ext.bin contains something, but I am not sure what it is. Is it code? Is it a linked and binary version of temp.inf? Who knows. Tell me.
Playing with temp.inf
This cute, innocently named file has most of the guts. Its very easy to follow, but I will document the syntax as best as I currently understand it.
The file starts with "doll_ext.bin". I don't know what this is yet.
Subroutines are defined throughout most of this file, with a label, like PL_001:. There are GAME and STORY labels near the bottom that link to these labels.
The PLAY function allows you to concatinate various audio files, delays, and variables which link to dynamic sounds (such as the child's name). Elements are delimited with +.
Honestly, I found Elmo's cultural insensitivity a little insulting, especially with its very Westernized selection of foods. So for this example, Elmo needs to like to eat whale and seals too, since my daughter loves that stuff we keep in the freezer.
I have seen the following variable elements:
#CHILDSNAME -- child's name (as defined) #FRIEND1 -- friend 1 (as defined) #FRIEND2 -- friend 2 (as defined) #FRIEND3 -- friend 3 (as defined) #PAUSE -- This is a significant pause, enough for a child to respond #FAVCOLOR -- color (as defined) #FAVANIMAL -- animal (as defined) #FAVFOOD1 #FAVFOOD2 #FAVFOOD3 #AGE #FAVDRINK
[nMSEC] -- n Delay in milliseconds <BIRTHDAYSONG> -- some sort of special routine, no references
Linking it all together
If you really want to start screwing around with Elmo, its best to modify a program or game. I suspect that, by comparing behavor and looking at the list of items, the device will select a random line or go in sequence down the list. Here's what I mean:
- PL_024,PL_136,PL_137,PL_138,PL_140,PL_084,PL_619 PL_124,PL_126,PL_128,PL_125,PL_127,PL_129 PL_097,PL_094,PL_134,PL_131,PL_133,PL_135,PL_131A,PL_133A,PL_135A
This plays a sequence of audio files. Now, I have not been daring enough to modify games or create new ones (not sure if it will run GAME9_LABEL just because its there), so let me know what you find out.
Crazy memory table
I don't understand why this device has a bunch of built-in sounds, especially when the software pushes out most of the phrases. This could be just stuff that was built in the demo mode, since the demo is part of this list too. It indicates the byte offset from memory location 0 where the audio exists. I am not certain how it detects "end of file".
INTERNAL_SPEECH_TABLE InternalVersion00 00077571 SP18918 00077597 YOURE_A_GREAT_PRETENDER 00087751 YOURE 00090681 YOUR_BIRTHDAY 00096185 YOUR 00097171 YEAH 00101307 YAY 00112121 WITH_ELMO ..... 00397170 COUNT 00400292 BREAKFAST 00404230 BEDTIME 00408366 AWAKE 00411872 AND 00415192 A1_ELMO_TRYME_8K 00518868 A
I am hoping that A1_ELMO_TRYME_8K offers us a little glimpse into what kind of files we are dealing with. I am guessing that if one word (about a second) equals about 4-5kB, these samples are either operating at 4000Hz, or they are ADPCM (4 bits/sample) at 8KHz audio bandwidth. ... or maybe the demo is just a higher sample rate so it fools us all into thinking this thing is actually inteligible.
No hack would be complete without a demonstration video. The audio is a little raspy (cell phone mic sucks, along with the low Elmo sample rate), but enjoy its glory. Click the movie for the...your know...the movie!
# Figure out how to make that .rbf file! # Figure out if user I/O and game routines are scriptable # Figure out how time events work, or if they are just hardcoded to play a certain label # Figure out if more games can be added, and if elmo will look for an endless supply # Figure out hardware specifications # Figure out how to hack firmware
Assuming that once this little file format issue is figured out, the hacker can actually remove the small, plastic device that goes inside Elmo, and "voice enable" just about any other stuff animal. I think your biggest issue is finding someone to do the vocals, but Dad or Mom could voice-enable a favorite teddy bear while he/she was away on business. I guess that would be cute.