The game is a variation of the popular typing game where words fall off the screen and you need to type them before they get to you. The slight variation consists in changing the game mechanics from typing to speaking. In our game, letters will fall off the screen and the player will need to say a word starting with that letter to earn points, destroy the letter and avoid certain annihilation.
At the end of the last article we had completed the beginnings of a game with a beautiful black background and some letters falling off the screen. Let’s continue building our game by adding some player interaction through the Web Speech API!
The Web Speech API
The Web Speech API let’s you incorporate voice interaction right into your web applications. You can either take advantage of the Speech Synthesis API to make your computer talk to your users or the Speech Recognition API to teach your programs to hear and recognize your users enabling new and cool interactions.
Prior to this hackathon I had experimented a little with the Speech Synthesis API but I had done nothing at all with Speech Recognition. Because of that and the super-uber-mega-tight schedule of the hackathon I chose the easy way forward and used the annyang library to cover my speech recognition needs.
Using Annyang to Gather Player Input
The annyang library offers a super simple API to recognize a person’s speech. You define
commands using text patterns and tie these patterns to callback functions. For instance, if you want your application to do something when the user says
cucumber you define a command like this:
1 2 3
If you want to extract part of the user’s speech you can do it as well. Imagine that you want to order a pizza with arbitrary condiments, you could type the following:
1 2 3 4 5 6
*ingredients part of the command would be recognized by the Speech Recognition API and passed as an argument to your function.
In our particular case we want to be able to understand words that we can later match with the letters that are falling down the screen. For instance, if an
l appears on the screen, the player could start shouting
lentils! lenses! lotion! to eliminate it and earn… let’s say… 100 points.
We can set up a simple command to recognize anything spoken by the player like this:
1 2 3 4 5 6
And you hook up the commands and get the speech recognition started with these two simple lines:
1 2 3 4 5
But how can we incorporate this into the game? We need to somehow wrap this user input in an observable. There are different ways to achieve this but the one I went for was using a
Subject is a special Rx.js object that doubles as an observer and an observable. When it plays the role of an observer it can subscribe to other observables or be pushed new values like the words pronounced by a player. And as an observable it can emit these values and you can combine it with other observables or subscribe to it.
Curious About Subjects?
You can create a subject via the
Once created we can update
commands to push values into the subject (taking advantage of its observer facet):
1 2 3 4 5 6 7
Now we need to include these words into the game loop. Each time a player says a word, we want to compare that word with the letters that are falling off the screen and if there’s a match, we want to eliminate that letter and increase the user’s score.
We can create a new observable
newWordAndGame$ to represent the status of the game every time a user says a new distinct word:
1 2 3 4 5
This new observable combines the
userWords$ and the
game$ observables. If you don’t remember how the
game$ observable looked like here you have a refresher:
1 2 3 4 5
distinctUntilChanged operator ensures that new values are only emitted when the player says a new word. We can subscribe to the resulting observable and perform our word matching to see whether or not we can eliminate a letter from the screen and give the player some points:
1 2 3
matchWords method will implement the matching logic. It could look like this:
1 2 3 4 5 6 7 8 9
findLetterMatchingWord would retrieve the first letter of those in the screen that is the starting letter of the word said by the player (for instance, if the player says
violoncello that would be the first
v within our
1 2 3
removeLetter function would remove the matched letter from our collection of letter game objects:
1 2 3 4
updateScore function would update the score. We will leave it empty for the time being since we don’t have any score yet in our game:
1 2 3
Since we need a convenient way to access the
letters within our game objects (The
gameObjects.letters property doesn’t exist yet at this stage) we update the
game$ observable to expose them:
1 2 3 4 5
This means that we’ll have to make a small update in our
subscribe method to conform to the new API:
1 2 3 4 5 6 7 8 9
If you run the game now you should be able to interact with it as a player. As soon as you open the game in your browser you’ll be prompted to Allow or Block access to the microphone. Allow it and when you see a letter falling down the screen say a word that starts with that letter. The letter should disappear.
If that doesn’t happen check the console and see if there are any errors. You can also add a visual help to see the words that you utter directly within the game. Add the following element to your html markup:
And update the
userWordsAndGame$.subscribe to append the uttered words to the
1 2 3 4 5 6 7 8 9 10 11
Now you should be able to verify the words or sentences recognized by the Web Speech API more readily as they’ll appear on the screen beside the canvas.
Why didn’t We Include userWords$ in the Original game$ Observable?
Now you may be thinking… Why didn’t we make the
userWords$ be a part of the
game$ observable like we did with the previous ones?
Well if we had done so it would’ve been more difficult to know when a user had uttered a word. Since
combineLatest emits a value any time a combined observable emits a value we would have had new values being pushed on every game tick, every letter, every word uttered, etc…
Separating word handling into a different observable makes it easier to compose it with the
distinctUntilChanged operator and unequivocally know when the user utters a new word.
Let’s continue by adding a way to manage scores. We need to be able to both compute scores and have a way to display them within the game.
We will create a new game object to represent the idea of a
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
And now that we have a way to represent scores we need to include it in the game. Just like we did with the player words, we will create a subject so that we can push in new scores as the user words match the falling letters:
We can now implement the
updateScore method from before:
1 2 3
And we will add it to our game loop so it can be drawn in the screen. In order to do that we create the
score$ observable that takes the score values and maps then into
Score game objects:
1 2 3 4
We use the
scan operator much in the same way that we did in the first part of the series but in this case to accumulate scores.
The next step is to include the score in our game loop:
1 2 3 4 5
And now if you start your game, you’ll be able to see a
0 standing in the top left-most corner. If you say a word that matches any of the letters in the screen you should be able to see how your score increases. So say a word and your score goes up… 200!???
Wait… There’s Something Weird Happening? Every time I say a word I get 200 points! Wat!??
So, you wanted the player’s score to go up 100 points whenever he uttered the right word, but somehow, any time you say a word that eliminates a letter you are getting 200 points. Moreover, if you look closely it looks like there’s 2 words falling down every 2 seconds instead of only one. What’s happening here?
Well what is happening is that we have created a cold observable with two subscribers when what we really wanted was a hot observable.
Cold? Hot? Waaaaat!?
Cold and Hot Observables
The type of observables we have been creating in this game are what are known in Rx.js jargon as cold observables. Cold observables only start running when someone subscribes to them. That is, they are inert, dead, they don’t do anything at all until someone calls
suscribe on them.
Moreover, the values pushed by a cold observable are not shared accross subscribers. If you, for instance, have a cold observable that produces a sequence of integers with two different subscribers, the whole sequence of integers will be pushed to both subscribers regardless of when they subscribed to the observable:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
You can find this example in JsFiddle if you’re curious and want to tinker yourself.
If now that you know about cold observables you take a second look at our game you’ll realize that we have two subscribers that are subscribed to the
game$ observable. As a result of that, we have double the
letters being created and double the scores being accumulated.
What we want is to use the same sequence regardless of the number of subscribers. What we want is a Hot observable. Hot observables, in opposition to cold ones, emit values even before they have active subscribers. You can convert cold into hot observables in different ways and, in this case, we will use the
We can modify the previous example to illustrate its use:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22
So! Just like in this example, we can use the
share operator in our
game$ observable to fix the problem:
1 2 3 4 5 6
Now you should be able to go back to game and see how everything works as it is supposed to. Yeeey!!!
Great! Now we have a game that you can play! Cool right?
We have built on top of what we had done in the previous article adding player interaction through voice commands and we are able to affect the state of the game by removing letters and increasing the player score. We’ve also learnt some new Rx.js concepts like subjects and hot and cold observables.
There’s still some stuff left though, for one, the game never finishes. It doesn’t matter how many letters fall off the screen you can continue playing, and that sucks, because one of the fun and exciting things about games is that you can lose.
So in the next article of the series we will focus on a way to represent player lives and add some thrill to the game. We will wrap the series by adding some graphics, doing some refactoring and reflecting over the whole business.
Would you Like to Learn More About The Web Speech API and Rx.js?
Take a look at these great articles: