Football game using Speech Recognition in Unity

Speech recognition is an interdisciplinary subfield of computer science and computational linguistics that develops methodologies and technologies that enable the recognition and translation of spoken language into text by computers.

There are various solutions are available to enable STT in unity platform. Here we are going to use the UnityEngine built in Windows.Speech package (Available on Unity 2019.2+). The heart of the module is the KeywordRecognizer, Which will returns the confidence of the keyword.

recognizer = new KeywordRecognizer(keywords, confidence); recognizer.OnPhraseRecognized += Recognizer_OnPhraseRecognized; recognizer.Start();

I have created a Unity script (SpeechRecognitionEngine.cs) which will find the confidence of left, right, top and bottom directions and used for the football game navigation.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.UI;
using UnityEngine.Windows.Speech;
public class SpeechRecognitionEngine : MonoBehaviour
{
public string[] keywords = new string[] { "up", "down", "left", "right" };
public ConfidenceLevel confidence = ConfidenceLevel.Medium;
public float speed = 1;
public Text results;
public Image target;
protected PhraseRecognizer recognizer;
protected string word = "right";
private void Start()
{
if (keywords != null)
{
recognizer = new KeywordRecognizer(keywords, confidence);
recognizer.OnPhraseRecognized += Recognizer_OnPhraseRecognized;
recognizer.Start();
}
}
private void Recognizer_OnPhraseRecognized(PhraseRecognizedEventArgs args)
{
word = args.text;
results.text = "You said: <b>" + word + "</b>";
}
private void Update()
{
var x = target.transform.position.x;
var y = target.transform.position.y;
switch (word)
{
case "up":
y += speed;
break;
case "down":
y -= speed;
break;
case "left":
x -= speed;
break;
case "right":
x += speed;
break;
}
target.transform.position = new Vector3(x, y, 0);
}
private void OnApplicationQuit()
{
if (recognizer != null && recognizer.IsRunning)
{
recognizer.OnPhraseRecognized -= Recognizer_OnPhraseRecognized;
recognizer.Stop();
}
}
}

Prerequisites

  • Unity3D 2019.2+

Source Code

public string[] keywords = new string[] { "up", "down", "left", "right" };
public ConfidenceLevel confidence = ConfidenceLevel.Medium;
public float speed = 1;
public Text results;
public Image target;
protected PhraseRecognizer recognizer;
protected string word = "right";
private void Start()
{
if (keywords != null)
{
recognizer = new KeywordRecognizer(keywords, confidence);
recognizer.OnPhraseRecognized += Recognizer_OnPhraseRecognized;
recognizer.Start();
}
}
private void Recognizer_OnPhraseRecognized(PhraseRecognizedEventArgs args)
{
word = args.text;
results.text = "You said: <b>" + word + "</b>";
}
private void OnApplicationQuit()
{
if (recognizer != null && recognizer.IsRunning)
{
recognizer.OnPhraseRecognized -= Recognizer_OnPhraseRecognized;
recognizer.Stop();
}
}

Exporting the project

  • Windows
  • Android

Speech Recognition in Unity

I hope you had fun reading and/or following along. In the next story, we will look into how to build more features and interactions into this App. Stay tuned!

If you are interested in further exploring, here is my GitHub repo :

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store