Continuous Speech Recognition on Android with different Locale

Speech to text with android built-in SpeechRecognizer class.

Photo by Malte Helmhold on Unsplash

Let’s create a small application which takes voice as input and return the text in different Locale.

UI:

First create .xml file. With a Button, ProgressBar, Spinner and couple of TextViews.

<?xml version="1.0" encoding="utf-8"?>
<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical">

    <TableRow
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:gravity="center_vertical">

        <Button
            android:id="@+id/btnStartListen"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content"
            android:text="Start Listening" />

        <Space
            android:layout_width="0dp"
            android:layout_height="wrap_content"
            android:layout_weight="1" />

        <Spinner
            android:id="@+id/spinner1"
            android:layout_width="wrap_content"
            android:layout_height="wrap_content" />
    </TableRow>

    <ProgressBar
        android:id="@+id/progressBar1"
        style="?android:attr/progressBarStyleHorizontal"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:layout_margin="20dp"
        android:indeterminate="true"
        android:visibility="gone"
        tools:visibility="visible" />

    <TextView
        android:id="@+id/textView1"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="50dp"
        tools:text="Output will be displayed here" />

    <TextView
        android:id="@+id/tvError"
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:layout_marginTop="50dp"
        tools:text="Error will be displayed here" />

</LinearLayout>

Code:

Now we will continue to Kotlin part with taking two variables.

private var speechRecognizer: SpeechRecognizer? = null
private var recognizerIntent: Intent? = null

Set click listener to start listening.

private fun setListeners() {
    binding.btnStartListen.setOnClickListener {
        startListening()
    }
}

Use below function to start listening.

private fun startListening() {
    speechRecognizer!!.startListening(recognizerIntent)
    binding.progressBar1.visibility = View.VISIBLE
}

Add android.permission.RECORD_AUDIO in AndroidMenifest.xml and get runtime permission.

private fun checkPermissions() {
    val permissionCheck =
        ContextCompat.checkSelfPermission(applicationContext, Manifest.permission.RECORD_AUDIO)
    if (permissionCheck != PackageManager.PERMISSION_GRANTED) {
        ActivityCompat.requestPermissions(
            this@MainActivity,
            arrayOf(Manifest.permission.RECORD_AUDIO),
            PERMISSIONS_REQUEST_RECORD_AUDIO
        )
        return
    }
}

override fun onRequestPermissionsResult(
        requestCode: Int,
        permissions: Array<String>, grantResults: IntArray
    ) {
        super.onRequestPermissionsResult(requestCode, permissions, grantResults)
        if (requestCode == PERMISSIONS_REQUEST_RECORD_AUDIO) {
            if (grantResults.isNotEmpty() && grantResults[0] == PackageManager.PERMISSION_GRANTED) {
                startListening()
            } else {
                Toast.makeText(mContext, "Permission Denied!", Toast.LENGTH_SHORT).show()
                finish()
            }
        }
    }

• Check SpeechRecognizer is supported or not
• Prepare speechRecognizer.
• set RecognitionListener which overrides multiple methods.

private fun resetSpeechRecognizer() {
    if (speechRecognizer != null) speechRecognizer!!.destroy()
    speechRecognizer = SpeechRecognizer.createSpeechRecognizer(mContext)
    errorLog(
        "Availability:"+SpeechRecognizer.isRecognitionAvailable(mContext)
    )
    if (SpeechRecognizer.isRecognitionAvailable(mContext))
        speechRecognizer!!.setRecognitionListener(mRecognitionListener)
    else finish()
}

private val mRecognitionListener = object : RecognitionListener {
        override fun onBeginningOfSpeech() {
            errorLog("onBeginningOfSpeech")
            binding.progressBar1.isIndeterminate = false
            binding.progressBar1.max = 10
        }

        override fun onBufferReceived(buffer: ByteArray) {
            errorLog("onBufferReceived: $buffer")
        }

        override fun onEndOfSpeech() {
            errorLog("onEndOfSpeech")
            binding.progressBar1.isIndeterminate = true
            speechRecognizer!!.stopListening()
        }

        override fun onResults(results: Bundle) {
            errorLog("onResults")
            val matches: ArrayList<String>? = results
                .getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION)
            var text = ""
            for (result in matches!!) text += """
     $result
     
     """.trimIndent()
            binding.textView1.text = text
            if (IS_CONTINUES_LISTEN) {
                startListening()
            } else {
                binding.progressBar1.visibility = View.GONE
            }
        }

        override fun onError(errorCode: Int) {
            val errorMessage = getErrorText(errorCode)
            errorLog("FAILED $errorMessage")
            binding.tvError.text = errorMessage

            // rest voice recogniser
            resetSpeechRecognizer()
            startListening()
        }

        override fun onEvent(arg0: Int, arg1: Bundle) {
            errorLog("onEvent")
        }

        override fun onPartialResults(arg0: Bundle) {
            errorLog("onPartialResults")
        }

        override fun onReadyForSpeech(arg0: Bundle) {
            errorLog("onReadyForSpeech")
        }

        override fun onRmsChanged(rmsdB: Float) {
            binding.progressBar1.progress = rmsdB.toInt()
        }
    }

Prepare recognition Intent.

private fun setRecogniserIntent() {
    recognizerIntent = Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH)
    recognizerIntent!!.putExtra(
        RecognizerIntent.EXTRA_LANGUAGE_PREFERENCE,
        selectedLanguage
    )
    recognizerIntent!!.putExtra(
        RecognizerIntent.EXTRA_LANGUAGE,
        selectedLanguage
    )
    recognizerIntent!!.putExtra(
        RecognizerIntent.EXTRA_LANGUAGE_MODEL,
        RecognizerIntent.LANGUAGE_MODEL_FREE_FORM
    )
    recognizerIntent!!.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS, RESULTS_LIMIT)
}

Done 🎉. You can now test the application.

Note: For Continues listening, You can manage IS_CONTINUES_LISTEN const. We set startListening() just after onResult() and onResume().

Optional:

We can add now additional things like Locale. Prepare a selection of supported Locale and select.

private fun prepareLocales() {
        val availableLocales =
            Locale.getAvailableLocales() //Alternatively you can check https://cloud.google.com/speech-to-text/docs/speech-to-text-supported-languages

        val adapterLocalization: ArrayAdapter<Any?> = ArrayAdapter<Any?>(
            mContext,
            android.R.layout.simple_spinner_item,
            availableLocales
        )
        adapterLocalization.setDropDownViewResource(android.R.layout.simple_spinner_dropdown_item)
        binding.spinner1.onItemSelectedListener = object : AdapterView.OnItemSelectedListener {
            override fun onItemSelected(
                parent: AdapterView<*>?,
                view: View?,
                position: Int,
                id: Long
            ) {
                selectedLanguage = availableLocales[position].toString()

                resetSpeechRecognizer()
                setRecogniserIntent()
            }

            override fun onNothingSelected(parent: AdapterView<*>?) {
//                TODO("Not yet implemented")
            }
        }

        binding.spinner1.adapter = adapterLocalization

        // Set "en" as selected language by default
        for (i in availableLocales.indices) {
            val locale = availableLocales[i]
            if (locale.toString().equals("en", true)) {
                binding.spinner1.setSelection(i)
                break
            }
        }
    }

I have tested it works fine with en, hi and gu Locale.

Download:

Find out the sample code from below 👇 GitHub link. Feel free to explore, download, generate issues or contribute Code:

GitHub - RumitPatel/android-continues-speech-recognition: This project is a demonstration to…

Output:

Happy coding. :-)