“To be successful, the first thing to do is to fall in love with your work — Sister Mary Lauretta”
Well, the Google Summer of Code 2016 is reaching its final week as I get ready to submit my work. It has been one of those best three-four months of serious effort and commitment. To be frank, this has to be one of those to which I was fully motivated and have put my 100%.
Well, at first, the results of training wasn’t that promising and I was actually let down. But then, me and my mentor had a series of discussions on submitting, during which she suggested me to retrain the model excluding the data set or audio files of those speakers which produced the most errors. So after completing the batch test, I noticed that four of the data set was having the worst accuracy, which was shockingly below 20%. This was causing the overall accuracy to dip from a normal one.
So, I decided to delete those four data set and retrain the model. It was not that of a big deal, so I thought its not gonna be drastic change from the current model. But the result put me into a state of shock for about 2–3 seconds. It said
TOTAL Words: 12708 Correct: 12375 Errors: 520
TOTAL Percent correct = 97.38% Error = 4.09% Accuracy = 95.91%
TOTAL Insertions: 187 Deletions: 36 Substitutions: 297
SENTENCE ERROR: 9.1% (365/3993) WORD ERROR RATE: 4.1% (519/12708)
Now, this looks juicy and near to perfect. But the thing is, the sentences are tested as they where trained. So, if we change the structure of sentence that we ultimately give to recognize, it will still be having issues putting out the correct hypothesis. Nevertheless, it was far more better than it was when I was using the previous model.
So I guess I will settle with this for now as the aim of the GSoC project was to start the project and show proof of that this can be done, but will keep training better ones in the near future.
Google Summer of Code 2016 — Submission
- Since the whole project was carried under my personal Github repository, I will link the commits in it here : Commits
- Project Repository : ml-am-lm-cmusphinx
- On top of that, we (me and the organization) had a series of discussions regarding the project over here: Discourse IndicProject
Well, I have been documenting my way through the project over here at Medium starting from the month of May. The blogs can be read from here.
What can be done in near future?
Well, this model is still in its early stage and is still not the one that can be used error free, let alone be applied on applications.
The data set is still buggy and have to improved with better cleaner audio data and a more tuned Language Model.
Speech Recognition development is rather slow and is obviously community based. All these are possible with collaborated work towards achieving a user acceptable level of practical accuracy rather than quoting a statistical, theoretical accuracy.
All necessary steps and procedure have been documented in the README sections of the repository.
puts "thank you everyone!"