Students’ Reflections on the Engramo English Pilot Project at CTU in Prague
Basic information
In the winter term of 2020, Engramo got a unique chance to run a pilot project at the Language Department of the Faculty of Electrical Engineering, CTU in Prague. With the covid-19 pandemic forcing all classes to be online, the English teacher, Petra Jennings M.A., welcomed this opportunity as it complemented well the combination of typical online classes she provided herself and the self-study of Raymond Murphy’s textbooks which was the only extra resource her students were using.
Collaboration Details
Petra Jennings shared the syllabus with Engramo so that we could prepare tailor-made online sets of exercises that accompanied all of the 12 topics she dealt with during the term. Students were introduced to Engramo, they registered under their faculty email and were required to complete at least 40 sets of exercises during the term in order to meet the criteria for passing this subject.
Petra Jennings taught 31 students split into 2 groups. Both her and us regularly asked students for sharing their feedback. They had the opportunity to evaluate Engramo orally during her online lessons or to send her an email. We asked them about their first impressions straight after the introductory test, once again in the middle of the term and finally when they handed in their final test. We made sure our questionnaires were brief so as not to waste their time and anonymous to generate relevant answers. Regrettably, less than half of them were willing to spend their time answering our questions (all in all, 11 people filled out the forms), which makes the conclusions we arrived at a little weak. Nonetheless, we are sure that even a small amount of shared reviews matter a lot and we will definitely attempt to resolve all the complaints the students raised.
First impressions
In the beginning, students shared their first impressions of the app. We decided to let them express themselves freely, answering an open question that followed the intro test they took after getting familiar with the app.
Most of the students appreciated its design and the clear structure, some of them noticed the unusual variety of exercise types in the app and they also shared their opinions regarding the difficulty and entertainment each of the exercise types provided.
Mid-term feedback
Another, more detailed, questionnaire was sent to students at the end of November. They mostly appreciated our brief and clear explanations of new grammar rules (6x), and they said it was easy and not too time-consuming to use the app (5x). On the other hand, 5 of them were rather disappointed by the information provided on the dashboard (it was not clear how many sets of exercises they had done), some described the statistics in there as misleading (3x) and they said they would like to see explanations of their incorrect answers (5x).
All in all, they agreed that Engramo really helped to improve their level of English but they added that this only involved certain skills.
Post-term feedback
Once again, many students highlighted that Engramo was easy to use, they appreciated the clear grammar explanations and examples provided in our Grammar Book (8x) and they also noticed the high language level of Engramo (compared to the other apps on the market), which was especially important for those who had already achieved language certificates and wanted to keep or increase their English level.
On the negative side, 4 students mentioned again that they would appreciate it if their mistakes were explained, too. They would also welcome clearer ways of correcting mistakes (4x) and one of the students complained about the lack of an option to choose what aspects of grammar to study on his/her own.
In the end, we asked if they thought Engramo improved their grammar knowledge. In reaction to this question, 8 students agreed (eg. one of them literally said “It truly helped me to improve my English.”), 2 were not sure and 1 disagreed.
Internal feedback provided to the school
During the evaluation of Ms. Jennings’ subject at the very end of the term, a few students got somewhat more critical towards Engramo. It is possible that these students were those who failed the final test conducted by us, but it is not clear. Perhaps they were just waiting for the final mark to share their honest feedback (though all our forms were completely anonymous).
At this occasion, several students stated that the application was not efficient enough and that some exercises repeated over the course of the term, which, they felt, made them rely more on the routine use of it than on their grammar knowledge in order to succeed within the app.
Going forward
Although it is hard to draw conclusions from such brief, restricted and anonymous feedback, we took it very seriously and have decided to incorporate some changes into the app so that future students do not have similar objections.
Firstly, we have improved the way we correct mistakes in Engramo. We prepared a video and a mock mission to explain it all. We have changed the visuals and we also now provide clearer explanations of our corrections within every exercise.
Secondly, we checked the algorithm and found a mistake that was restricting the variety of specific exercises displayed to students, which means that there will be no more repeating exercises in the app.
Moreover, we are currently working on a so called “manual mode” of the app that would allow both teachers and students to choose the grammar topic they wish to practise. We are also working on a calculator on the dashboard that will provide information about the number of accomplished sets of exercises, the most frequent mistakes, etc.
As for the other key complaint, we regret that we cannot provide explanations of all mistakes within the app. Since we employ a great variety of exercises (including “find and correct the mistake” or “paraphrase”) it is not possible to predict all incorrect answers and explain them. Nevertheless, we believe that the interesting and unique mixture of exercise types makes up for that inability.
We are now looking for more opportunities to cooperate with universities and language centres and (partly thanks to this project and the feedback it generated) we strongly believe that we will not disappoint our future students and collaborators!
