Replacing expensive teacher time with an automated testing system to check the knowledge level of students who want to enroll in the English course.
A company that offers distance learning of English approached us to develop a chatbot with the main function to test the language level of learners enrolled in the course. Usually this task was done by teachers, and in one month the company had to spend a considerable amount of money on it. Since not everyone who goes through this process decides to become a student, the idea of replacing an expensive teacher with an automatic testing system makes perfect sense.
Chatbot as an element of the website and mobile application, providing options to test grammar knowledge, listening comprehension, speaking, pronunciation and vocabulary.
In examining the level of language proficiency, the company is required to assess the following skills:
- grammar knowledge,
- listening comprehension,
- speaking and pronunciation,
We had to choose a platform to implement the user interface of the bot. Despite the popularity of various messenger platforms (Facebook Messenger, Telegram, Viber, etc.), we did not consider them suitable for our task, as they have limited functionality for implementing the user interface (UI) of chatbots. Therefore, we decided to develop a chatbot as an element of the website and mobile application of the educational company using web and mobile programming .
To test grammar skills, we have implemented a multiple-choice quiz that examines the basic aspects of grammar. There is a list of the main grammar sections that are tested; for each section the student answers as many questions as specified in the test parameters. All test parameters are configurable - the system administrator sets the number of tasks to be completed for each skill.
To examine the level of listening comprehension a student has to listen to the audio recording and to write the corresponding text.
To test speaking and pronunciation skills the following approach is used: the student receives a series of questions in text form and answers aloud. The time for the response is limited and is set as one of the test parameters.
The last skill to be tested is vocabulary. Here, as in the grammar review, the student is offered questions to identify the meaning of the word with answer options. The definitions are also given in English. The questions come from different thematic areas of vocabulary. To motivate a student to take the test, encouraging messages are sometimes displayed along with the tasks, such as: "You are almost there!" or "You are doing great!". The company's promotional messages are also displayed: various discounts for training, etc.
The results of the test are reported to the student and, if necessary, all tasks, all answers and mistakes can be viewed. This makes the process friendlier - the companies that simply report the results without analysis discourage the desire to retake the test if the result was weak.
In addition to the general test, thematic tests are also conducted according to the same mechanism. For example, there is a separate test on phrasal verbs or on a particular aspect of grammar or vocabulary. For those who pass a certain amount of tests, the company offers a discount or a number of free lessons.
Test statistics are constantly accumulating, the data on the conversion from testers to students is collected.
Besides the student's application, the solution also includes an application for the teacher, which allows to enter new tests and delete old ones, download and delete audio recordings, and view statistics.
Kinesis for playing audio recordings, Amazon S3 - to store audio recordings. Implementation of Amazon Transcribe service on back-end.
As well as with many of our other projects, we used Amazon services.
The data for the test are stored in a PostgreSQL database and entered by teachers through a web application.There are more questions in the database predefined for each learning section than are required for one test. The questions are randomly selected for the particular test, so that the tests are not repeated. For each task, the student`s app makes a request to the backend via the website REST API.
For the listening comprehension tests, we store dozens of audio recordings of 15-20 seconds duration in the database, Amazon Kinesis is used for playing audio recordings. Amazon S3 is used to store audio recordings.
While testing speaking and pronunciation skills the voice answers are recorded to a file and transmitted to the back-end. The Amazon Transcribe service is used on the back-end and transfers speech to text. The text is compared to the sample stored in the database, if the mistakes amount exceeds the predefined value, the response is considered incorrect. This value is determined in the test bot settings.
Like the grammar check, the vocabulary test is implemented using the REST API.
We used React Native to develop a mobile application for students. The web application for teachers was developed using React.