Learning with Chatbots

About a year ago, we (MindCET eX, an R&D team exploring alternative solutions to educational problems) were approached by one of the leading hi-tech companies in the development of chatbots that was interested to find out whether chatbots could provide a significant pedagogical value to students. We were both skeptical – chatbots had been successfully used mostly by service providers – and at the same time very excited to take on the challenge and explore the educational potential of this growing phenomenon. We gathered a team of entrepreneurs who worked closely with teachers, students, and content providers, and set out on the road. After 6 months, Mr.H, Nacho, and Tony were born, and they have spearheaded the development of many other chatbots currently being piloted by schools in Israel (as, for example, Pablo – a chatbot developed by teachers).

We learned that chatbots can definitely provide new pedagogical solutions that are relevant to the new generations. Students related to these new educational solutions with ease, curiosity, and self-motivation. Observing their interactions with these bots, we found that students naturally developed a comfortable relationship, generating even emotional responses (e.g., swearing at and later apologizing to the bot!), which has led to the pedagogical engagement needed in any learning solution. Moreover, students understood the role and limitations of the bot, using it according to their specific needs and not becoming frustrated through wrong expectations.

Get your copy >>> Chatbot Magazine

The story of Mr.H

Mr.H is a personal assistant that helps students study for the national 11th grade History Matriculation Exam in Israel through enabling practice on questions based on previous exams.

Our Challenge: History was selected as the subject discipline in order to understand whether chatbots could deal with text-rich answers and provide pedagogical value, especially in Hebrew due to the still limited capabilities for NLP.

About Mr.H: During the conversation, Mr.H displays questions to the student, gets and analyzes the answers, and provides the user with feedback that includes whether the answer is correct, incorrect, or partly correct and further indicates what is missing for a correct answer. Mr.H. is offered in FB Messenger and WebChat. Mr.H. was published in social networks (targeting students directly) 4 weeks before the Matriculation Exam of 2017 and 638 students voluntarily used it consistently.

What we’ve learned from the users: Students interacted with Mr H. for 3 main uses: “The explorer” – users playfully trying to understand Mr.H’s capabilities; “I just want the answers” – users not interested in Mr.H’s feedback and just wanting to see the correct answers to the questions; “I want feedback” – users wanting to learn from Mr.H by getting feedback about their answers.

75% ranked the user experience at 4 or 5 (highest: 5) and 70% found Mr.H. “special and interesting.”

The educational benefit of the bot in comparison with other interfaces is its natural availability to the student and the student’s ability to connect emotionally with it. Many students talked to the bot like a friend, trying to draw encouragement from it before the examination, or simply trying to converse with it. This took place even though the bot does not attempt to misrepresent itself as a real person, and from the outset presents itself as a computerized chat. Chatbots create a unique opportunity that might be characterized as the “empathic robot paradox” – on the one hand, the learner is aware that he is conversing with an automaton, a machine, and so he can allow himself to fail or to be seen in a more forgiving light, because there is no human being on the other side; on the other hand, the dialogue framework creates an empathic infrastructure, and the learner finds himself also being rewarded on that axis. In the course of the observation sessions in the classroom, we saw that the students were sitting in small groups or alone, talking among themselves about the questions and answers that came up in the course of practicing with the bot, laughing at its answers, or reacting to it in an emotional way (for example, “How sweet!!”). It is clear that the users see the bot, at some level, as having a personality, a new entity and not merely as an application.

The story of PABLO

PABLO accompanies teachers during online courses, helping them to practice and learn new content. PABLO offers micro-learning content with integrated open and multiple-choice questions, videos, and a collection of relevant examples; it also connects the learner with the course facilitators.

Building PABLO: Enabling teachers to develop their own chatbots. Content developers and teachers, without any expertise in programming, built PABLO using ChatFuel, a platform that enables anyone to build a chatbot. The main development challenge was to allow the chatbot to answer a wide range of questions and expand its discourse and content. In ChatFuel, control is in the hands of the content developer and the chatbot progresses through examining and understanding how the learners are using it.

Benefits:  Teachers, as users, expressed their satisfaction in learning with PABLO, especially because it is varied, focused, and allows the repetition of a certain content as many times as one needs, eliminating any shame or awkwardness that learners might feel. Teachers, as developers, expressed their satisfaction in developing a tool based on their needs.

The story of Nacho

Nacho is an online Biology course assistant, that helps 10th grade students to practice what they’ve learnt by providing a personalized, friendly, space for practicing. Nacho was developed with a strong human friend-like personality in order to explore whether it could trigger significant student engagement. Nacho shares its own life experiences while providing Biology questions to the students, who have enthusiastically expressed their joy and interest in getting to know Nacho better. Nacho is currently being offered to thousands of students enrolled in online Biology courses.

The story of Tony

Tony helps 5th-7th graders understand the content of ancient Biblical text. Tony allows the students to read verses, and provides explanations (text and images), questions, and relevant feedback. Tony was developed with a strong user-friendly approach by using animated gifs, jokes, and emotional language to explore whether it could facilitate students’ understanding of texts using less familiar language. Students displayed a positive response while chatting with Tony and to the animated gifs. Tony is currently being piloted in flipped classroom Bible teaching in primary schools in Israel.

What’s next?

During 2017, our developments and pilots of chatbots for educational environments opened up new horizons about chatbots’ pedagogical potential. In the editorial, Dr. Cecilia Waismann raised questions about chatbots’ viability and efficiency when the goal is to help develop learning skills for educational purposes. Our first chatbot pedagogical approach tried to deal with efficiency in learning. However, for the time being, we can only affirm the potential of the conversation itself. Current developments are exploring longer pedagogical processes, aiming to reinforce the conversation as a powerful learning mechanism of the chatbot. We will keep you posted!

Get your copy >>> Chatbot Magazine