Collaborative learning with augmented reality and social robots

Augmented reality (AR) facilitates an interactive experience in a physical environment where the three-dimensional objects that exist in the physical world are extended with computer-generated perceptual information. Social robots are a special kind of robots that can interact and communicate with people by following social etiquette attached to a specific role. Both augmented reality and social robots facilitate the creation of new interactive pedagogic methods to help increase students’ knowledge through an immersive and personalized learning manner. In this proposal, we explore the use of these two emerging technologies to design collaborative learning schemes involving both teachers and students. Specifically, we target the subjects of earth science education and cybersecurity education to demonstrate how interactive and collaborative learning technologies would strengthen students’ knowledge and motivate them to pursue careers in STEM and ICT fields.

Our first effort calls for the design of a Mixed Reality Tornado Simulator which provides a collaborative earth science education curriculum in STEM classes to students of Grades 4 to 6. With the Mixed Reality Tornado Simulator, the abstract knowledge of tornado is transformed into tangible interactions that students and teachers could configure and manipulate. We plan to analyze the immersive experience, situational interest, and knowledge of the participants after human-subject experiment to evaluate the learning performance.

Our second effort designs interactive story-telling programs on social robots to teach foundational cybersecurity concepts, such as password, phishing, and social media privacy, to students of Grades 3 to 6. The goal of this effort aims to increase children’s knowledge of basic cybersecurity concepts by using social robots. Specifically, we embedded these cybersecurity concepts into three interactive stories programmed on the Zenbo social robot. We also invited four teachers into a focus group to iteratively improve the stories and explore the best ways to adopt the stories into classroom instruction. We plan to conduct a multistage evaluation of both teachers and students to examine students' learning performance of cybersecurity concepts, interest in cybersecurity, engagement level, as well as teacher’s opinions of bringing the social robot into their classroom.​

Our third effort proposes an interactive AR-based environment to increase middle school students' knowledge and interest in cybersecurity. We have designed interactive activities that make abstract cybersecurity concepts (such as firewall, steganography, and phishing) more tangible by using the emerging AR technology to teach knowledge of cybersecurity, capture situational interest, and develop students’ self-efficacy for cybersecurity and, more broadly, CS careers. To evaluate the learning performance and our AR-based design, we have programmed the Mixed Reality Phishing application for the initial study. The Mixed Reality Phishing application enables students to interact with virtual objects by manipulating a physical object to provide a tangible interaction form. We have collected and customized the phishing contents based on the popular social media platforms of different countries to provide a realistic training scenario. We plan to analyze the immersive experience, situational interest, and workload of the participants after the human-subject experiment to evaluate the learning performance and user experience of the participants.