Jisc’s National Centre for AI in tertiary education (NCAI) has teamed up with TeacherMatic to run a yearlong pilot to evaluate the use of the generative AI tool to create teaching and learning content.
We have just wrapped up phase one, which started late May 2023, in collaboration with eight further education colleges: Basingstoke College of Technology, Milton Keynes College, New City College, Newcastle and Stafford Colleges Group, Northampton College, NPTC Group of Colleges, Southern Regional College and West College Scotland. The pilot aim is to assess the educational generative AI platform known as TeacherMatic.
What is TeacherMatic?
TeacherMatic is a user-friendly platform that uses the power of generative AI, specifically this GPT-3 model. The main goal is to help teachers generate all sorts of educational materials, from lesson plans and grading criteria to quizzes and learning activities. Plus, you can customise it too, so it fits the needs of each learner.
About the pilot
Participating institutions received 50 TeacherMatic licenses, enabling a total of 400 users to explore and use the generative AI tool to create teaching and learning content. For some participants this is the first time they have used generative AI to produce and generate content for teaching and learning. To ensure successful implementation all users attended a training session ran by TeacherMatic and the NCAI team. Following the first phase, focus group meetings were held in June and July 2023 to gather participants’ thoughts and feedback on TeacherMatic. Phase two is scheduled from summer 2023 to winter 2023/2024 and this phase will allow users to continue to produce content with the use of generative AI TeacherMatic and review how the tool can aid them in day-to-day tasks.
Where are we now?
We have completed phase one and reviewed initial thoughts about TeacherMatic during focus groups with each pilot institution. In the meetings, about five to ten TeacherMatic users joined us in an hour-long discussion and responded to a set of twenty-five questions. The questions aimed to evaluate current resource creation methods, review subject areas and usability of TeacherMatic, assess the quality of generated resources, evaluate available tools and features, and discussed how TeacherMatic could save time and hopefully reduces workloads.
Current situation and resource creation methods
Most of the participants said that they currently make their own teaching resources and assessment materials for each subject area. They also mentioned that there is a growing trend of sharing materials across departments and campuses, thanks to better IT systems and connectivity. Participants highlighted there is an increasing trend of sharing materials across departments and campuses, helped by improved IT systems and connectivity. We agreed that creating resources consumes a significant amount of time. Participants were excited and see TeacherMatic as a valuable tool, explaining that it could also be used for checking understanding as well as generating. They said that TeacherMatic can effectively reduce the time spent on creating resources, particularly for schemes of learning and summative assessment quizzes.
TeacherMatic usability
Feedback was overwhelmingly positive, with users finding the platform’s user experience and design intuitive and easy to navigate. The designers have considered usability and included a blue icon, which provides generator information, along with how-to guides and video demonstrations. A small number of participants felt confident using generative AI tools like Chat GPT and Google Bard, the majority had not used generative AI before and found TeacherMatic easy to use.
Evaluation of generated resource quality
Predominantly the feedback is that TeacherMatic effectively produces content at a suitable level, although users were aware of the need to review and verify the accuracy of generated content. Overall, users said that TeacherMatic can significantly save time compared to traditional content creation methods.
Bloom’s Taxonomy of learning has been added to generators and the level of complexity can be adjusted to suit the learners which users liked. The complexity sliders were particularly useful when using multiple-choice questions and discussion generators. The feedback was positive about the variety of the generators, such as chat with TeacherMatic, course planning generators and course activity generators. The most popular generator was the URL-based multiple-choice question generator, as it allows users to generate questions based on a webpage. Feedback about the lesson plan generator was again positive as it includes an option for the user to generate content and plan in employability, differentiation, or use of technology.
Evaluation of time saving and workload reduction
Participants were asked if they believed TeacherMatic and the use of generative AI could save time. The overwhelming response was yes, with users explaining that they were soon able to generate suitable content quickly and easily.
Feedback for course planning generators was positive. Participants liked how quickly it generated a scheme of work, which can be easily copied and pasted into an institution’s template or exported as a PDF or Microsoft Word file. It is also flexible, as the user can choose between a table or bullet points.
The FAQ, flashcards, course description, title generator, and topic notes generators received positive feedback for their ability to quickly create additional course content.
The majority thought that course activity generators were great time-saving tools and the multiple-choice generator created content quickly. They liked that the content could easily integrate into VLEs using the export function, including GIFT format, Kahoot and Quizizz.
The analogy maker generator was praised for breaking down concepts and adding variety to potentially dry topics. The step-by-step generator was found to be useful for providing instructions, particularly for practical lessons with multiple activities happening at once.
Moving forward
Feedback from users taking part in phase one of the TeacherMatic pilot was encouragingly positive and TeacherMatic met or exceeded their expectations. The platform was praised for its ease of use and the suitability of the generated content. The variety and effectiveness of the generators were positive along with how TeacherMatic opens the door for people who have not used generative AI before. This first phase shows us how promising TeacherMatic is, and we will continue with the longer phase two to see what participants can do given more time.
Find out more by visiting our Artificial Intelligence page to view publications and resources, join us for events and discover what AI has to offer through our range of interactive online demos.
For regular updates from the team sign up to our mailing list.
Get in touch with the team directly at AI@jisc.ac.uk