Content Alignment: Using Lesson Inspector on GenAI Lesson Plans: Aila & Teachmate (and a dragon) 

Even before GenAI tools burst into our pedagogically-informed lives, we pondered on how to judge the 'goodness' of a lesson plan. There is no definition for a 'good' lesson plan, as essentially this concept could be broadly thought of as dependent upon:

• Context (the needs of your specific learners)

• Teacher identity (your epistemological beliefs about knowledge etc.)

• School ethos (what values does your school promote?)

• Policy (what frameworks do you have to work within?)

When asked to define a 'good' lesson plan, most teachers struggle to condense their thoughts into a formula or set of beliefs. This is something that educationalists intuitively understand but may find challenging to operationalise. Yet in my experience, it is something student teachers and ECTs commonly yearn for.

Introducing Lesson Inspector for Content Alignment

Enter the Lesson Inspector; a tool we've developed to evaluate lesson plans based on the themes present in the Core Content Framework (CCF), Initial Teacher Training Core Content Framework (ITTECF), and the Teachers' Standards. Our Lesson Inspector offers two metrics: a quantitative score and qualitative evaluation/analysis by criteria. This qualitative analysis provided by Lesson Inspector offers potential for content alignment of the LLM with the aforementioned Statutory frameworks.

In this discussion, we use two lesson plan generators (Aila and Teachmate) to produce lesson plans on the same topic of KS4 waves, with the same objective (below). We then upload these to Lesson Inspector and create Inspection reports for them both. We will discuss:

• User Process: The steps and methodology each tool uses and how the user inputs their lesson information.

• Report content : Such as prior knowledge and other relevant themes included in the Lesson Inspector report.

For a fair comparison, both tools were given the same objective from the AQA (2015) GCSE Physics Specification: "Students should be able to describe wave motion in terms of their amplitude, wavelength, frequency, and period."

Aila vs. Teachmate User Process

Aila is free. Aila is Government funded. These are immediately reasons to consider using Aila. One positive about the process of using Aila is that it is designed to involve the teacher at every step of the process. This means that after each input and generation cycle, the user has the opportunity to refine the generated output. This can be quite versatile in terms of tailoring to specific needs, but does come at the expense of requiring more active user input into the process, meaning that it takes a bit longer to generate a plan. We will explore this user-centric functionality is in another post. 

When considering the time scales though, do keep in mind that although quicker, teachmate is NOT free. It cost me £6.99 at the time of publishing this. However most non-government backed generators also charge, so this is not uncommon.

The interfaces are completely different: Teachmate has minimal inputs (see screenshot) and Aila seeks verification along each step of the generation process. See the Aila video below, to see how this process looked for our lesson plan generation on waves. This video is sped-up by a factor of 6, so keep this in mind! (Although this kind of duration is not uncommon in GenAI tools). In this video all that was entered into the interfaces was the topic, subject, phase and objective - the same as for Teachmate.

Output: The Waves Lesson Plans 

You'll probably want to see what was produced. See below for the .pdfs. It's easy to see how comparing these two can start to sketch out a skeleton for the diversity of GenAI-produced lesson plans

Lesson Inspector Analysis (Reports)

Both the lesson plans were put through our Lesson Inspector. See below for the .pdf versions of the reports (we are working on making these look a lot prettier!).There are several pages to each.

Summary - 'Correct Calibration'

Once you've recovered from the hilarious analysis of that final lesson plan, it's clear that a perfect zero can indeed be scored. This brings us back to the initial idea of content alignment involving correct calibration of curriculum standards.

If we assume a static curriculum standard based on the CCF/ITTECF and the Teachers' Standards, we need to consider the ability of the Lesson Inspector to guide users towards the correct calibration of these GenAI tools when creating lesson plans. Our scoring metric and evaluation/analysis framework provide a teacher with a quick and easy sense check against the frameworks they work within. These two metrics have very different impacts - the quantitative score could be used as an immediate quality indicator, although we have designed it to essentially be a 'hook' to ensure the report reader reflects upon the qualitative analysis - a means to promote reflective practice upon curriculum goals.

If we return to our initial understanding of the values that contribute to a 'good' lesson plan, we can argue that content alignment is at least partially achieved if the Lesson Inspector reports facilitate and promote reflective practice. These reports provide a space for teachers to critically consider the output and reflect on how it aligns with their values, beliefs, and ideas about the curriculum.

Consider an example: a teacher who adheres to a very teacher-led approach and does not favour student-led practice. The Lesson Inspector report suggests student-led practice as an area for improvement and notes the reliance on teacher-led methods. Does this mean the content is not aligned? The teacher reflects on these comments from the report and critically thinks about their ideal use of teacher-led versus student-led practice. They ultimately decide they are satisfied with their initial plan. Does this mean the report is not useful? Does it mean the content is misaligned?

The Lesson Inspector cannot answer these questions of pedagogical choice and teacher identity. However, it can provide a set of structured prompts to help users explore and solidify their concepts of a 'correctly calibrated' learning experience.