Michael, thank you for these notes. It has been a while since I taught, but after leaving academia I came back to teach software engineering from industry for a decade and did it through roleplays, client consultations, scenario exploration, technical critiques and project critiques as much as possible. It didn't just teach critical thought and technical analysis but conflict management, negotiation, team work, adaptation and consensus-building. It drew out students who had previously hidden behind books and screens.
I was delighted to see these represented, and the student response to this was very warm because they could see immediate application -- course feedback would see comments like "the most useful thing I studied in my degree." From an employer's perspective, these were all critical capabilities that tertiary graduates often lack.
I'm hoping you're right about max class sizes but I doubt the higher education sector in Australia will revert back to small classes. You should see our workload calculations! It's spreadsheet formulas!
Respectfully, I disagree that the educator/ student relationship is always going to be positive. It's structurally unequal so assuming positivity as an outcome feels to me like an act of power by those that hold it.
Anyway, I was introduced to programmatic assessment very recently and I feel like that's a game changer. In the degrees we offer, our most secure, authentic and professionally valid assessment is a practicuum placement. Its pass/ fail.
Program based assessment means we're going to look at assessment design over 8, 10 or 12 subjects rather than focus on each individual subject. It also reframes the conversation from 'beating AI' to how best to support students to become qualified professionals. Many of the ideas you talk about for assessment will be included but at key program points, not in each subject. I'll keep reading to learn more!
Thank you for sharing these ideas. But i have three niggling thoughts whenever I read about oral defence and assessing processes. The first is that it assumes the relationship between assessor and assessed is neutral and objective. This is never true with plenty of research to back it up. Secondly and related, equity and universal design prompts us to think about how to support various learning needs and assessment supports. I feel like the ideas don't always sit well in that space. And finally, and this is already mentioned, but I teach online to 60 - 120 students/class (about 8 classes a year) who are trying to get a professional qualification as fast as possible and usually prioritise their effort towards their 2-3 assessments per study period. I have 60 minutes in total allocated to mark/ grade per student each class. I often don't even do the marking as its sent out to casuals. I'm about to bring back tests for post graduate students as its relatively more rigorous. I do already set complex problem solving, case study based assessments but am seeing a rise in made up references or no references! Its a frustrating scenario but I need to work in my constraints or give up education. Thanks for getting this far if you did! Any suggestions?
Great points. Thanks! Regarding the first point, none of these assessment methods is new. Many of them have been used extensively for decades with great success. Yes, it changes the relationship between instructor and student, but I would argue it does that in a positive way. If you have an art or design school where you are, I recommend giving them a visit. Assessment in art and design is almost exclusively critique based and is quite successful at that. The other two points are very valid concerns that require careful planning. Access can be accomplished by providing options. And even the scalability issue can be addressed. I think you can probably scale some techniques to class sizes of about 100 students (I have an article coming up). But I would agree that this is a limit. I am fairly convinced that future educational institutions will no longer be able to offer classes of 150 students or more (I have an article about that coming up as well).
A great piece that really made me think. I teach online at a large college in Southern Ontario. My students are on a career path, with just enough academic work offered to ensure they have the core skills they need. I wonder: How can these ideas be applied to a more practically focused curriculum?
These methods actually work better for curricula that are practically focused. You can always situate everything within the context of the students’ hand on experience. I have a few articles in the pipeline where I go into specifics about all these methods.
These are very good pedagogie. Clever, creative, and engaging. But, as you note, a small class size is a prereq (perhaps not as much for the simulations), and probably not just smaller, but very small...maybe less than 20? I'm afraid the entire business model for higher education rests on larger class sessions and this may be the ultimate barrier. So much rests on class size: staffing and workloads, credits and time to completion, aid, general education, curriculum, unions, etc. Whenever I broach class sizes as the main barrier to effective education in the age of AI, I get shrugs and "well that's not going to happen" hand waves. And it's true...what would that look like? Higher ed has never once engaged in such whole scale revision. And what of asyncronistic online education? If effective AI education requires a mentor-mentee, real-time dynamic (which I fully agree it does), that means the death of the other cash cow for colleges. The problem is so huge. My money is on simply muddling through for quite awhile. No stakeholder has an incentive to press for change.
In my opinion, that ‘industrial’ business model is no longer sustainable. I’m in the US, so tuition is a driving factor. As a parent, I would no longer invest in a College that runs large classes. It is like throwing money out the window.
I think class sizes up to 25 are no problem. With a bit of creative thinking you can probably work it with class sizes around 50. Anything above 100 is, quite frankly, just a waste of everybody’s time and money.
I have the advantage of being in a design college. We rarely have classes with more than 25 students. I realize this is a luxury, but it works well.
Saved this post! I’ve gotten interested in student debate, which has been advocated by some prominent writers on Substack, but I’ve been finding it really challenging to educate myself about how to run them effectively. I really appreciate your going into some of the nuts and bolts of these strategies. And this sounds like a really neat conference.
At first I was quite excited to read this article. As the instructional designer at a small community college I've been helping faculty navigate the challenges of AI, particularly in assessment. I've been beating the drum since Day One that we must, must, must rethink our assessment strategies. (Since before that, since cheating was an issue long before November 2022!)
Unfortunately, most of my faculty will look at this list and say, that's wonderful for a graduate seminar or even a capstone, but I teach introductory classes. Survey courses. Gateway and remedial classes where the bulk of the content is in fact basic factual and procedural knowledge, not higher-level application and evaluation. I'm not sure I see how these techniques will work for Intro to Biology or World Civ or Anatomy & Physiology I.
We have to operate at scale, though fortunately not in auditoriums filled with hundreds. But still, two dozen students taking two minutes each to explain a concept or solve a problem - there's a hour when you allow for standing up and sitting down. And will the last ones to go have an unfair advantage over the first?
Good ideas, but I'm struggling to apply them in my context.
It is possible, but it is true that the implementation of authentic assessment methods is most challenging in the environment you are mentioning. It does require rethinking some fundamental concepts about how higher education is working now. Large class sizes are a particular problem. I am planning to address this in more detail in one of my upcoming posts. I did mention potential solutions a few times but I have not yet written something dedicated to that specific question.
I feel that assessments for writing classes will become a lot more discussion oriented. You are no longer primarily evaluating the text itself but rather the thinking that went into writing the text. Quite a few of the authentic assessment methods mentioned here fit that bill.
It might depend what kind Agnese, but for fiction there are possibilities in both developmental editing and substantive editing. Developmental editing can be partially supported by AI, while substantive editing requires a lot more human judgement and ear -- so both could be useful in different ways (and one also supports the other.)
For developmental editing support I have an article with some AI prompts linked below.
Michael, thank you for these notes. It has been a while since I taught, but after leaving academia I came back to teach software engineering from industry for a decade and did it through roleplays, client consultations, scenario exploration, technical critiques and project critiques as much as possible. It didn't just teach critical thought and technical analysis but conflict management, negotiation, team work, adaptation and consensus-building. It drew out students who had previously hidden behind books and screens.
I was delighted to see these represented, and the student response to this was very warm because they could see immediate application -- course feedback would see comments like "the most useful thing I studied in my degree." From an employer's perspective, these were all critical capabilities that tertiary graduates often lack.
Thanks for sharing. Most people don’t realize that many of these methods are universal and can be used in almost any discipline.
I'm hoping you're right about max class sizes but I doubt the higher education sector in Australia will revert back to small classes. You should see our workload calculations! It's spreadsheet formulas!
Respectfully, I disagree that the educator/ student relationship is always going to be positive. It's structurally unequal so assuming positivity as an outcome feels to me like an act of power by those that hold it.
Anyway, I was introduced to programmatic assessment very recently and I feel like that's a game changer. In the degrees we offer, our most secure, authentic and professionally valid assessment is a practicuum placement. Its pass/ fail.
Program based assessment means we're going to look at assessment design over 8, 10 or 12 subjects rather than focus on each individual subject. It also reframes the conversation from 'beating AI' to how best to support students to become qualified professionals. Many of the ideas you talk about for assessment will be included but at key program points, not in each subject. I'll keep reading to learn more!
Thank you for sharing these ideas. But i have three niggling thoughts whenever I read about oral defence and assessing processes. The first is that it assumes the relationship between assessor and assessed is neutral and objective. This is never true with plenty of research to back it up. Secondly and related, equity and universal design prompts us to think about how to support various learning needs and assessment supports. I feel like the ideas don't always sit well in that space. And finally, and this is already mentioned, but I teach online to 60 - 120 students/class (about 8 classes a year) who are trying to get a professional qualification as fast as possible and usually prioritise their effort towards their 2-3 assessments per study period. I have 60 minutes in total allocated to mark/ grade per student each class. I often don't even do the marking as its sent out to casuals. I'm about to bring back tests for post graduate students as its relatively more rigorous. I do already set complex problem solving, case study based assessments but am seeing a rise in made up references or no references! Its a frustrating scenario but I need to work in my constraints or give up education. Thanks for getting this far if you did! Any suggestions?
Great points. Thanks! Regarding the first point, none of these assessment methods is new. Many of them have been used extensively for decades with great success. Yes, it changes the relationship between instructor and student, but I would argue it does that in a positive way. If you have an art or design school where you are, I recommend giving them a visit. Assessment in art and design is almost exclusively critique based and is quite successful at that. The other two points are very valid concerns that require careful planning. Access can be accomplished by providing options. And even the scalability issue can be addressed. I think you can probably scale some techniques to class sizes of about 100 students (I have an article coming up). But I would agree that this is a limit. I am fairly convinced that future educational institutions will no longer be able to offer classes of 150 students or more (I have an article about that coming up as well).
A great piece that really made me think. I teach online at a large college in Southern Ontario. My students are on a career path, with just enough academic work offered to ensure they have the core skills they need. I wonder: How can these ideas be applied to a more practically focused curriculum?
These methods actually work better for curricula that are practically focused. You can always situate everything within the context of the students’ hand on experience. I have a few articles in the pipeline where I go into specifics about all these methods.
I look forward to reading them. Thanks for the quick reply.
These are very good pedagogie. Clever, creative, and engaging. But, as you note, a small class size is a prereq (perhaps not as much for the simulations), and probably not just smaller, but very small...maybe less than 20? I'm afraid the entire business model for higher education rests on larger class sessions and this may be the ultimate barrier. So much rests on class size: staffing and workloads, credits and time to completion, aid, general education, curriculum, unions, etc. Whenever I broach class sizes as the main barrier to effective education in the age of AI, I get shrugs and "well that's not going to happen" hand waves. And it's true...what would that look like? Higher ed has never once engaged in such whole scale revision. And what of asyncronistic online education? If effective AI education requires a mentor-mentee, real-time dynamic (which I fully agree it does), that means the death of the other cash cow for colleges. The problem is so huge. My money is on simply muddling through for quite awhile. No stakeholder has an incentive to press for change.
In my opinion, that ‘industrial’ business model is no longer sustainable. I’m in the US, so tuition is a driving factor. As a parent, I would no longer invest in a College that runs large classes. It is like throwing money out the window.
I think class sizes up to 25 are no problem. With a bit of creative thinking you can probably work it with class sizes around 50. Anything above 100 is, quite frankly, just a waste of everybody’s time and money.
I have the advantage of being in a design college. We rarely have classes with more than 25 students. I realize this is a luxury, but it works well.
Saved this post! I’ve gotten interested in student debate, which has been advocated by some prominent writers on Substack, but I’ve been finding it really challenging to educate myself about how to run them effectively. I really appreciate your going into some of the nuts and bolts of these strategies. And this sounds like a really neat conference.
Thanks! And yes, the Seville conference is my favorite. It's an incredibly beautiful venue and everybody is very nice and open to new ideas.
At first I was quite excited to read this article. As the instructional designer at a small community college I've been helping faculty navigate the challenges of AI, particularly in assessment. I've been beating the drum since Day One that we must, must, must rethink our assessment strategies. (Since before that, since cheating was an issue long before November 2022!)
Unfortunately, most of my faculty will look at this list and say, that's wonderful for a graduate seminar or even a capstone, but I teach introductory classes. Survey courses. Gateway and remedial classes where the bulk of the content is in fact basic factual and procedural knowledge, not higher-level application and evaluation. I'm not sure I see how these techniques will work for Intro to Biology or World Civ or Anatomy & Physiology I.
We have to operate at scale, though fortunately not in auditoriums filled with hundreds. But still, two dozen students taking two minutes each to explain a concept or solve a problem - there's a hour when you allow for standing up and sitting down. And will the last ones to go have an unfair advantage over the first?
Good ideas, but I'm struggling to apply them in my context.
It is possible, but it is true that the implementation of authentic assessment methods is most challenging in the environment you are mentioning. It does require rethinking some fundamental concepts about how higher education is working now. Large class sizes are a particular problem. I am planning to address this in more detail in one of my upcoming posts. I did mention potential solutions a few times but I have not yet written something dedicated to that specific question.
And what about those of us who teach writing?
I feel that assessments for writing classes will become a lot more discussion oriented. You are no longer primarily evaluating the text itself but rather the thinking that went into writing the text. Quite a few of the authentic assessment methods mentioned here fit that bill.
It might depend what kind Agnese, but for fiction there are possibilities in both developmental editing and substantive editing. Developmental editing can be partially supported by AI, while substantive editing requires a lot more human judgement and ear -- so both could be useful in different ways (and one also supports the other.)
For developmental editing support I have an article with some AI prompts linked below.
https://reciprocalinquiry.substack.com/p/from-solid-ground-ai-support-for