If AI Can Complete the Assignment, How Do We Know What Students Actually Learned?
In this Article
As we ponder assessment in an era where AI can complete almost any assignment, the purpose of assessment takes centre stage. If assessment provides a process for us to measure how much learning has taken place, merely creating tasks that AI can complete blunts our ability to measure the actual learning. After all, if AI has done most of the work, a well-written essay or a perfectly completed assignment tells us very little about what a learner actually knows, understands or can do independently.
Rather than developing better AI detector tools to catch students in the act, how about rethinking assessment and how to measure learning in better ways? The past few decades taught us a lot about assessment design, as last week’s blog showed. We now move forward from that point, and in this blog we ask how we can measure learning.
The research confirms what teachers and trainers already know: good performance during instruction cannot automatically be read as evidence of lasting learning (Soderstrom and Bjork, 2015). So in an age of AI, we need to not only distinguish what learning has taken place, and what has “stuck”, but also distinguish between immediate performance and durable learning. Additionally, if assessment is meant to help us understand an individual’s knowledge, capability, judgement or ability to perform a particular task, then the evidence we collect needs to tell us something meaningful about these learning activities.
That sounds obvious enough, until AI enters the picture.
A student can now use AI to help write an essay, answer a Moodle quiz, respond to a case study, prepare a presentation, write a reflection or develop a plan of action. Each of those activities still produces assessment evidence. The real challenge is deciding what that evidence lets us say about the student’s learning.
Which brings us to the question at the heart of this blog:
What evidence would show us what the learner actually knows and can do?
Once we ask that, the assessment problem becomes a little easier to unpack. We need to think about the different kinds of capability we are trying to see, and then work out how to gather evidence that shows us what the learner actually knows, can do, and can judge when AI may have helped produce the work.
That gives us four useful questions to work with.
1. What Can the Learner Do Independently?
Some knowledge and capability needs to be available when AI, Google, notes or a helpful colleague are not. In the real world, a nurse may need to recognise an emergency, teachers have to spot misconceptions and respond in a pedagogically appropriate way, and a lawyer must understand complex problems quickly and accurately identify the legal precedent while cross-examining a witness in court.
A student studying professions like these needs enough knowledge to recognise what is happening, make sense of it and respond appropriately. Even when AI is available to help, and students choose to use it, developing their independent knowledge for real-life moments like these is critical. Taken further, students also need to learn how to judge whether an AI-generated answer is useful, accurate or completely off track, which means they need to understand the subject well enough themselves to make this call.
Assessments included in their courses therefore need to show:
What does this learner need to know, understand, or be able to do independently?
The evidence might come from a Moodle quiz that requires retrieval, an oral explanation, a practical demonstration, a fresh problem to solve or an observed performance. The important thing is to design such assessments so that we can see what the learner can actually do without relying on AI or other external assistance.
Retrieval and spacing are critical as well. Being able to answer a question immediately after a lesson is one thing, but being able to retrieve that knowledge a week or a month later, and use it in a different situation, is quite another. When learners can retrieve and apply important knowledge over time, we have stronger evidence that the learning has stuck and will be available when they need it.
2. What Can the Learner Do With Appropriate Support?
Most work in the ‘real world’ includes some form of group work, team work or support of various kinds. People use reference material, colleagues, templates, calculators, policies, search engines, job aids and increasingly AI.
So when thinking about assessment, we should also ask:
What can the learner accomplish when they use the tools and support that will realistically be available to them?
A student might use AI to brainstorm possible approaches, improve a draft, generate alternatives, explore different explanations or help structure a response. That performance gives us useful evidence of their application of the learning.
If the workplace expects someone to use AI competently, then our assessments also need to provide some measure of how well learners can work with this support structure. Current AI-assessment guidance increasingly recognises this need to understand students’ learning while also developing the capabilities required to work responsibly with AI.
For those of us looking for practical ways to rethink assessment in the age of AI, the Australian Tertiary Education Quality and Standards Agency (TEQSA) has published a useful resource, Enacting assessment reform in a time of artificial intelligence, that explores how institutions are putting assessment reform principles into practice.
3. Can the Learner Judge the Support They Use?
AI can produce a well-polished and convincing assignment, but it may be completely inappropriate for the situation. A student who accepts the first answer that appears on the screen is very different from one who has interrogated the content, adapted it to their question, and shaped it into a sound and accurate assignment.
In an age of AI and assessment, we therefore also need to ask:
Can the learner judge the quality of the support they are using?
Can they spot an error? Recognise a weak answer? Compare alternatives? Reject a suggestion that does not fit the context? Adapt an idea? Explain why they chose one option?
This is evaluative judgement, which Tai et al. (2018) define as “the capability to make decisions about the quality of work of self and others.” The researchers argue that developing this ability should be an important goal of higher education, helping students recognise what good work looks like, and making sound judgements about their own work and that of others.
In an age where AI can produce convincing answers in seconds, surely knowing how to judge the quality of those answers is becoming just as important as knowing how to produce them.
Interestingly, AI itself gives us some useful ways to assess this kind of judgement.
- Give students an AI-generated response and ask them to critique it.
- Ask which parts they would use and which they would discard.
- Let them compare two suggestions and justify their choice.
- Ask them to show how they adapted an AI-generated idea for a particular context.
Now their judgement becomes part of the assessment evidence.
4. Does the Learning Travel?
The fourth question we should ponder is whether a strong performance at a point in time is enough to say that learning has indeed ‘stuck’. It leads us to ask:
Can the learner use the learning again, later, somewhere else or under different conditions?
Perhaps they receive feedback and improve the next attempt. Maybe they apply the concept to a new problem. They might retrieve important knowledge several weeks later, perform the skill in another context or explain why their approach changed.
This gives us a pattern that aligns directly with the findings from research:
first attempt → feedback → action → new situation → further evidence
As we discussed last week, the more complex the capability we are assessing, the more useful and critical that pattern becomes.
Teaching, leadership, professional judgement and workplace capability, for example, benefit tremendously when evidence of learning is gathered across several tasks, contexts and points in time before we can make a confident claim about the learning.
So What Should We Look For?
When designing assessment in an AI-rich environment, four questions can help:
- What must the learner be able to do independently?
- What should they be able to do with appropriate support?
- How well can they judge and use that support?
- Can they carry the learning into another task, context or point in time?
Each question gives us a different piece of evidence.
Together, they give us a stronger basis for answering the question we started with:
What does this learner actually know, understand, do and have the judgement to decide?
Next week, we’ll put these questions to work with Becky, a student teacher completing teaching practice. We’ll look at how Moodle, and Limina’s PinBoard and the Reflection Journal, her lesson planning, classroom observation, feedback and AI can produce the program-level evidence needed to build a much richer picture of her learning as a student teacher.
Sign up for our newsletter
If you’ve enjoyed this blog, consider signing up for our newsletter to ensure your copy lands in your inbox each Thursday afternoon.
References
Kane, M. T. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73. Validating the Interpretations and Uses of Test Scores – Kane – 2013 – Journal of Educational Measurement – Wiley Online Library
Lodge, J. M., Bearman, M., Dawson, P., Gniel, H., Harper, R., Liu, D., McLean, J., & Ucnik, L. (2025). Enacting assessment reform in a time of artificial intelligence. Tertiary Education Quality and Standards Agency. Enacting assessment reform in a time of artificial intelligence
Soderstrom, N. C., & Bjork, R. A. (2015). Learning versus performance: An integrative review. Perspectives on Psychological Science, 10(2), 176–199. Learning Versus Performance – Nicholas C. Soderstrom, Robert A. Bjork, 2015
Tai, J., Ajjawi, R., Boud, D., Dawson, P., & Panadero, E. (2018). Developing evaluative judgement: Enabling students to make decisions about the quality of work. Higher Education, 76, 467–481. Developing evaluative judgement: enabling students to make decisions about the quality of work | Higher Education | Springer Nature Link
Dr Isabel Tarling – CEO
Isabel Tarling is a learning sciences scholar, programme designer and education researcher, with a PhD from the University of Cape Town. Her research in online and technology-supported learning forms the foundation for much of her writing. She aims to build easy-to-understand bridges between evidence-based research and its everyday application in real-world contexts. Isabel is the founder of Limina, where these ideas are put to work to reimagine learning for organisations of all shapes and sizes.
Research has a habit of getting stuck between the pages of journals and books, or somewhere between academic language and everyday practice. Our blogs help get those ideas moving. We dig into the research, unpack complicated ideas and work out what they mean for people who design, support and care about learning. Then we draw out useful principles, processes and strategies that can travel into practice, across all the wonderfully varied places and ways in which learning happens.