So What Are Your Students Actually Learning From the Online Courses You Built?
In this Article
A note before I start: This week’s blog is a little different. It is less how-to and more reflection on a project that has occupied much of my thinking over the past few months.
In March this year, Alex and Gerald from JEMBA approached us to help get their Moodle LMS up and running, and convert a substantial curriculum, developed over many years of face-to-face teaching, into online learning.
The LMS part was relatively straightforward. Within weeks, their platform was Learning Ready and waiting for content. The courses were another matter altogether.
Like many organisations moving existing learning online, JEMBA had been exploring an AI-supported course builder. The early results looked promising. They uploaded existing PDFs and slides, gave the AI clear instructions, and it organised the content, generated activities and produced an interactive online course remarkably quickly.
It was worth exploring properly.
Over the next few months, we worked with the JEMBA team to test and refine the process. We improved prompts, rebuilt courses, reviewed the results, changed the prompts and tried again.
Each iteration got slicker.
The content was neatly chunked. There were plenty of interactions. Some of the AI-generated imagery was stunning. The result increasingly looked like a professionally built online course.
Yet one question kept bothering us.
What Are the Students Actually Learning?
Let me pause to give you some context to explain our mounting frustration at that point. JEMBA’s curriculum does more than transfer information; much of the learning asks students to reflect, question assumptions, consider different perspectives, make connections and develop judgement.
The AI course builder could take complex source material and produce content about these things. That turned out to be very different from designing the learning experience through which somebody actually does them. As a result, a course on reflection became content about reflection. A course to shift perspectives turned into an activity explaining different perspectives, and contemplation morphed into something to read about rather than something the learner was given time and space to practise.
The courses had all the visible parts of online learning, but something essential was disappearing in the AI-course-building process.
To me it felt like the AI couldn’t capture the deeply transformational learning, leaving the course essentially soulless (if that is even a thing!). You’re so right – this all sounds rather dramatic; allow me to explain. Imagine looking at a robot-produced reproduction of Van Gogh’s Starry Night: all the right images, shapes and colours are correctly replicated. Yet standing in front of the original is something else entirely, almost as if you’re experiencing a small part of Vincent through the textured paint streaks and colourful vibrance. This is what I felt like — the AI-reproduced content just couldn’t capture the essence of Alex and Gerald’s courses.
So We Changed How We Worked
By July, we had spent enough time re-prompting, re-building and re-testing to know that another round was unlikely to solve the problem.
We stopped trying to generate the transformative learning experiences from the JEMBA courses with the AI course builder and returned to storyboarding from scratch; one might even say “by hand”!
As we did 10 years ago, our learning design team asked the questions that should shape every learning experience design. What does the learner need to take from this and use in their life? What needs to happen before the next conceptual leap? Where do they need private space to think, and where should they encounter other people’s ideas? When should we explain something, and when should the learner do the intellectual work themselves, and how much cognitive load are we creating? How do we support cognitive dissonance when new learning challenges the assumptions students already hold?
Interestingly, AI did not disappear from the process. Quite the opposite.
The AI intern sat beside me throughout the redesign. It reviewed storyboards, spotted patterns, challenged decisions, suggested alternatives and helped hold large amounts of content in view. It drafted and refined dialogue, helped develop learner-facing explanations, prototyped visual ideas and became a rapid production partner.
The working relationship looked something like this:
Human intent → AI proposition → human critique → AI revision → human judgement → further iteration
AI was everywhere I was, but in a very different role than when we started. My learning-design judgement stayed front and centre, and AI fulfilled much more of a supportive, fast and capable intern role.
Looking Back, We Can Now See the Pattern
The JEMBA experience became one of the foundations for the Limina AI Learning Design Continuum we introduced last week.
At the surface level, where learning deals with facts, definitions, procedures and clearly specified outcomes, the AI intern can carry a great deal of the work.
As the learning becomes deeper, AI remains a strong partner, but the learning designer needs to direct more closely. Where sequencing, interpretation, prior knowledge and the connections learners need to make become increasingly important, or where the learning involves cognitive dissonance that challenges learners to think or act differently, experienced human judgement is critical.
The JEMBA learning design experience helped us surface this continuum in practice. Most significantly, it helped us clarify that the further learning moved towards judgement, meaning-making and transformation, the less useful AI became as an autonomous course builder and the more useful it became as a partner to an experienced human learning designer.
The Freedom to Find a Better Way
Two prototype courses are now complete, with approximately 40 ultimately planned. AI remains deeply embedded in the work, including the production of engaging video content.
None of this experimentation would have happened without Alex and Gerald. They gave us something unusually valuable as clients: permission not to know the answer at the beginning. They allowed us to experiment, rethink the approach, and invent new ways of working when the original process did not give us the learning outcomes we wanted.
The result has not only changed how we create the JEMBA courses, but how we think about AI-supported course development at Limina.
It also made us more focused on the question that shapes all our learning projects:
So What Are Your Students Actually Learning From the Online Courses We Built?
That question comes before we reach for the AI course builder. It helps us decide what the learner needs to walk away with (after finishing their course), what the learning designer needs to create to make this happen, and where AI can make the work more effective.
Ready to Ask the Right Question About Your Online Learning?
A polished course is not necessarily an effective one. Book a complimentary 30-minute conversation with Limina to explore what your learners actually need to take away from their courses, and how learning design and AI can work together to make that learning happen.
Free Download
Sign up for our newsletter to receive this month’s decision guide for learning teams. It gives you the decision tree for knowing which parts of learning design AI can help with and the parts you still need a human designer for.
Dr Isabel Tarling – CEO
Isabel Tarling is a learning sciences scholar, programme designer and education researcher, with a PhD from the University of Cape Town. Her research in online and technology-supported learning forms the foundation for much of her writing. She aims to build easy-to-understand bridges between evidence-based research and its everyday application in real-world contexts. Isabel is the founder of Limina, where these ideas are put to work to reimagine learning for organisations of all shapes and sizes.