Evaluation
This section aims to introduce you to some of the different ways you might seek to evaluate your short program to inform ongoing course improvement—because that’s really what the whole purpose of evaluation is.
It is important to note that short programs do not fall under the same formal evaluation and continuous improvement requirements as standard university courses. Rather, evaluation of your course falls to you and your local area to undertake and review.
Often when people talk about evaluating a course, the first thing that comes to mind is student evaluations—surveys or questionnaires completed by students that evaluate the course and the staff teaching it. While the student voice is indeed important, there are many other valuable ways to evaluate your course. We aim to introduce you to some of these below.
When engaging in any evaluation activities, before you start it is important to ask yourself some of the following questions to get the most out of the process:
- Why am I doing this evaluation? What is its purpose? What do I want to get out of this process?
- Who do I need to get feedback from? What data source/s will I need?
- How will I present or use the data I collect?
You may not have clear answers to all of these questions just yet, and that’s ok. The following table may be useful in considering what kinds of evaluation activities you need or want given your situation, what kinds of information you can obtain through these activities and who would be involved in the process.
As you read through the table, consider which evaluation activities may be of most use to you right now, what might you like to (or have to) consider in the future, and how could you incorporate these evaluative activities into the delivery of your course.
Who is involved?
|
Description of evaluation: What and why
|
Data sources: How the evaluation may occur
|
Institution/Academic Unit review
(annually)
|
Typically, evaluation activities at this level facilitate decision making around the university’s ongoing strategic provision of learning experiences and allocation of resources.
Evaluation activities tend to focus on intended short and long terms outcomes relating to:
- Revenue generation
- Student enrolment (as compared to past offerings, other Short Courses)
- Student enrolment into related award programs at UniSA
- Increased brand recognition for UniSA
- Employer satisfaction (where appropriate)
- Professional Organisation accreditation (where appropriate)
- Professional Organisation satisfaction (where appropriate)
- Student employment outcomes
RESOURCES TO GUIDE PREPARATION
|
- Post course surveys
- Enrolment data from SAS
- Financial data
- Institutional course evaluation data
- Anecdotal/informal data from students, employers or professional organisations
- Employer and Professional Organisation satisfaction will need to be gathered via existing networks
|
Self-reflection
(during and after each offering)
|
A personal exploration of whether the teaching strategy led to the intended outcome or if changes are needed for the next offering.
This self-reflection is best conducted as soon after the teaching and learning experience as possible.
|
- Teaching journal - start by asking...
- What worked well? Why?
- What didn’t work well? Why not?
- What will I do the same next time?
- What will I do differently next time?
- Learning Analytics relating to student engagement/activity in online learning spaces
- Assessment scores (if applicable)
|
Student Feedback*
(during and/or after each offering)
(NOTE: The course evaluation process used as part of standard degree courses does not apply to Short Courses, so all evaluation is your responsibility to set up and review.)
|
Feedback from students regarding their engagement, challenges, or appreciation of certain course elements to identify strengths and weaknesses of the course design or teaching strategy. This could be achieved using a combination of informal and formal feedback.
RESOURCES TO GUIDE PREPARATION
- Some ideas for informal student feedback are presented on the Delivery page.
|
- Course evaluation data (myCourseExperience is not available so you’ll need to make your own!)
- Within delivery and post-delivery student surveys
- Post course follow-up re: career advancement, employment, or enrolment in other UniSA award programs or other Short Courses
- Ad hoc feedback from students via email
- Feedback via discussion forum (or other Moodle tools)
|
Peer Feedback*
(during or after initial offering, subsequently when course design or teaching strategy changes)
|
Peer observation of a course or teaching practice by colleagues can inform both the continual improvement of the course being observed, and also be a way of sharing/demonstrating good teaching practice.
Evidence of colleagues subsequently applying the same approach after peer observation can also be a measure of impact and effectiveness.
There are many guiding frameworks that can assist in determining exactly what you would like your peers to provide comment on, and how to prepare all involved for a constructive and positive experience—giving and receiving feedback is not without challenges.
RESOURCES TO GUIDE PREPARATION
|
- Observation of teaching (in the classroom if applicable)
- Review of online course design
- Review of online teacher presence and community formation (via discussion forums, videos, etc.)
|
Theory/scholarship
(conducted whenever there is interest)
|
For those interested in engaging in the scholarship of learning and teaching.
A comparison of the teaching approach or course design with the findings published in the literature to reveal whether the outcomes are similar or different from what others have reported.
If outcomes are different, there may be opportunity to publish or disseminate the new findings through conference proceedings, journals, forums or institutional events.
RESOURCES TO GUIDE PREPARATION
|
- Self-review
- Student + peer feedback (if you’ve received ethics approval beforehand)
- Moodle and associated learning data (Learning analytics)
|
*To ensure the feedback you get is the feedback you want, it is important to be clear from the outset about what exactly you would like your audience to comment on. If there is a particular learning activity, video recording or section of the course you are wanting constructive feedback on, be sure this is clearly communicated prior to making your request.
Importantly, you should now be able to recognise that course evaluation is not a cookie-cutter, one-size-fits-all activity. Rather, course evaluation should be treated as dynamic, ongoing and constantly evolving depending on the needs of the course and your own professional development. If you have any questions about evaluating your course or your own teaching practices, feel free to contact the Teaching Innovation Unit to discuss further.