Early institutional messaging about generative AI mostly told students not to use it. For health management students this was a problem because they’re moving into planning and policy roles where AI is already shaping decisions, and they needed to form judgements about what these systems produce and how they might be useful (or not). In Term 1 2025 I redesigned the final assessment in PHCM9391 so that students had to use a large language model to analyse a real organisation’s strategic plan and then critically appraise its output in a structured reflection.
Students were taught about, given an explicit professional rationale for why health managers need to evaluate AI performance and recommendations, and transparent AI collaboration guidelines (included as a labelled appendix addressed to both students and AI tools, framing integrity as a matter of professional ethics). The task came after 70% of the course's marks and feedback had already been received, so students had a reasonable idea of how they had performed in the course so far.
I evaluated the redesign with four colleagues in the School of Population Health, drawing on 156 course evaluation responses. The mean grade rose to 87.8% (SD 7.9) from 81.7% (SD 12.8) the previous year, though clearer task structure and changes in the cohort mean the redesign is unlikely to be the only contributing factor. The task normalised engagement with AI and exposed students to its limits, but deeper critical evaluation was harder to achieve than we expected. Many students engaged instrumentally, and some carried anxiety from prohibitive messages in other courses. We reported this mixed experience openly, because it suggests to me that critical AI literacy needs to be sequenced across a whole program, which no course or assessment can acheive on its own.
We presented the work at the UNSW INSPIRED 2026 conference and the Council of Academic Public Health Institutions Australasia Teaching and Learning Forum in 2025. A student in the course who also teaches wrote to me that the task had prompted her and a colleague at the University of Sydney to think about how they might use AI proactively in the assessments they set.
