Sunday 26 June 2016

To assess or not to assess?



I'm wondering about the issue of 'assessment' in any induction program. Of course, we know for the purposes of grant providers and our universities, we have to 'evaluate' the program, but do we 'assess' the participants? (And therein lies another conundrum: we call staff attending our programs 'participants' not 'students', so are we somehow suggesting they are not 'learning' something?) The concepts that Tai and Peter identified as at the heart of professional development for staff are useful; they tell us much about the 'philosophy' of our programs, but not really if our staff have actually 'learnt' them. 

We have insufficient 'evidence' that professional development can DIRECTLY improve student learning, although there is evidence (perhaps not enough) that those who have undertaken longer programs have higher student evaluations, and are more 'student-centred' in their teaching approaches. 

But should we 'assess' staff in an induction program? And what form should that assessment take?
I would like to suggest that we consider self and peer assessment at least. As 'postgraduates' our staff should be relatively mature in their ability to discern and honestly assess their own learning, if we have fairly explicit learning objectives or outcomes, not something vague like 'shows a positive disposition to student-centred learning'.

But how would we construct a peer assessment? If the program is mainly online, it is RELATIVELY difficult to 'test' attitudes, and knowledge and skills, all of which make up the 'learning' we'd expect. We wouldn't want an 'essay' would we? But a Discussion Board wouldn't be an adequate form of testing either. 

I'm not sure of the answers here, but I think it would be a useful exercise to ask 'participants' to devise a form of 'test' that might be appropriate for peer assessment. Any thoughts?



Liu, N. & Carless, D. (2006). Peer feedback: the learning element of peer assessment, Teaching in Higher Education, 11:3, 279-290, DOI: 10.1080/13562510600680582

Yoni Ryan

Adjunct Professor in Higher Education: QUT, 
U Tasmania Senior Fellow of the Higher Education Academy (SFHEA) Learning and Teaching Unit 

5 comments:

  1. I think there definitely should be assessment, but agree that traditional forms like essays would be unlikely to assess much of the learning we're actually trying to enable. In our new self-paced Graduate Certificate in tertiary Education that we're launching on 5 July, we're assessing by portfolio. Participants (yes, we call than that, too) are expected to:
    - engage with the learning resources (practical and theoretical),
    - apply that learning in their own teaching contexts,
    - evaluate its impact (potentially including student and peer evaluations), and
    - reflect on all that in a portfolio, thereby creating evidence and a narrative addressing the course learning outcomes.

    Projects of this nature for new academics might be somewhat constrained because of limited scope for making changes in their teaching contexts, but in my experience there's always something that can be achieved. And the process of going through such a learning/applying/evaluating/reflecting process constitutes SoTL in itself.

    ReplyDelete
  2. I'm open but not convinced on the idea of a summative assessment for any induction program. I'd really like to hear views on this at our face-to-face meeting.

    I do like the peer and self review as part of the induction. I'm not sure that we can assume our staff are mature in their ability to discern and assess their own learning, but that is a CQU perspective. A definite gap that we could fill with the induction.

    Lots of questions and discussions to follow. Keep them coming.

    Julie

    ReplyDelete
  3. Hi All - assessment in an induction program would likely be unusual. I am super keen to introduce the concepts of peer and self assessment, and rather than have that in a separate module, perhaps we could integrate through the program. We certainly could take a formative assessment approach.
    Peter the portfolio approach might be a little much for 20 hours of engagement.

    Look forward to hearing the views of others.

    Kym

    ReplyDelete
  4. Hi, Some form of assessment might be necessary to provide evidence to TEQSA that new teaching academics have "skills in teaching, learning and assessment relevant to the needs of the student cohorts involved".

    Peter's idea of peer observation activities with a reflective component has merit, as do 'quizzes/reflective writing' on aspects of session 'pre-work' worked for those who 'engaged' in these activities which were embedded in a recent semester long induction run at UQ.

    Requiring completion of these activities is the 'tricky' part and may depend on the degree to which the induction is 'mandated' by the institution as part of T&L orientation that meets TEQSA 'audit' requirements.

    A developmental and voluntary approach is preferable but if these components are not completed do we grant the certificate of completion?

    As Julie says, lots of questions, and the capacity for self-review by staff varies, but the real assessment is the formal appraisal and starting academics on an early path of self-reflection, review and data collection prepares them better for this event.

    Integration in the program is ideal Kym and makes for better discussion of 'what's happening' in their classroom during the program.

    Cheers
    Ellen Dearden

    ReplyDelete
  5. In responding to Yoni and others on the challenge of ‘to assess or not ‘ -
    There is the definite expectation that our ‘ participants’ are learning something, in fact we anticipate they will experience changes/ transformations in how they perceive learning and teaching – that is regularly written as the intended learning outcomes of preparation for teaching programs. What we need to have expressed / and made explicit is how do participants feel about these changes/ and how are they recognised in their practice and behaviour.
    Is the evidence of changes/ transformation in the way in which participants can now talk of learning and teaching? – By using the language/ and nuances of the field or in their greater familiarity with the teaching and learning literature so they can now acknowledge what they are doing and why, and be able to verbalise the process. These transformations are often seen / raised in self reflections and come to the surface through professional conversations and dialogue. So is it the essence of the teaching induction/ professional learning program to facilitate this discourse? By providing the means to share and experience professional conversations with others, and in so doing try out the new knowledges and conceptions in practice – perhaps through blogs/ wikis/ online discussions / social media/ new technologies in the future. Once these dimensions are embraced is there any going back, or has a threshold been entered that cannot be reversed?
    Asking pertinent questions and prompting participants to challenge their thinking and verbalise the answers to probing questions could be sufficient.
    We have been undertaking a guided process of reflection in development of Fellowship applications through the UK Higher Education Academy. This has involved a series of interview prompts to support self-reflection and articulation of teaching practices and rationale.
    Might the opportunity to observe others undertaking this transformation (peer observation and conversations) and reflecting on one’s own approach and rationale/impact for teaching practice be enough?
    Something to consider carefully
    cheers
    Caroline

    ReplyDelete