Sunday, November 09, 2008

From Taipei, Part 2

Spent a really interesting morning on Saturday with Wanchen' s students from the Fu Jen Catholic University. Their task was to read an evaluation report I had sent them, comment on it and raise questions. They did a really good job, considering that some of the reports were rather complex and long! The questions they raised came under the following categories (which I think I'll use when reorganising my website as part of the general Museum website rebuild project).

Planning an evaluation:

  • Working with those who commissioned the evaluation (for example, exhibition project teams, program managers, other staff and specialists)
  • Understanding the content of the program to be evaluated
  • Developing the evaluation questions – what do we already know and what are the gaps?
  • Determining the target audience
  • Designing the method

Undertaking the evaluation:

  • Different methods (questionnaires, online surveys, focus groups, in-depth interviews, tracking, observations)
  • Sampling – how to maximise sample based on target audience; determining the sample size; maximising response rates
  • Working with consultants

Analysing results/writing the report:

  • How to write reports that people will read (for example, layering information, including Executive summaries)
  • Mixture of text and facts/figures
  • Identifying what it all means

Applying the findings:

  • How to get findings accepted
  • How to use the findings in planning future programs
  • Engaging and involving staff in the evaluation (below are points from one of my Monday papers):
    • Include staff in planning the evaluation
    • Involve staff in the data gathering phases – attend focus groups, conduct surveys, undertake observations
    • Work through the findings together – the final report should never be a surprise
    • Include staff in debriefs – and have an open discussion at the debrief about what the findings mean both for the current exhibition/program and for future projects
    • Have an objective third-party undertake evaluation if findings may be of a sensitive or controversial nature, or when you need specialist skills
    • Use current organisational communication systems to publicise findings in usable ways
    • Use new media to disseminate information (for example make films or use sound bytes)

The session was a great and I thank you all for having me. Ended the day with visits to the Longshan Temple, Red Theatre Tea House and an awesome vegetarian restaurant including music, magic and kung-fu performances – thanks Tzu-yu, Yider and Cookie. (Oh, and also managed to buy myself a pair of nice brown boots for $41AUD!!)

No comments: