Climate Change and Museums

Fiona Cameron asked me about audience research we have done on the topic of climate change. We conducted several studies as part of a larger research project investigating future exhibition topics.

The first study in 2006 consisted of two quantitative surveys — one with an online sample of 538 respondents from the greater Sydney region (mix of visitors and non-visitors to museums and galleries); and one with 319 visitors to the Australian Museum. Climate Change was chosen as the preferred exhibiton topic by 21% of online respondents and by 22% of the onsite sample. It was of high interest to 39% of Australian Museum visitors and 26% of online respondents.

Why is this topic so appealing? A range of reasons were given by respondents:

* It’s an important subject that we all need to know about:
“I think climate change should be of interest to everybody because it is our future. Will we survive?”
“I think we need to know more about climate change as it’s already happening.”
“Topical and at the forefront of everyone’s attention. Will foster necessary public debate. Is of great interest to all ages and backgrounds.”

* It’s a topic that children need to be educated about:
“Climate change is an important issue and one that I could take my children to see and discuss.”
“Educating the community and my kids about this pressing issue so we can make immediate changes.

* There is a lot of misinformation around and we can trust the Museum to give us the real facts:
“I think we need to know more unbiased reasonably proven facts regarding climate change.”
“Climate change is very current and something that the whole world, and even more, needs more in-depth study, shown and interpreted for us normal people out here.”

* We need to know what we can do:
“… we all need to be made aware, with tangible examples, of what’s happening and how we can help.”
“Climate’s a very good indicator of the environment, must learn so we can save the planet before things go very wrong.”

The second study in 2007 consisted of focus groups with a range of target audiences to unpack the issues further. Again, climate change was clearly the most popular exhibition topic across all groups because:

* It’s an issue that’s become personal
* In the past 12 months, the issue has moved beyond the realm of science into the mainstream
* The sense of personal responsibility has started to hit home
* People are confused and overwhelmed – can’t trust politicians, industry or lobby groups
* With all the media coverage and THAT MOVIE, there is significant background knowledge, but much of it is contradictory, making the issue even more confusing!
* With so much conflicting information regarding climate change, audiences are craving an authority on the issue to clear things up once and for all, and assist them in adopting a stance

We found that participants felt the Museum was in a perfect position to be that authority, as audiences see us as:
* A trusted source of scientific information and not aligned to a political party, lobby group or big business partner:
"It’s a perfect fit for the Museum, it’s science!"
"I think the Museum has a great chance to be the authority on this issue, people trust the Museum and its science."

Some questions people wanted an exhibition to help them with were:
What are acceptable levels of warming?
What are the possible outcomes, at varying degrees of warming?
Are these changes due to emissions or part of normal cycles?
What is our ability to reverse the effects?
What can I personally do?
What is renewable energy and what is its role in beating climate change?
How does my everyday existence contribute now?
Is there a tipping point, and have we passed it?

So, the interest is there, what are museums and like institutions doing about it??

Logging On: Culture, participation and the web

Reading this interesting report at the moment - Logging On: Culture, participation and the web. Published on the Demos website and written by John Holden, it is a fascinating account of a major online initiative funded by the Department for Culture, Media and Sport in the UK. Some of the key points I picked up [my comments in square brackets]:
* Trends in internet developments show move from passive --> active; unidirectional --> interactive and from few --> many
* Early use of internet in museums was to digitise content
* 2007 use of internet was transformed in the UK through uptake of broadband with the net becoming the primary way that people access information. As well online commerce has transformed the music and film industries [who have had to respond to the illegal use of downloading to making it easy to do and minimal cost]
* “Rather than people using technology to adapt to their own needs, they are now using the capacity and functionality of technology to create and shape new potential.” (p.17)
* “... new web tools, greater computing power, better connectivity and increased uptake have changed the perception of the cultural possibilities of the internet from being a passive pool of digital information to being a vibrant fourth dimensions of life with limitless opportunities for interaction.” (p.17)
* Museums need strategy of user-generated content – this was why culture online was successful in actively engaging young people and adults in generating their own content
* The organisations in the pilot learned, as well as users developing their own IT skills and deeper understanding of the content
* [lessons for us – share resources such as developer expertise and project management; build projects across institutions and content areas. Is climate change/sustainability one we could do in Sydney?? Indigenous/Pacific stories are another]
* Lessons section (p.33 on):
# Need an external broker to bring partners together
# Clear project plan, roles and responsibilities
# Some institutions not used to rigorous project management systems – through the commissioning process gave them contractual obligations to deliver
# Need absolute clarity about who creates, controls [and signs off on] content
# IP needs to be sorted
# Need a recognised and established lead partner (i.e. the big museums) who can bring organisational resources to the project
# Start small-scale and move up from there
* Challenges:
# Bureaucracy often want control – therefore be clear on goals and responsibilities up-front
# Focus on user means that child protection is a big issue
# Sustainability needs to be factored in
# Need specialists in both the technology and the content [and the audience??]
* Useful section on moderation pp.39-40
* The Tate considers its website as the 5th gallery
* People don’t want to abandon the physical object, just want to find new ways to communicate about it
* “New web tools enable users collectively to express their preferences through their actions, without having to be asked.” (p.44)
* All the tools that Web 2.0 offers "... extend beyond simply the digitisation of data to models of linking and sharing information." (p.45)

Using visuals in evaluation

Digital Eva asks:
"I am also looking (or this is what basically my project aim) at how designer could design a tool in assisting summative evaluation by integrating drawings as the medium to reveal or gather information about audiences' experience."

I have used drawings quite successfully when evaluating exhibitions - it's a great tool for both adults and children. When I evaluated the Indigenous Australians exhibition I gave people a blank page with one question: "Draw how the exhibition made you feel". The responses were really amazing. As part of a front-end evaluation we asked children aged from 2-6 years to keep a journal of their museum experiences. The results were quite incredible as they mixed drawings with photos that they had taken thoughout the Museum. As part of our current research project, Culturally Diverse Audiences and Museums, we again used the journal method, this time to record participants' leisure activities and where museums fitted within their lives. In this study we also used photographs when interviewing people post-visit as a way to prompt memories of their experiences and as a deeper way to discuss their visit. They were photos that they had taken.

Also when doing a trawl, came across this useful resource Social Research Update published quarterly by the Department of Sociology, University of Surrey. There is an article there Visual research methods, although a tad old it has some interesting ideas. The article Using diaries in social research is also really good and relevant too.

I think that drawing and visual methods are a really good tool, especially when coupled with interviews (as you still need to get them to interpret their material). I'm wondering if there are any other examples out there??

A suitcase of portable meaning

Lynda, thanks for inviting me to post here.

As I was writing my paper, After the Show: Making sense after the event, for the Museums Australia Conference 2007, I was ruminating on the many ways that metaphors help exhibition visitors to make sense of their visiting experience. The National Gallery of Australia exhibition, Egyptian Treasures from the Louvre: Journey to the Afterlife, used the metaphor of the journey in its title and structure. When we spoke to visitors while evaluating the exhibition, we found that they often referred to the concept of a journey when speaking about their experience of the exhibition.



This had me thinking about whether the journey metaphor might apply to any exhibition.

It also had me thinking about the power of metaphors in any kind of communication. Looking around, I could see that metaphors are recommended in various communication fields. For example, Brian Clark espouses the power of the metaphor in writing on his Copywriter blog.

Given that museum exhibitions aim for excellent communication, I suddenly see the potential for more exhibitions to be conceived and structured around a powerful metaphor that can be a convenient suitcase of portable meaning for visitors. Or a handle they can grasp when they get challenged or disoriented. (See, it's fun playing with metaphors.)

What is your experience? How useful are metaphors as shaping-points for exhibitions? What exhibitions have you seen or presented that used metaphors? What were some lost opportunities?

Gillian Savage, Environmetrics.

Museums remixed

Lynda
have a look at this blog from an American Musem conference on:

visitor-authored experiences and other forms of user participation in physical museums and online. This blog now serves as both a resource and a forum for those interested in continuing the dialogue regarding visitor-authored experiences. Whether or not you attended the sessions, please contribute your thoughts to our blog!

Exploratorium Explainers

Lynda have a look at this blog - perhaps a good one to discuss for the forum, or the regular web meetings. It is put together by floor staff at the Exploratorium. Not exactly audience research, but a good slant on museum blogs.

Writing Survey Questions

My offsider, Pauline, is embarking on the new world of question writing as we develop surveys for our Museum Shop and Science in the Suburbs. This got me thinking about what I would recommend as resources for question-writing.

Found a good online resource from the Committee on Audience Research and Evaluation called Visitor Studies 201 presented at the 2006 AAM, with a downloadable pdf on developing questions. There is a 'how to' on writing questions posted by the Evaluation and Visitor Studies SIG with a link to an online guide to designing and conducting visitor surveys by Julie Leones, The University of Arizona.

As mentioned before, Judy Diamond's 1999 book Practical Evaluation Guide: tools for museums and other informal educational settings (Walnut Creek: AltaMira Press) is a good starting point. For general advice on sampling and suggested questions for a whole range of areas try:
* McManus, P. (1991). Towards Understanding the Needs of Museum Visitors. In G. Lord & B. Lord (Eds.), The Manual of Museum Planning (pp. 35-52). London: HMSO. (I think there's a newer version of this book and it's in our Research Library).

You could also look at general social research texts - one I have used is:
* de Vaus, D. (1991). Surveys in Social Research (3rd ed.). London: UCL Press, especially Chapter 6 (also in our Research Library).

Good luck!

Museums and Web 2.0

While at the MA conference I have been asked by several people here what Web 2.0 is ... this is worrying! My straw poll of museum professionals at this conference revealed that noone has heard of Web 2.0, let alone doing any thinking about what it might mean for museums. So I promised to send them something, and have blogged it instead (I have been kind and sent them an email also!).

The three things I recommend people view/read are:
* The Machine is us/ing us
* Nina Simon's blog post about implementing Web 2.0 in museums
* paper discussing organisational barriers and Web 2.0 We are going to discuss this on our wiki

Of course there's lots more but I don't want to overwhelm people! Jerry's papers presented at the MA conference had a great summary of how strategy needs to come before tools so, at the risk of now overwhelming people I would add this to your list.

Museums Australia Conference 2007

HI all, am attending the annual Museums Australia conference in Canberra and am going to attempt to blog my experiences. There are no internet facilities at the conference (!!) so will check in when I can.

A review of literature on audience research and evaluation for museum websites

There is a clear conception within the literature of the potential within the museum area for online development, given the already inherent value a museum has in terms of its reservoir of high quality content, often with its own intellectual property rights, and an established reputation for quality information provision, objectivity, and authority. (Both Daniel Cunliffe et al and Tom Hennes make similar points).

The challenge for museums on the web is to take advantage of and develop this potential, creating an online entity that is not just a pale reflection of the physical museum, but an exciting expansion into new ways of accessing, interacting with, exploring and developing resources held by museums.

Historically website design has been focused on the visual design, and when overhauling their sites, institutions such as museums may be faced with redesigning a site that may still ‘look’ good, but behind the scenes is dysfunctional, based on obsolete code and almost impossible to add to or update in a simple manner. Added to this is the challenge of addressing web 2.0 issues, including a desire to make the site more interactive and to bring the user into content manipulation and production. This adds another area of concern, with challenges to issues of control and management.

The literature shows that there are basic criteria to assess website usability and audience/user profiles that have been developed over the last 10-15 years, and these are seen as working effectively when applied as a group, rather than depending on any one method or aspect. A clear consensus emerges in regards to key issues.

These include: the necessity of usability testing, assessment and research at all stages of the web development process; the development of user profiles, as virtual users cannot be assumed to match physical visitors, about whom much research has already been done; the importance of a conceptual approach to the site, where the site is not just a reflection of the physical museum but an active entity in itself; and finally situating the virtual museum within a marketplace context.

To achieve this, the agreement is that there needs to be a lot of preparation, research, testing, trials and then more testing.

The earliest article, Usability Evaluation for Museum Websites, offers a thorough analysis of these criteria, which include:
1. user testing- with prototypes within the museum, monitor and observe interaction behaviour, use feedback. The user is usually given a set of tasks to carry out and is monitored. Can be monitored by video, think aloud protocol, observer’s notebook (problems include finding a typical user).
2. interviews- questionnaires about a prototype, an existing site, or ongoing feedback. Could be done via a kiosk in the museum.
3. log file analysis- looking at the data gathered automatically, includes such info as number of hits, type of platform, where the user came from, time spent on page and quite a lot more (problems include amount of data, relevance of data, interpretation, identification of unique visitors, caching) Then how do you extrapolate intention from info – why from what?
4. online questionnaires/feedback – the two major concerns are: self selection of sample; and response rate to draw reliable conclusions. Often there is a low response rate (between 2 and 5%)
5. Heuristic evaluation – doesn’t involve users, evaluates site against a set of agreed criteria or guidelines, research already available.
6. Procedures for ongoing maintenance and development must also be considered.

The article Audiences, Visitors, Users: Reconceptualising Users of Museum On-line Content and Services expands on these criteria to develop a more formal set of principles for web site development and evaluation. They put museum web site development into the context of the marketplace and come up with six design principles:
• acknowledge the marketplace web sites exist within
• have clear site goals, from the perspectives of the market, user, and organisation
• design in relation to these goals
• design an information space, not a museum replica
• test with target users at all stages, many times
• evaluate in terms of your goals, not just according to user demographic or products on site.

There are many examples in the literature (SFMOMA, Museum of Victoria) of organisations that take these basic criteria and offer case studies on how they apply them to their own redesigns, the challenges that emerged, and how effective the process was.
Some general points that come from these case studies are:
o Usability is important in developing an effective, interesting and user friendly website.
o There is a connection between website usage and attendance, however the audiences do not necessarily correlate completely.
o The different approaches – heuristic, user surveys, questionnaires, log file data – all contribute to a more complete picture of users and usability.
o How a user gets to the site is largely based on internet searches, rather than access through the home page.
o Museums have a very active role to play in developing their physical content for the online environment, and need to look at issues of intellectual property, copyright, etc.
o There is a lot of potential for museum sites – both directly and indirectly related to their physical role (see Hennes, Sherry Hsi)
o There are more novel methods of assessing user interaction for example the kiosk approach of the Exploratorium that can have a role in website evaluation.
o Web 2.0 – do the users want it? How do museums implement it in ways that do not threaten users, or waste resources? (see SFMOMA article, fresh and new blog re percentage of active users vs passive)
o How then do museums assess the use of this interactive content – data file analysis has not yet caught up with server side includes, active content, feeds etc.

Overall, for those embarking on web site redesign, or initial design, there is already a good body of research and practical examples to draw on for methods, ideas and examples. The challenge of web 2.0 and how to assess the role it plays in website assessment and audience research wil be another are for research and assessment over the next few years, as museums continue to take up the challenge of implementing web 2.0 concepts and ideas.

Evaluating teacher/student experiences at museums

My colleague here at the Museum, Pauline, is digging around for recent published studies of evaluations with teachers about their experiences, expectations and needs when visiting museums and cultural institutions generally.

The recent post I wrote in response to Alison's request is also useful background.

Pauline has promised to post some of the things she has found so far, but I was wondering if anyone had some studies, information, findings etc they'd like to share with us??

What are the most effective ways to recruit new audiences and build repeat visitations to a cultural institution?

This question came from Alison Wishart, State Library of Queensland (who promises to blog a response!): "My assumption is that one of the most effective ways is to give school students, who visit the cultural institution with their school group, an exciting and rewarding experience so that the students then go home and tell their parents about the fantastic cultural institution and the family visits as a group again and again. Is there any written research in this area that you could point me to?"

Alison, research that I know of has continually found that the characteristic which most impacts on adult museum visits is whether they were taken to museums as children and the types of experiences they engaged in. The "big picture" references are:
* Falk, J., & Dierking, L. (1992). The Museum Experience. Washington: Whalesback Books.
* Falk, J., & Dierking, L. (2000). Learning from Museums: Visitor Experiences and the Making of Meaning. Walnut Creek: AltaMira Press.

Prince investigated a range of attitudes and perceptions that were key to museum visiting. He found that visitors’ previous experiences with museums, as well as with learning and education generally, determined whether people then visited museums and the subsequent experiences they remembered. The reference for his work is:
* Prince, D. (1990). Factors Influencing Museum Visits: An Empirical Evaluation of Audience Selection. Museum Management and Curatorship, 9, 149-168.

Other references for these conclusions from a family perspective are:
* Ellenbogen, K. (2002). Museums in Family Life: An Ethnographic Case Study. In G. Leinhardt & K. Crowley & K. Knutson (Eds.), Learning Conversations in Museums (pp. 81-101). Mahwah: Lawrence Erlbaum Associates.
* Ellenbogen, K., Luke, J., & Dierking, L. (2004). Family Learning Research in Museums: An Emerging Disciplinary Matrix? Science Education, 88(Supplement 1), S48-S58.

and from a school-visit perspective are:
* Falk, J., & Dierking, L. (1997). School Field trips: Assessing Their Long-Term Impact. Curator, 40(3), 211-218.

The latter is particularly useful. A general overview of school visits and museums, with a comprehensive reference list is:
* Griffin, J. (2004). Research on Students and Museums: Looking More Closely at the Students in School Groups. Science Education, 88(Supplement 1), S60-S70.

I'm not sure if there have been any long-term tracking studies of visitation, but certainly Ellen's work is worth checking out.

Of course your broader question about new audiences generally is a whole new area but I have started small here. The big issue of how to attract and keep youth audiences (for example) can be found on here on my website. We also found in our study of audiences with disabilities that if museums offered great learning experiences, not just accessible ones, these groups would visit more regularly (and bring a whole lot more people with them).

Look forward to hearing from you.

audience research wiki

In the spirit of trying all forms of web 2.0 media, we have set up an audience research wiki , as a dicussion forum for audience research. Here we will post articles that particularly address museum websites and audience research. Please have a look, and add your comments and thoughts to our discussion pages. If you are new to wikis, have a look around, and also have a look at Chris Barry's education section wiki.

Innovative audience research

This post from Chris Larry: "Thought you might find interesting the work we are doing at American History Workshop with our Telling Lives story capture technology. This technology allows audiences to answer question to a video camera that is than collected as a response database." I think that this is a very innovative approach to seeking audience feedback. Are there any others out there?

Chris: do you see this as a sustainable way to continue capturing feedback? How has the organisation used the information in their future development of programs?

Doing audience research

Mel asked "What are traditional methods of finding out about your audience and is the internet affecting the way you do audience research?"

Audience research is a discipline of museum practice that provides information about visitors and non-visitors to museums and other cultural institutions, influencing the ways museums think about and meet the needs of their audiences and stakeholders. Audience research is also a strategic management tool that provides data to assist museums more effectively plan and develop exhibitions and programs; to meet their corporate goals; and to learn as organisations.

In audience research a range of methods are used to address issues such as who visits; motivation; behaviour; satisfaction and learning; as well as who does not visit and why. Areas examined include visitor demographics, visitor behaviour, leisure habits and learning strategies. Methods are chosen in accordance with the questions that need to be answered.

In a 1990 paper, Chan Screven noted that the most popular methods inlcude structured and open-ended interviews, questionnaires and rating scales, informal conversations, tracing visitor movements through exhibits and unobtrusive observations of stops, time spent, exhibit usage, and reactions to mocked-up exhibit components.

Other ways of conducting audience research include video-taping visitor behaviour and audio-taping conversations. Focus groups; telephone surveys; and online surveys and panels are also employed to research current and potential audiences as well as non-visitors.

A list of beginning readings about audience research in museums can be found on the
FAQ section of my website - on there you'll also find a downloadable list of beginning references.

I also wrote a paper on the history and practice of audience research in cultural institutions: Kelly, L. (2005). Evaluation, Research and Communities of Practice: Program Evaluation in Museums. Archival Science, 4(1-2), 45-69. This paper contains a good list of references. More information about it can be found at the publishers' website.

I'll get to the internet part of your question shortly.

Research on museum websites developed for teachers

I received the following email from the VSG list:

"Jessica from Randi Korn & Associates is looking for fairly recent studies of museum Web sites developed for teachers. She is trying to comb the literature to get an idea of how teachers are using the Web, specifically museum Web sites."

Jessica, I did a detailed study in November 2005 to gain a better understanding of how emerging internet technologies are affecting the learning environment, with a specific focus on teachers and high-school students. The overall objective of the research project was to provide us with guidance on how to best develop a website that meets the needs of students and teachers in the primary and secondary levels across a range of curriculum areas. General objectives were to gain insights into how students and teachers are using the internet and what they are looking for when they access websites.

We conducted a series of five focus groups with students and teachers from a mix of public and private educational institutions. There were primary school teachers from years 5-6; secondary school teachers from years 7-10 (mainly science teachers and co-ordinators); students aged 13 to 16; and a group of teachers specialising in technology and computing. The aim of the discussion was to look at participants’ use of the internet, their knowledge of media, and their reaction to a range of websites.

Ahead of time, participants received a letter outlining the aims of the research and providing a list of websites to review. These museum, gallery and general sites had targeted materials for teachers and allowed a two-way exchange of information - these proved to be extremey useful as a way to seek general and specific feedback - here's the list of the sites we used. As well, participants brought along a list of the website ‘favourites’ they used for recreation and education, which again gave us useful insights into how they were using the web.

The paper from this study (Kelly and Breault, 2006) will be published in the proceedings of the 2006 ICOM-CECA conference this year. I'm happy to email this paper to anyone who wants it.

Also, I'd be interested to hear of others' experiences, BUT rather than email - which only reaches a select few - why not blog your responses for many to share??

ExhibitFiles Website

Check out ExhibitFiles - a great new website for exhibitions and museum staff - top stuff!

There is an interview with the site developers on Nina Simon's site which gives good insights to the thinking behind the development of ExhibitFiles.

Do you have an audience research question?

Ask it here and I'll try to answer it for you. You may even be able to supply an answer yourself!