A review of literature on audience research and evaluation for museum websites

There is a clear conception within the literature of the potential within the museum area for online development, given the already inherent value a museum has in terms of its reservoir of high quality content, often with its own intellectual property rights, and an established reputation for quality information provision, objectivity, and authority. (Both Daniel Cunliffe et al and Tom Hennes make similar points).

The challenge for museums on the web is to take advantage of and develop this potential, creating an online entity that is not just a pale reflection of the physical museum, but an exciting expansion into new ways of accessing, interacting with, exploring and developing resources held by museums.

Historically website design has been focused on the visual design, and when overhauling their sites, institutions such as museums may be faced with redesigning a site that may still ‘look’ good, but behind the scenes is dysfunctional, based on obsolete code and almost impossible to add to or update in a simple manner. Added to this is the challenge of addressing web 2.0 issues, including a desire to make the site more interactive and to bring the user into content manipulation and production. This adds another area of concern, with challenges to issues of control and management.

The literature shows that there are basic criteria to assess website usability and audience/user profiles that have been developed over the last 10-15 years, and these are seen as working effectively when applied as a group, rather than depending on any one method or aspect. A clear consensus emerges in regards to key issues.

These include: the necessity of usability testing, assessment and research at all stages of the web development process; the development of user profiles, as virtual users cannot be assumed to match physical visitors, about whom much research has already been done; the importance of a conceptual approach to the site, where the site is not just a reflection of the physical museum but an active entity in itself; and finally situating the virtual museum within a marketplace context.

To achieve this, the agreement is that there needs to be a lot of preparation, research, testing, trials and then more testing.

The earliest article, Usability Evaluation for Museum Websites, offers a thorough analysis of these criteria, which include:
1. user testing- with prototypes within the museum, monitor and observe interaction behaviour, use feedback. The user is usually given a set of tasks to carry out and is monitored. Can be monitored by video, think aloud protocol, observer’s notebook (problems include finding a typical user).
2. interviews- questionnaires about a prototype, an existing site, or ongoing feedback. Could be done via a kiosk in the museum.
3. log file analysis- looking at the data gathered automatically, includes such info as number of hits, type of platform, where the user came from, time spent on page and quite a lot more (problems include amount of data, relevance of data, interpretation, identification of unique visitors, caching) Then how do you extrapolate intention from info – why from what?
4. online questionnaires/feedback – the two major concerns are: self selection of sample; and response rate to draw reliable conclusions. Often there is a low response rate (between 2 and 5%)
5. Heuristic evaluation – doesn’t involve users, evaluates site against a set of agreed criteria or guidelines, research already available.
6. Procedures for ongoing maintenance and development must also be considered.

The article Audiences, Visitors, Users: Reconceptualising Users of Museum On-line Content and Services expands on these criteria to develop a more formal set of principles for web site development and evaluation. They put museum web site development into the context of the marketplace and come up with six design principles:
• acknowledge the marketplace web sites exist within
• have clear site goals, from the perspectives of the market, user, and organisation
• design in relation to these goals
• design an information space, not a museum replica
• test with target users at all stages, many times
• evaluate in terms of your goals, not just according to user demographic or products on site.

There are many examples in the literature (SFMOMA, Museum of Victoria) of organisations that take these basic criteria and offer case studies on how they apply them to their own redesigns, the challenges that emerged, and how effective the process was.
Some general points that come from these case studies are:
o Usability is important in developing an effective, interesting and user friendly website.
o There is a connection between website usage and attendance, however the audiences do not necessarily correlate completely.
o The different approaches – heuristic, user surveys, questionnaires, log file data – all contribute to a more complete picture of users and usability.
o How a user gets to the site is largely based on internet searches, rather than access through the home page.
o Museums have a very active role to play in developing their physical content for the online environment, and need to look at issues of intellectual property, copyright, etc.
o There is a lot of potential for museum sites – both directly and indirectly related to their physical role (see Hennes, Sherry Hsi)
o There are more novel methods of assessing user interaction for example the kiosk approach of the Exploratorium that can have a role in website evaluation.
o Web 2.0 – do the users want it? How do museums implement it in ways that do not threaten users, or waste resources? (see SFMOMA article, fresh and new blog re percentage of active users vs passive)
o How then do museums assess the use of this interactive content – data file analysis has not yet caught up with server side includes, active content, feeds etc.

Overall, for those embarking on web site redesign, or initial design, there is already a good body of research and practical examples to draw on for methods, ideas and examples. The challenge of web 2.0 and how to assess the role it plays in website assessment and audience research wil be another are for research and assessment over the next few years, as museums continue to take up the challenge of implementing web 2.0 concepts and ideas.