Designing for Teens: MW2008 Day 3

Went to this really fascinating session this afternoon (after shopping!!). The points that were raised were how important it is to work closely with teens when designing websites for them. Also teens really wanted to use a range of media (although not podcasts of artists' talking!) and have two-way conversation, with the instituting and each other.

The first paper about a project called ArtPad at the Glenbow consulted teens by showing them works of contemporary art and finding out what questions they had. They found that the front-end evaluation really challenged their assumptions and informed the process.

Anne Neilson from the Statens Museum for Kunst worked with 90 teens over 400 days! They now have 500 profiles on-line and they also use these as a laboratory – a way to see what their audience

Advantages of their site:

• It's not commercial
  
• It's in Danish – important for them to express themselves in Danish
  
• It's democratic
  
• It's a small and safe community – and exists on-site too
  
• It's not just flashy show off – it's serious, you can really discuss and engage with each other
  
• There's room for different kinds of users and user behaviour
  
• The context of the Museum – gives some authority and identity
  

Challenges:

• Education – teachers aren't that keen to use it
  
• Continuum from museum to community
  
• Community member grow "too old"
  
• Resources –time and funding
  
• The internet ecosystem – at moment doesn't link well into other sites and how should this be done? The audience wants to keep it for themselves – it's a safe place for their inner art nerd
  

Walker Art Centre project have a WalkerArtCentreTeenArtsCouncil (WACTAC). They started with using this group as way to talk to teens about what they're doing online:

• Messaging each other privately and publicly
  
• Links to other cool thing they've seen
  
• Shows/events
  
• Videos, YouTube
  
• Showing their work (art, videos, etc) and posting to their pages
  
• Customising their profiles to show their identity and be different from others
  

What is the Walker doing that works?

• Have a good set of blogs that people are actively using – institutional acceptance of blogs and how to use and maintain them
  

What else do they need?

• Institutional program info
  
• Maybe enticing teens into doing reviews and criticism?
  
• Public face of WACTAC?
  
• Should be an open-vessel – ready for expansion
  
• Teens want to be more playful and fun, fresh set of views
  

The new site for teens is divided into business side and teens side of things – looked a bit too busy for me but I'm definitely not the audience and they obviously like it! The backgrounds can be customised by the user and they demonstrated a slideshow of designs – this enables people to design their own look and take ownership of it. It's also clever how they have used outside services like delicious and upcoming (an events calendar feed).

What did they learn:

• Make teens part of the process, not just part of the product
  
• With great power comes great responsibility! But by giving them ownership they take responsibility
  
• Thinks about the site as an educational program – don't focus on marketing
  

Overall I think the session should have been called designing with teens, not for teens as that's what really came out. Here's a link to the educational websites study we did some time ago that found pretty much the same thing.

Why metrics suck – Seb Chan, MW2008 Day 2

Attended Seb's talk today about metrics. Points and my interpretations:

• Start with asking why do we have a website?
  
• We want to engage audiences – but how
  
• Need to build metrics for each reason
  
• Metrics don't calculate return on investment (ROI)
  
• People don't come in through the front page, come via Google which has changed the way that people can search (see example of search for Powerhouse Museum) – there is a basic search problem here
  
• Lot of metrics that measure time spent often used to buy ads on commercial sites
  
• Need to track email users – what links do they click on, what are the read rates, asked how are museums segmenting their audiences? (only 2 did!)
  
• When developing email campaigns need to think about how you're going to track subscribers – should be segmenting them too
  
• Reinvigorate – a view of the site in real time
  
• AttentionMeter – compares different counting systems (I think? Got lost at this point...)
  
• Compete is another comparison site
  
• Hitwise actually gives more data than I was looking at each month
  
• Flickr commons measures – how many people favourite them and how many tags, how many "sensible" comments
  
• egosurf – fun site to see how popular you are!!
  
• Citation analyser – Domain Tools
  

Also refer to the post I did on measuring "online" success. I'm still unconvinced as I think we need to do more talking to people – when we evaluated educational websites, museum sites that we thought were good the students often hated! Seb did state that while we need to do qualitative research we need to segment the audiences first – I'm still not sure about this...

Measuring “success” in the on-line environment

Been thinking quote a bit about this topic as we go into rebuilding the Museum's website. When doing my doctorate came across a quote that nicely encapsulates the focus we should be having: not asking how many people visit, but how valuable are their visits (reference to follow – can't lay my hands on t at the moment).

In our Museums and the Web paper Angelina and I look at different ways on-line users have been classified, and report on studies at the Australian Museum that have attempted to unpack these findings further. Seb Chan's paper, Towards New Metrics of Success For On-Line Museum Projects, argues for a more holistic way to look at on-line visitation, using new tools and broadening the segmentations first proposed by Peacock and Bronwbill.

My Museum colleague, Russ Weakley, set out some ideas about ways to measure visitor engagement with our new website which seem to be quite useful on a broader scale. Here's some text from an email he sent about this issue:

We need to look at methods that measure quality rather than quantity. We may have 2 million visits a year (for now), but how many were satisfied? How many got what they wanted. How many engaged? ... More importantly, we will soon be focussing much more on HOW people are using the site and if we have made any financial gains, not how many people visit. This means looking at things like:

- how many comments?
- how many tags?
- how many discussions between users?
- how many questions have users answered for themselves?
- how many images have users uploaded?
- how many members do we have?
- how active are these members (do they join and never return)?
- how many people are being converted from free to paid membership!!!
- how many products have been sold?
- how many event pages have been visited due to site-wide advertising?
- how many real donations have been made via the website and why?

Russ also pointed out that we need to go beyond comparing ourselves to other museums and galleries to looking at how we compare to sites like Wikipedia, MySpace, Facebook, Flickr, YouTube, etc. More importantly, is how we compare to other sites that are producing similar nature focussed content (for our Museum) within the commercial and private sector.

So, much food for thought and I'll be keeping my eye on this one.

E-kids’ college: afternoon session

What we did this afternoon was for the students to interview some of the Museum's scientists about what they do and their personal and professional interests. Mark and Amanda fielded many questions about their work and how they got interested in their field of work.

There were some really provocative and interesting questions from the students. One lovely one was How do we know what people know and what people don't know? Mark answered that by reminding us that all the answers aren't always to be found on the internet and that there are really cool books that you can use too. They also asked What was the most interesting thing they had discovered? Mark discovered a new species of wallaby, and Amanda's friend also discovered a new species of fish. Other questions were: What is the most weirdest and wackiest thing you've ever done in your job? Any scary experiences? Where's the most interesting place you've been to? Thanks Mark and Amanda, I know I learned more about you two who I usually see every day! Now I understand more about your passions and why you got into doing what you do. Amanda reminded us to hang in there as it may take some time to get some paid work.

The discussions have been really productive, some ideas that emerged at the end were that they:

• don't want to use mobiles for accessing internet as they are too hard to use (with small keyboards, etc), the screens are no good and they don't have a mouse to use, although when I asked them about whether they'd like to use some kind of mobile device throughout the Museum to gather content they were really keen
• suggested that we have a blog with question of the week that they can email in
• would like to have a section on the website where they could leave a comment/feedback about their experiences and see what others have said
• really wanted to talk to scientists via some kind of video link, where you log in and talk to them in real time online
• felt we should be uploading video content both to our own site and YouTube (they admit their views on this had changed over the day)
  

The day finished with us going on a tour of an area that the Museum is thinking about turning into a new media space. And after that we thanked them, gave each school a certificate and finished!

Thanks so much for all your efforts as we now have a goldmine of information we can use when developing our new website and other digital technologies.

E-kids’ College

It's an exciting day for us here at the Museum. Around 24 students from schools across NSW are attending the second of our Kids' Colleges. This one is looking at digital technologies, how young people are using the web and to explore how e-learning relate to museums and young people. I'll attempt to blog as we go with my impressions so this may be a long and fragmented post ...

A review of literature on audience research and evaluation for museum websites

There is a clear conception within the literature of the potential within the museum area for online development, given the already inherent value a museum has in terms of its reservoir of high quality content, often with its own intellectual property rights, and an established reputation for quality information provision, objectivity, and authority. (Both Daniel Cunliffe et al and Tom Hennes make similar points).

The challenge for museums on the web is to take advantage of and develop this potential, creating an online entity that is not just a pale reflection of the physical museum, but an exciting expansion into new ways of accessing, interacting with, exploring and developing resources held by museums.

Historically website design has been focused on the visual design, and when overhauling their sites, institutions such as museums may be faced with redesigning a site that may still ‘look’ good, but behind the scenes is dysfunctional, based on obsolete code and almost impossible to add to or update in a simple manner. Added to this is the challenge of addressing web 2.0 issues, including a desire to make the site more interactive and to bring the user into content manipulation and production. This adds another area of concern, with challenges to issues of control and management.

The literature shows that there are basic criteria to assess website usability and audience/user profiles that have been developed over the last 10-15 years, and these are seen as working effectively when applied as a group, rather than depending on any one method or aspect. A clear consensus emerges in regards to key issues.

These include: the necessity of usability testing, assessment and research at all stages of the web development process; the development of user profiles, as virtual users cannot be assumed to match physical visitors, about whom much research has already been done; the importance of a conceptual approach to the site, where the site is not just a reflection of the physical museum but an active entity in itself; and finally situating the virtual museum within a marketplace context.

To achieve this, the agreement is that there needs to be a lot of preparation, research, testing, trials and then more testing.

The earliest article, Usability Evaluation for Museum Websites, offers a thorough analysis of these criteria, which include:
1. user testing- with prototypes within the museum, monitor and observe interaction behaviour, use feedback. The user is usually given a set of tasks to carry out and is monitored. Can be monitored by video, think aloud protocol, observer’s notebook (problems include finding a typical user).
2. interviews- questionnaires about a prototype, an existing site, or ongoing feedback. Could be done via a kiosk in the museum.
3. log file analysis- looking at the data gathered automatically, includes such info as number of hits, type of platform, where the user came from, time spent on page and quite a lot more (problems include amount of data, relevance of data, interpretation, identification of unique visitors, caching) Then how do you extrapolate intention from info – why from what?
4. online questionnaires/feedback – the two major concerns are: self selection of sample; and response rate to draw reliable conclusions. Often there is a low response rate (between 2 and 5%)
5. Heuristic evaluation – doesn’t involve users, evaluates site against a set of agreed criteria or guidelines, research already available.
6. Procedures for ongoing maintenance and development must also be considered.

The article Audiences, Visitors, Users: Reconceptualising Users of Museum On-line Content and Services expands on these criteria to develop a more formal set of principles for web site development and evaluation. They put museum web site development into the context of the marketplace and come up with six design principles:
• acknowledge the marketplace web sites exist within
• have clear site goals, from the perspectives of the market, user, and organisation
• design in relation to these goals
• design an information space, not a museum replica
• test with target users at all stages, many times
• evaluate in terms of your goals, not just according to user demographic or products on site.

There are many examples in the literature (SFMOMA, Museum of Victoria) of organisations that take these basic criteria and offer case studies on how they apply them to their own redesigns, the challenges that emerged, and how effective the process was.
Some general points that come from these case studies are:
o Usability is important in developing an effective, interesting and user friendly website.
o There is a connection between website usage and attendance, however the audiences do not necessarily correlate completely.
o The different approaches – heuristic, user surveys, questionnaires, log file data – all contribute to a more complete picture of users and usability.
o How a user gets to the site is largely based on internet searches, rather than access through the home page.
o Museums have a very active role to play in developing their physical content for the online environment, and need to look at issues of intellectual property, copyright, etc.
o There is a lot of potential for museum sites – both directly and indirectly related to their physical role (see Hennes, Sherry Hsi)
o There are more novel methods of assessing user interaction for example the kiosk approach of the Exploratorium that can have a role in website evaluation.
o Web 2.0 – do the users want it? How do museums implement it in ways that do not threaten users, or waste resources? (see SFMOMA article, fresh and new blog re percentage of active users vs passive)
o How then do museums assess the use of this interactive content – data file analysis has not yet caught up with server side includes, active content, feeds etc.

Overall, for those embarking on web site redesign, or initial design, there is already a good body of research and practical examples to draw on for methods, ideas and examples. The challenge of web 2.0 and how to assess the role it plays in website assessment and audience research wil be another are for research and assessment over the next few years, as museums continue to take up the challenge of implementing web 2.0 concepts and ideas.