The observation room at the testing facility; the testing is happening in the background, behind the semi-transparent glass (CC-by-nc by Neil Kandalgaonkar).

Getting there

We reached out to some UX firms and published a Call for proposals in February. Several firms submitted proposals; after serious consideration, we chose to work with gotomedia, a San Francisco-based firm that seemed to align best with our goals & values.

The study was planned to take place in March, but was postponed because the prototype was not ready. In the meantime, we asked some of our co-workers to test it in order to uncover the most obvious flaws & bugs.

Goals & testing conditions

A few weeks ago, the actual testing eventually took place. We tested ten users: five locally in San Francisco, and five remotely within the US. We considered conducting similar testing abroad, in order to identify language-specific issues; but in the end, it turned out that we wouldn’t learn a lot by simply replicating the same test script.

Multilingualism on Commons (and Wikimedia websites generally) is a huge piece of work that deserves dedicated efforts, and dedicated UX studies. The main reason for which we decided to hold the testing halfway through the project, and not at its very beginning, was that we could test both the current upload interface, and our prototype.

On the one hand, during our preliminary research phase, we identified a large number of issues with the current interface; but we still needed to formally record the user experience and validate our preliminary conclusions. On the other hand, we wanted to do a reality-check with our prototype, to see if the direction we had chosen was appropriate, and to identify areas of improvement.

Highlight videos

The testing sessions went pretty smoothly. The gotomedia folks did a fantastic job at preparing the “highlight videos” in time for our conferences in Gdańsk (WikiSym & Wikimania). The audiences really liked them, although we didn’t have time to show all of them.

Highlight videos are edited summaries of the main findings of the study. In our case, we have three highlight videos: one about the testing of the current interface on Commons, one about the testing of the prototype, and the last one about how we could improve the prototype.

Long story short: the current interface is a nightmare, and the prototype is way better, even if there are some minor things to improve. The good news is, all the items to improve were already planned features at the time of testing, and they have either already been added, or will be before the upload wizard is released.

Namely, one of the main remaining issues is the fact that users don’t really understand copyright and free licenses. That’s why we’ve been working on a licensing tutorial at the same time, to be released jointly with the new upload wizard.

See for yourself

The highlight videos are now available on Wikimedia Commons; per our agreement with gotomedia, all the videos were released under the Creative Commons Attribution – Share alike 3.0 license.

In the tradition of Wikipedia’s Neutral point of view policy, we’ll try to upload the unedited videos to Commons as well, in order to let the community draw their own conclusions.

If you would like to draw our attention to things we’ve missed, or even edit your own highlight videos yourself, you are warmly invited to do so. You can watch the highlight videos below (if it works) or on Commons. The links to Commons are available below if you want to download the video files on your computer.

Your feedback and comments are much welcome.

Current interface highlight video

Prototype highlight video

Room for improvement highlight video

Files

• Current interface testing: File page on Commons – Download OGV file (4m11s, 29.89 MB)
• Prototype testing: File page on Commons – Download OGV file (5m32s, 35.43 MB)
• Room for improvement: File page on Commons – Download OGV file (3m51s, 23.02 MB)

As a consequence, I will be available on IRC to answer questions related to the Multimedia usability project this Thursday, February 4, 2010 @ 1700 UTC. You can use the fixed time world clock to check the time in your timezone. Please join us in #wikimedia-office on Freenode using your favorite IRC client; you may also use web-based clients (check out the IRC office hours page on meta-wiki for more information on how to do that).

Looking forward to your questions.

Golden Gate Bridge, Rodefeld, CC-by

The last couple of weeks have been quite busy for me. I moved from Toulouse, France, to San Francisco, California as a consequence of my hiring by the Wikimedia Foundation (the non-profit that runs Wikipedia) where I work as Product Manager for Multimedia Usability.

I just got my Internet connection at home set up, so you should see more regular blog articles now. I have a few drafts in the works and a lot of updates to post related to my work at the Foundation. In the meantime, I invite you to read the summary published today on the Wikimedia blog.

I would like to ask for your help to accomplish this research. It can take a lot of time if only one person is doing it. On the contrary, if ten or twenty people step in and all do a small part, we can collect helpful data very quickly. Besides, it is always better to have several people with different perspectives look at the data we collect, in order to better see the “big picture”.

Your help is crucial in order to move quickly towards the requirements definition phase. I have already prepared a list of websites and a few questions we are asking ourselves; they should facilitate the collection of data that can then be used directly by the team.

Please join us and make your contribution to the Domain research page on the Usability wiki.

I also follow another set of RSS feeds related to User experience (UX), Interaction design (IxD) and Usability in general. Until now, I have been reading them for my own benefit; but with my new job, it makes sense to pick a few interesting pieces of information for Wikimedians who want to better understand the work of the Wikimedia usability team(s). As a consequence, I will try to maintain a “UX & IxD newsletter” on this weblog, starting with this one.

Japanese Cast Iron Tea Kettle. Friends of Donald Norman will understand. (CC-by by Steven Depolo)

User research

What is the point of user experience research? It may seem obvious to any designer, but it is harder to explain to clients or, in my case, to the Wikimedia community. People who are not familiar with interaction design and product development in general often have a hard time understanding why it is critical to “lose” time in research (it is really “invest”) at the early stages, even when the course of action looks so obvious. David Sherwin provides a “cheat sheet” to explain the value of user experience research in plain English.

• Can You Say That in English? Explaining User Experience Research to Clients, David Sherwin, A List Apart, 3 Nov. 2009.

What are the benefits of using personas in product design? Personas are fictional model users based on behavioral patterns and goals of real users that we have studied. More than just stereotypes with a stock photograph stuck on a board, they are very much like other scientific models based on experimental data. As a trained scientist and a follower of the Cooper methodology, I make an intensive use of personas for my work on the Wikimedia Multimedia Usability project. Despite their broad use in design teams, few studies have tried to assess the actual effectiveness of personas; Frank Long has now published such a study.

• Real or Imaginary: The effectiveness of using personas in product design, Frank Long, Irish Ergonomics Review, Proceedings of the IES Conference 2009, Dublin, 20 Nov. 2009.

Design principles

Let users explore and discover your website. There is a trap MediaWiki developers easily fall into: the interface of MediaWiki (and, as a consequence, the one you see on Wikipedia and Wikimedia Commons) is cluttered by dozens of unnecessary links and verbose descriptions. On the other hand, the software is so complex that a lot of features remain hidden even to established participants. What we need is a simpler interface that provides the relevant links and hints when appropriate, and at the same time empowers and encourages users to be bold and explore the interface. Amber Simmons provides a few pieces of advice on how to improve discoverability in order to make websites more explorable.

• You Can Get There From Here: Websites for Learners, Amber Simmons, A List Apart, 3 Nov. 2009.

Product implementation

A babelfish for designers and developers. In the world of software and website development, it is not uncommon to find designers and developers working together. This is for instance the case with the Multimedia Usability project, where the core team is comprised of two people: me and a software developer. However, communication between designers and developers is not always easy, because of their different backgrounds and perspectives; it could be compared to chatting in a foreign language. This is something I have also experienced during my previous work as an interdisciplinary researcher: I was a physicist and microtechnologist working closely with chemists and biologists. In her latest article, Theresa Neil provides some good advice in order to facilitate the communication and collaboration between designers and developers.

• Designers vs Developers: Coming together to build the best RIAs, Theresa Neil, Designing Web interfaces, 10 Nov. 2009.