Over the past few months, I’ve been coordinating the preparation of a formal User experience (UX) study for the Multimedia usability project. Basically, it means observing how “real” users interact with the Wikimedia Commons in order to improve it. Videos of the testing have now been published in order to share them with the community.
The observation room at the testing facility; the testing is happening in the background, behind the semi-transparent glass (CC-by-nc by Neil Kandalgaonkar).
Getting there
We reached out to some UX firms and published a Call for proposals in February. Several firms submitted proposals; after serious consideration, we chose to work with gotomedia, a San Francisco-based firm that seemed to align best with our goals & values.
The study was planned to take place in March, but was postponed because the prototype was not ready. In the meantime, we asked some of our co-workers to test it in order to uncover the most obvious flaws & bugs.
Goals & testing conditions
A few weeks ago, the actual testing eventually took place. We tested ten users: five locally in San Francisco, and five remotely within the US. We considered conducting similar testing abroad, in order to identify language-specific issues; but in the end, it turned out that we wouldn’t learn a lot by simply replicating the same test script.
Multilingualism on Commons (and Wikimedia websites generally) is a huge piece of work that deserves dedicated efforts, and dedicated UX studies. The main reason for which we decided to hold the testing halfway through the project, and not at its very beginning, was that we could test both the current upload interface, and our prototype.
On the one hand, during our preliminary research phase, we identified a large number of issues with the current interface; but we still needed to formally record the user experience and validate our preliminary conclusions. On the other hand, we wanted to do a reality-check with our prototype, to see if the direction we had chosen was appropriate, and to identify areas of improvement.
Highlight videos
The testing sessions went pretty smoothly. The gotomedia folks did a fantastic job at preparing the “highlight videos” in time for our conferences in Gdańsk (WikiSym & Wikimania). The audiences really liked them, although we didn’t have time to show all of them.
Highlight videos are edited summaries of the main findings of the study. In our case, we have three highlight videos: one about the testing of the current interface on Commons, one about the testing of the prototype, and the last one about how we could improve the prototype.
Long story short: the current interface is a nightmare, and the prototype is way better, even if there are some minor things to improve. The good news is, all the items to improve were already planned features at the time of testing, and they have either already been added, or will be before the upload wizard is released.
Namely, one of the main remaining issues is the fact that users don’t really understand copyright and free licenses. That’s why we’ve been working on a licensing tutorial at the same time, to be released jointly with the new upload wizard.
See for yourself
The highlight videos are now available on Wikimedia Commons; per our agreement with gotomedia, all the videos were released under the Creative Commons Attribution – Share alike 3.0 license.
In the tradition of Wikipedia’s Neutral point of view policy, we’ll try to upload the unedited videos to Commons as well, in order to let the community draw their own conclusions.
If you would like to draw our attention to things we’ve missed, or even edit your own highlight videos yourself, you are warmly invited to do so. You can watch the highlight videos below (if it works) or on Commons. The links to Commons are available below if you want to download the video files on your computer.
Your feedback and comments are much welcome.
Current interface highlight video
Prototype highlight video
Room for improvement highlight video
Files
- Current interface testing: File page on Commons – Download OGV file (4m11s, 29.89 MB)
- Prototype testing: File page on Commons – Download OGV file (5m32s, 35.43 MB)
- Room for improvement: File page on Commons – Download OGV file (3m51s, 23.02 MB)
Photo gallery
unfoldscience: scientific communication
Journal en français
Hey, this embedded video player looks and works really well! HTML 5 stuff is getting better very rapidly; last time I remember testing one, it wasn’t nearly this good! Are these videos straight from Commons, or hosted on your server? What plugin or code do you use for embedding the videos?
(And great post, by the way. I’m really excited about the new uploader.)
I think this is the default embeddable player you get nowadays on the English Wikipedia and it has built in embedding support – it doesn’t seem to have full screen support though… For some reason this player is not available on Commons, which is strange and confusing.
Sage, the videos are embedded straight from Commons. I’m using Michael Dale’s MwEmbed thingie. It is still very experimental, though, and I didn’t really want to use it in the first place because it relies on unstable assets such as http://prototype.wikimedia.org .
Bence is right that it’s not enabled by default on Commons; you have to use a special trick to make the new player show, and it has indeed an integrated embedding support, as well as closed captioning. To enable it for a specific video, you have to add a special call at the end of the URL, for example http://commons.wikimedia.org/wiki/File:Multimedia_usability_project_2010_-_Prototype_testing.ogv?withJS=MediaWiki:MwEmbed.js
There are some complications with WordPress though, namely some quirks with the Wysiwyg preview. The code snippet has to be inserted in HTML view, but WordPress doesn’t handle it properly. When you switch to Wysiwyg view, it surrounds it with additional code which makes it not work, unless you save the article from the code view and never again open the Wysiwyg version. Quite inconvenient! I’ve talked to Michael about it, and he said they may devote some resources to fix this, possibly by creating a plugin to port the Add-media-wizard to WordPress.
Bence, I don’t think the new player is enabled by default on the English Wikipedia either; see for example this video, which still uses the default OggHandler player developed by Tim Starling. The new player is available as a gadget in the user preferences, though, as part of MwEmbed, both on the English Wikipedia and on Commons.
I can completely understand the rationale for not wanting to enable it by default considering it’s still experimental. But at some point in the future, we’ll have all these nifty features available in MediaWiki or extensions, and Commons will just rock :)
You’re right, I had the mwembed thingy enabled as a gadget, though no idea what it was for – apparently this is responsible for the “new” player that I have started seeing suddenly which has some quite nice features.
(The native browser video support is also quite acceptable nowadays , so a default embed code might be shown on the video’s description page – and for that matter, maybe even on the image pages there could be a “share/use” button.)