Posts tagged ‘MediaWiki’

Scaling up Software development for Wikimedia websites (Part II: Tools)

During the past few weeks, I have been thinking about a more structured way to manage software and product development within the Wikimedia community. The result is a list of ideas and recommendations I have compiled and submitted to the relevant staff members at the Wikimedia Foundation. I am also publishing them here in order to allow for a wider feedback. This article is the fourth and last of a series dedicated to this topic.

Disclaimer: The content of this article reflects only my personal opinion and is not an official plan or communication of the Wikimedia Foundation.

I have previously explained why the current setup of the Wikimedia bug tracker is not ideal. I have also advocated for a more managed & scientific software development strategy. This article aims to discuss an appropriate tool to support this strategy, and at the same time fix what is broken.

Software lifecycle & Project management

Right now, there is little project management of the technical activities at the Foundation. When someone does manage projects, they often use personal desktop applications that don’t allow collaborative work. There isn’t any real development roadmap or product requirements. If we want to be serious about structuring our activities, we need a project management strategy & the appropriate associated tools that allow us to manage things such as project scope, schedule, budget & financial resources, quality assurance, human resources, communications, risks & opportunities, procurement and coordination.

I asked around to see what the needs of the various members of the team were. Naoko Komura, who has been project-managing the Wikipedia Usability Initiative, and is now overseeing all UX programs, is particularly interested in the integration of the Project management platform with calendars & issue tracking. Operations staff members also said they were interested in Project management features such as time and task tracking.

I think it is way past time to have a more organized development process & software lifecycle management. The recent hiring of a new Chief Technical Officer is a step towards a more structured organization of technological activities. Of course, changing our bug tracker alone won’t automatically structure our activities, but it is a step in this direction.

Recommendation: Use tools that allow a better management of, and integration between, the different stages in our product(s) lifecycle.

Requirements

A few months ago, I asked Priyanka Dhanda (our new Code maintenance engineer) to explore open-source collaborative project management platforms that could be easily integrated with bugzilla (or that included a bug tracking system of their own). The Wikimedia tech community was invited to list requirements of such a tool, as well as possible solutions.

From the feedback we received, we can summarize what the required features for an issue tracker are: integration with our Version control system, being FLOSS, multiple projects support and ability to separate between “tasks” and other items such as bugs and feature requests. Additional, “nice to have” features include: ability to import data from existing Bugzilla instance and fined-grained e-mail subscription options

Similarly, the required features for a Project management tool are: being web-based to facilitate collaboration, multiple projects support, calendar/scheduling and roadmap, assignments & resource management and time tracking per task/user/project. Additional, “nice to have” features include: Gantt charts, public/private projects, fine-grained access to projects by user, basic accounting & budget management and requirements management.

Redmine

I haven’t found many Project management softwares that can be easily integrated with Bugzilla. However, I have discovered alternatives to Bugzilla that include project management features like those we’re looking for. Redmine seems to be a good project management software and provides an advanced issue tracking system as well. It supports multiple projects, public/private projects, calendars & Gantt chart, and a lot of other neat stuff. It also offers the ability to distinguish between features/improvements and bugs; this would be particularly useful to prioritize development efforts. We are now considering using Redmine as project management software and taking this opportunity to move our Bugzilla setup to Redmine. Initial research shows that a lot of people seem to praise Redmine compared to Bugzilla; there are migration scripts to import an existing Bugzilla setup into a new Redmine one.

Major projects such as TYPO3 are using Redmine. The software seems to benefit from a dynamic community of developers and it is also possible to sponsor custom development. There are many plug-ins to extend the default core features; popular plug-ins are usually included into the core software at some point.

Priyanka set up a local test instance of Redmine to let us play with it a bit; a public test platform was later made available for wider testing and the Wikimedia Tech community was invited to pitch in. So far, I personally like it and it fits my needs perfectly. Wider testing is now necessary to see if it fits the needs of the tech community.

Because the time I can devote to this change is limited, I haven’t reviewed other alternatives than Redmine, and don’t plan to unless another major alternative is suggested.

Recommendation: Move from Bugzilla to Redmine after careful preparation, especially regarding the organization of the platform.

Read also in this series

Scaling up Software development for Wikimedia websites (Part I: Human resources)

During the past few weeks, I have been thinking about a more structured way to manage software and product development within the Wikimedia community. The result is a list of ideas and recommendations I have compiled and submitted to the relevant staff members at the Wikimedia Foundation. I am also publishing them here in order to allow for a wider feedback. This article is the third of a series dedicated to this topic.

Disclaimer: The content of this article reflects only my personal opinion and is not an official plan or communication of the Wikimedia Foundation.

I am not going to give specific advice about how many developers the Wikimedia Foundation should hire: there are other people more knowledgeable about how many we need, and what they should work on. However, there are other key positions that I think are need to scale up.

Our human resources are currently focusing on what happens after the code has been written: we review it, we try to ensure quality, we try to automate testing, we file bugs, etc. However, there is little preparation before the development is actually done. This has led to a developer-driven design, resulting in an interface based on the implementation model. We need a more systematic approach to User experience and development management if we want to scale up properly.

Product management & Design

In the Software development world, Product managers and Designers are the most common bridges between users and engineers. Product managers identify the needs of users and prioritize features & improvements; their goal is to translate the users’ experience and feedback into explicit requirements to meet the users’ expectations. It is then the role of the designers to create innovative, well-thought solutions to address these issues and meet the requirements. Finally, developers provide feedback about the technical feasibility of these solutions, and implement them the best way possible. This is of course a simplistic summary, but it helps get the point across.

“Designer” can have a lot of different meanings. A lot of people think that “design” is just making things pretty. When I talk about designers, I think mainly of Interaction Designers, i.e. people who design solutions to improve the user experience and the way the product interacts with the user.

The community of MediaWiki users is amazingly large, partly because they include participants from all Wikimedia Websites. Similarly, MediaWiki also benefits from a large base of volunteer developers. Product managers and designers have been the missing piece in this picture; their bridging role is critical, simply because there aren’t any volunteers to take up this task. Admittedly, some users are also developers, and some developers keep themselves informed of the users’ wishes. But without product managers, clear requirements & prioritization are missing. And without designers, the technical solutions created by the developers don’t meet the users’ expectations and mental model.

In my experience, developers prefer to focus on the actual development and rarely enjoy meta-activities related to it. They usually dislike project management and writing product specifications, and rightly so: it is not their job. However, a successful software product strategy cannot rely only on development. We benefit from a large community of volunteer developers, but we lack management & design resources; it is the role of the Foundation to supplement this lack. It doesn’t mean we shouldn’t expand our development team: our number of paid core developers is ridiculously small. It only means we should also invest where the weakness lies.

Recommendation: Recruit Product managers and Designers to strengthen the development cycle of our technological platform

Research team

The Multimedia usability project has relied heavily on initial research in order to gather as much information as possible about users and their goals. A lot of useful information was already available, but a lot of specific metrics were also missing; collecting and analyzing them took a lot of time and it will still take months to get all the metrics we need. Research is critical in order to make the right decisions, especially about design. Research is the only way know our users in order to make the best design & management decisions. Good research is the best basis on which product managers, designers and developers can then respectively specify, design and build awesome solutions.

Right now the only researcher we have is Erik Zachte as Data analyst, but his job seems to essentially focus on providing operational metrics. While this work is much needed, we also need some more specific data on a case by case basis. I see at least three other positions needed:

  • A UX research specialist, who would conduct regular in-house low-cost usability testing, and generally manage UX studies
  • A Metrics engineer, who would develop integrated metrics in the software and be able to extract specific information from the database on a case by case basis.
  • A Community specialist, with a good knowledge of social psychology and online interaction, who would especially focus on improving the interaction between participants by identifying the community issues and proposing ways to fix them.

Recommendation: Build a Research team to guide design & strategic decisions about our technological platform

Volunteer developers

We benefit from a fantastic community of volunteer developers, but we underestimate their potential; I think we are not doing enough to support their work and engage them into our activities. In 2007, the Foundation hired Cary Bass to try and coordinate the large pool of volunteers willing to help us with meta activities. Similarly, we need a Dev community manager to care for our volunteer developers. We need someone who knows the developer community very well, and knows their strengths and weaknesses in order to find the right person for each job. We need someone who can help orient new volunteers, organize real-life meet-ups and manage projects such as the Google Summer of Code.

Recommendation: Recruit a Community manager to coordinate the efforts of volunteer developers.

Read also in this series

Wikimedia & MediaWiki bugs, issues and requests

During the past few weeks, I have been thinking about a more structured way to manage software and product development within the Wikimedia community. The result is a list of ideas and recommendations I have compiled and submitted to the relevant staff members at the Wikimedia Foundation. I am also publishing them here in order to allow for a wider feedback. This article is the first of a series dedicated to this topic.

Disclaimer: The content of this article reflects only my personal opinion and is not an official plan or communication of the Wikimedia Foundation.

Bugs & Bugzilla

Right now, the bug tracker we use is based on Bugzilla and located at bugzilla.wikimedia.org. Many major free projects use a generic “bugs” or “issues” prefix or suffix in their URL: bugs.kde.org, bugs.gentoo.org, issues.apache.org, www.debian.org/Bugs. Some projects use the “bugzilla” prefix like we currently do, like bugzilla.gnome.org. The latter is an example of a choice based on the implementation model: the name reflects the technical implementation of the bug tracker, not its actual purpose. A better name would be closer to the user model and describe the actual goal of the platform: to report and manage bugs and issues related to a specific project. If we do change our tracker, the current name will have to change too, because it is specific to a given tool.

Recommendation: Use a generic descriptive prefix rather than one based on the tool we use.

Wikimedia & MediaWiki

Another current issue is the confusion caused by the similar names used for the organization (Wikimedia) and the software (MediaWiki). A good example of this confusion is the number of MediaWiki users who join the #wikimedia IRC channel instead of #mediawiki to ask for software support. The confusion is even worsened by the fact that we have a unique bug tracker located at bugzilla.wikimedia.org, dealing with issues related to both Wikimedia websites and the MediaWiki software.

There are obviously strong ties between Wikimedia projects and MediaWiki: all Wikimedia projects use the MediaWiki software, and the MediaWiki software is primarily developed with Wikimedia projects in mind. However, there is also a growing community of MediaWiki users who are not Wikimedia users and we should provide them with tools relevant to them. This might be for instance a support forum dedicated to MediaWiki users.

Wikimedia projects and MediaWiki are separate products and they should be acknowledged as such: as a consequence, the separation between bugs in the MediaWiki software, and Wikimedia-specific operations & configuration requests should be made more explicit. Obviously, we would prefer to have a unique back-end to support both sites, particularly to be able to move bugs and requests from one platform to another, but this is easily configurable. Possible names could be dev.mediawiki.org and tech.wikimedia.org; both are currently unused. They are pretty wide prefixes, because we may host a real project management platform there, rather than just bug trackers.

Recommendation: Offer two different public-facing platforms for MediaWiki- and Wikimedia-related issue tracking.

Read also in this series

Ten features that would dramatically improve Wikimedia Commons

Logo of Wikimedia Commons

Logo of Wikimedia Commons

About two years ago, I said “Commons may be the next coolest project, as soon as developers find the time to improve its usability to make it more user-friendly”. Sadly, Wikimedia Commons hasn’t evolved much in terms of usability since then. MIT’s Technology Review recently published an article about improvements to come regarding the management of video content on Wikipedia and Wikimedia websites. I heard a lot of people say: “Good, but what about pictures?” Some technical improvements described by the Technology Review will be useful for both images and videos, such as the media and upload wizard currently developed by Michael Dale. However, Wikimedia Commons still needs many little (or big) features that would dramatically improve its user-friendliness.

Browsing & reusing

  1. Automatic localization: Websites such as Wikimedia Commons and meta-wiki host content in various languages and have a multilingual audience. These multilingual wikis should automagically detect the locale of the user’s browser and use it as language of the interface, especially for unregistered users. As for users with an account, their browser’s locale should be set as the default language in their preferences.
  2. Usage-centric page layout: It’s all very nice to know that such image is a “retouched picture” or that such diagram was “made using Inkscape”. But I think what most of the users want to know is: how to use the picture (in Wikimedia projects or elsewhere) and how to download it (using the best resolution available). Many people use the right-click-save-as method to save pictures from the Internet. If they do that on Commons, they will only save the low-resolution preview. There should be a big button « Download high-res », as well as snippets of code to embed a file with proper attribution.

Metadata

Full metadata support is the cornerstone of many other features. EXIF is probably the most known type of metadata, but there are also others such as IPTC or XMP.

  1. Pull metadata from files on upload: this idea is not a new one, yet it hasn’t been implemented. A fair amount of photographers add a lot of metadata to their files: author, description, copyright information, geotags, keywords, etc. and it is extremely cumbersome to have to redo all the work by hand during the upload.
  2. Store metadata in a database to make search and attribution easier, especially: description, license, media type (photo, diagram, map, etc.). It should be connected to the MediaWiki API to allow for easy extraction of these data.
  3. Push metadata to files on download: In the field of publishing, storing credit information directly into the file’s metadata is strongly recommended and is a standard practice to avoid losing track of it.

Related open bugs

  • bugzilla:6672: EXIF orientation not used (rotation from digital cameras)
  • bugzilla:3361: Image author, description, and copyright data saved in EXIF fields
  • bugzilla:16956: Show IPTC metadata on image description page
  • bugzilla:657: Pull copyright metadata from files on upload
  • bugzilla:11484: Include ISO rating in abbreviated exif metadata.

Editing

  1. Built-in basic editing features (lossless rotate, crop) and ability to save under another name (i.e. for crops). Similarly, a built-in geocoding feature using OpenStreetMap. Geocoding images means attaching geographic information about the place where the work was made. This may be made easier by the current initiative to integrate OpenStreetMap with Wikimedia projects. And of course it should save the coordinates as metadata.

Rating

  1. Some sort of community-managed rating feature; as someone said elsewhere, “Commons is a depository, and depositories are expected to host lots of junk”. A rating feature would allow the best of Commons to be presented first during the search, and junk to be presented last.

Searching

With currently more than 4.6 million files (and counting), it is becoming increasingly important to improve the search features of Wikimedia Commons.

  1. An “advanced search” feature similar to flickr’s. It should be possible to search by media type, by license, and to add toggles such as “safe mode” (explicit content) or “personality rights”.
  2. Multilingual search: Files on Commons are ordered in hierarchical categories, using English as lingua franca. If you want to find a file, you have to search in English. I imagine it is possible to use some dictionary (coupled to the language detection) to give good results for a search in any language.
  3. Google-Images-friendliness. A lot of people use Google Images to find pictures, but images from Wikimedia Commons rarely appear in these results (unless they are used on a Wikipedia page).

Note: All these ideas are given from a user point of view; their technical feasibility has to be assessed by a MediaWiki-literate developer.