Editor Engagement/2012 strategy planning (Features)
This is a strategy planning space for the Editor Engagement Features team for the 2012-2013 fiscal year (ending June 30, 2013).
Our current focus is on the third quarter of the Wikimedia Foundation's 2012-2013 fiscal year (January-March 2013).
Goals
[edit]
Our overall goal for 2012-2013 is to help reverse Wikipedia's editor decline by building features that can help recruit new editors and retain existing editors.
To that end, we are working on these key quarterly milestones for January-March 2013:
- Expand first beta version on MediaWiki.org, with more features and notifications
- Deploy a limited release of Echo on the English Wikipedia by the end of the quarter
- Create a design spec and product plan for user-to-user messaging on talk pages
- Discuss designs and plans with the community through a variety of channels
- Complete and test final features, code refactoring and database cluster
- Prepare for deployment on multiple projects, once approved by their communities
An overall timeline for E2 feature development in 2013 is shown on the right, for discussion purposes. This timeline is approximate and subject to change, as our plans get periodically adjusted.
Each project's roadmap is provided with more details in the sections below, as well as in these slides.
Project Roadmaps
[edit]Echo (Notifications)
[edit]
This quarter (January-March 2013)
- focus on new users (but make it usable by experienced editors)
- complete core features (flyout, "archive", prefs, job queue, bundling, metrics, HTML email)
- add a few more notifications ("happy path" for new users, useful notices for power users)
- deploy limited release on en-wiki (different defaults for new and experienced users)
- fix bugs + critical feature tweaks (as needed, prioritized by urgency)
- provide in-house developer support (through hooks, i18n and dev guidelines)
- estimate maintenance + i18n support (2013-2014 plan)
Next quarter (April-June 2013, if time and resources allow)
- add even more notifications (based on community feedback and testing)
- develop a few more features (e.g. relative timestamp, 'mark as unread')
- get started on api framework (for 3rd party support outside mediawiki)
- explore cross-wiki notifications (limited notices from commons/meta?)
- discuss custom localization (on other language wiki, as needed)
See our feature requirements or test page for more information about our work in progress.
Flow (Messaging)
[edit]
This quarter (January-March 2013)
- focus on user-to-user messaging (improving user talk pages)
- create first designs and prototypes (for most important use cases)
- first user tests and interviews (with new and experienced users)
- start community outreach (on en-wiki, Teahouse and IRC chats)
- prepare design spec + product plan (to guide development next quarter)
Next quarter (April-June 2013)
- build rapid testing framework (to quickly test UI designs)
- start front-end development (core messaging features)
- build back-end infrastructure (to support large messaging database)
- deploy first release (on mediawiki.org)
See our project page for more information about our work in progress.
Article Feedback
[edit]
This quarter (January-March 2013)
- complete final features (simpler moderation tools, better filters, auto-archive)
- complete code refactoring (now under review, including database tools)
- support community discussions (about release on English, French or German Wikipedia)
- deploy and test new code (on prototype and other sites, based on community votes)
- deploy new database cluster (as needed, to support millions of posts)
- fix bugs + critical feature tweaks (as needed, prioritized by urgency)
Next quarter (April-June 2013)
- prepare for international releases (e.g. French and/or German Wikipedia)
- deploy and promote final version (in collaboration with community for each project)
- discuss and support other deployments (with a range of interested projects)
See our features under consideration for more information about our work in progress.
Research
[edit]Some of these research metrics and findings helped inform our plans for this quarter.
Funnel
[edit]
Here are some rough estimates of our monthly user funnel for the English Wikipedia:
- Readers: ~300M
- New accounts: ~150K
- New editors (10+ edits): 6K
- Active editors (5+ edits/month): 32K
- Very active editors (100+ edits/month): 3.2K
While these are just estimates from a range of different sources, they give us a sense of scale. For example, there seems to be on the order of 9,000 readers for each active editor on the English Wikipedia. And only about one in 25 new members who register go on to become auto-confirmed editors (with 10+ edits).
These examples suggest a large untapped potential for editor engagement, in three distinct groups: readers, new members and new editors.
(Source for editor counts: Wikimedia Report Card for October 2012)
Editor Decline
[edit]We’re currently seeing a 0.5% y-o-y decline in active editors for all projects combined from Oct. '11 to Oct. '12. We're also observing a larger 5% y-o-y decline in active editors for the English Wikipedia for the same time period (34.3K in Oct. '11 vs. 32.5k Oct. '12), according to the Wikimedia Report Card for October 2012. We understand that these numbers are not entirely reliable, but they give us a sense of scope about the current decline, which most observers agree has gone on for a while, though it was more pronounced a few years ago, and has flattened somewhat in recent years.
As of October 2012, WMF reported 80,000 active editors for all projects, of which 8,000 were for Commons, which could mean a net 72,000 active editors for all projects other than Commons. This suggests a gap between this report and the WMF's target of 86,000 active editors by July 1, 2013, but part of this gap is due to changes in metrics methodology, as explained above.
Reversing this decline will take sustained effort on multiple fronts, from editor engagement features and experiments to visual editor, internationalization and site performance improvements.
(Source for editor counts: Wikimedia Report Card for October 2012)
New Members
[edit]
One of our best opportunities for engaging new users is by inviting them to first create an account, so we can effectively communicate with them through notifications and other tools. And we are signing up on the order of 125-150k new users per month on the English Wikipedia alone, which is significant.
Yet our research suggests that about 65% of the new users who register end up not making any edits after they sign up.
It seems that a few well-designed notifications could invite these new users to contribute and could help convert them into editors. Otherwise, notifications would have almost no impact at all on these new members, which would be unfortunate.
To address this finding, we recommend a discussion of how to make the watchlist (and some of the "notifications" inherent in watchlist) more visible to new users. At a minimum, we want to let users know that the watchlist exists and potentially make some preference changes to make watchlists more consumable. Ideas worth considering include informing new users on how to use the watchlist, and/or automatically add pages that you start or edit to your watchlist.
Examples of such invitations could include a reminder that they can edit, an invitation to check their watchlist, suggestions of articles to improve, tips on how to edit, etc.
(Source for new account activity: Dario Taraborelli's daily registration dashboard for English Wikipedia.)
First Edits
[edit]
Last year, Maryana Pinchak did a study of first edits for 100 users, to better understand how we can support them as they start their journey as Wikipedia contributors. This led us to map out that journey on this board, which also shows what types of contributions they made after their 100th edit.
Here's a summary of their first edits:
- Edit an article: 74%
- Create a page: 16%
- Create user space: 7%
- Comment on talk page: 4%
Some of this behavioral data can help inform our work on notifications, given our focus on new editors for the Echo project. We want to make sure that the notifications we send are mapping well with these users' first steps.
For example, we now only offer edit notifications if a user's edit is reverted, and currently do not send any positive or neutral notifications for any other events that relate to that edit. And the edit reversion notification now sends these new users to the diff page, which is very confusing. Both of these issues would seem like a very high priority for our work this quarter, in order to support these new users during their critical first steps.
We recommend providing a "happy path" for new editors over time, using one or more positive notifications once they have completed an unreverted edit. The rationale for this is that oftentimes, this user receives no response from the community and it would be good to introduce some positive feedback. The assumption here is that under some definition of unreverted, these users are well-intentioned and could become constructive editors.
Some of the other details in this research can help us determine when to send follow-up notifications to these new users.
(Source for first edits numbers: Maryana Pinchuk's handcoding study of new editors)
Feedback Registrations
[edit]Our latest research suggests that Article Feedback appears effective in getting new users to contribute to Wikipedia, as summarized on the WMF blog (see full slides).
For example, 2.7 percent of readers who post feedback go on to create a new account, after being invited to sign up. And 3 percent of these new users go on to edit articles within 24 hours from signing up.
At this rate, we project that Article Feedback can generate several hundred thousand new registrations per year on the English Wikipedia, which is significant. While we cannot accurately estimate how many of these new contributors would become active editors (5 edits/month), it appears that this program can have some impact in reversing the current editor decline, if combined effectively with other engagement programs such as notifications.
Feedback Moderation
[edit]Our research suggests that about 39% of the reader feedback is found useful by editors, but it is buried under a lot of noise, which requires some form of moderation and/or filtering. But editor moderation activity is quite low when compared to the volume of feedback posted every day. Less than 10% of feedback posted every single day receives any kind of moderation within a month, whether by readers or registered editors.
These issues could be addressed by better moderation tools and filters, both automated and manual. And many editors can’t easily find comments for articles they edit, an issue which could be addressed by making feedback more visible to editors on the article pages.
(Source for feedback usefulness estimate: Dario Taraborelli and Aaron Halfaker's feedback evaluation report.
Messaging
[edit]We don't currently have any new research to share about Flow, our user-to-user messaging initiative, since this project is still in early design phase.
As stated in our Engineering Goals for 2012-13, the current talkpages used by Wikipedia editors to communicate have many serious problems. Among the issues:
- there are unclear social conventions of how to respond
- there is no single canonical place for this interaction to live
- the concept of a conversation is vague (both from a user and technical perspective), notifications (see above) to all parties are crude/non-existent.
- the UI does not represent anything resembling a modern discussion system (does not resemble any existing web affordance).
- there is no consistent or relatable framework for surfacing a new messaging event (update) to the user (in the form of a notification)
As messaging actually covers a wealth of user-user interactions and requires a notification system to close the loop, the primary focus for this 2012-13 fiscal year is only on a single user-user "talk page"-like discussion, to be spec'd out in a "mobile-first" design manner, in order to create realistic constraints on the complexity of the problem.