Analytics/Wikimetrics/Stories

------ DONE ------

These stories have been implemented and are live on https://metrics.wmflabs.org/

User creates cohort with CentralAuth insertions

Done - Show story

Story: As a Wikimetrics user, I want to create a new cohort that is expanded to include wiki users on every wiki they have been active on, so that I can then create reports that include activity across wikis.; Event occurs after user clicks "Upload a new cohort" on the "My cohorts" page, they then have an option (checkbox) to expand the cohort; Activity on other wikis is given by CentralAuth; Bug ID 66843; Commitment to develop in sprint: http://sb.wmflabs.org/t/analytics-developers/2014-10-16/; If a wiki user in a cohort logs into a previously unvisited wiki after the cohort was created, the cohort will not be updated dynamically to include this wiki user on this newly visited wiki.

UX: Option is presented only when user is about to upload a new cohort; checkbox text is: "Expand cohort to all user’s accounts across projects (via CentralAuth)"

Tag your own cohort with existing tags

Done - Show story

Story: As a WikiMetrics user, I want to manage my cohorts by selecting one or more tags from a list of standard tags.; Event occurs when user begins typing the name of a tag into the "add tag" input box on the page for one of their own cohorts. A list appears with existing tags, typing filters list to existing tag list, and selecting a suggested tag auto-completes it in the input box (like google).

Notes

Tag List: Analytics/Wikimetrics/Standard_tags
See also Analytics/Wikimetrics/Adding_New_Features/Tag_Cohorts
review-remove existing cannonical tags which may not conform to tag list above
add a link to wikipage with default tags (e.g. /analytics/wikimetrics/default_tags)

UX change

It is redundant for a user to tag a cohort with it's project (e.g. frwiki) because the default project was already specified at the cohort creation time. So rather than adding each project as a standard tag, we can update the UI to expose the default project specified at upload time.
Also, typing in a project in the search should filter the cohorts with that default project. For cohorts that have users from multiple projects (including CentralAuth cohorts), all the projects should be listed.

Deleting a username from a cohort

Done - Show story

Story: As a Wikimetrics user, I want to be able to delete one or more usernames/user_ids from my cohort because they are invalid or because the user has opted out of tracking, without re-uploading my cohort file.; User searches for a username in a search field, and deletes name from that cohort by clicking "remove" next to the name returned by the search.; User has the option to search through all names in the cohort, or just invalid names.

Notes

Status: Some development work has been done. Wireframes have been approved. Task needs to be re-estimated
Task: https://phabricator.wikimedia.org/T75350
Possible issues: with centralauth, you may see multiples. That could be cumbersome. Suggest collapsing to a single username, to make it easier/quicker to delete.
See also [1], [2]

Wikimetrics admin reporting

Done - Show story

Story: Grantmaking staffer has means to run some basic reporting on the usage of wikimetrics. Creating reporting tools is out of scope. Access to the database will be sufficient.

Notes

Analytics will create read-only user account for Wikimetrics database for Grantmaking Learning Strategist
Grantmaking Learning Strategist will create reporting script that runs as a weekly/monthly cron job.
Analytics will update Wikimetrics interface to link to WMF Labs terms of use: https://wikitech.wikimedia.org/wiki/Wikitech:Labs_Terms_of_use
data needed for next Grantmaking QR (end of December 2014)
Phabricator tasks: https://phabricator.wikimedia.org/T76106

------ HIGHEST PRIORITY -----

Stories below are up next to implement. Scheduled stories will be added to the latest sprint at http://sb.wmflabs.org/t/analytics-developers/.

View your tagged cohorts

Story: As a Wikimetrics user, I want to browse all of my cohorts with a particular tag.; List of tagged cohorts is displayed when user clicks on a tag on the page for one of their own cohorts, AND when user clicks on a tag in a list of all tags.

Notes

need a UI for page that displays lists of all cohorts with a given tag, list of all tags
for all tags page, we should visually separate system-defined vs. user-defined tags
related task on Phabricator: https://phabricator.wikimedia.org/T75071

Example content on a page:

Tag Name	Tagged Cohorts
gp:Editing-Workshop	Workshop-A Workshop-B Workshop-C
gp:Edit-a-thon	Workshop-C
this-is-a-test	test-2014-13-12-A test-2014-13-12-B
not tagged

------ NEXT TO SPEC OUT ------

Stories below are the next high priority to be discussed and defined.

------ IDEATION ------

Stories below are not fully scoped or defined.

List usernames in reports that include individual results

Story: When a user download date from a report that contains individual results in JSON, the file contains both user names and user ids

Notes

currently, these data files only contain user ids, even if the cohort was created with user names

Report cohort edits across wikis using CentralAuth Extension

Stories: Cohort creator configures a report to return results for cohort users' activity across all projects where those users made edits.; This is different from Story "User creates cohort with CentralAuth insertions" because it happens during the report generation. The cohort is expanded at that time.

Notes

should be able to report both aggregated metrics per project (sum all users) and metrics for individual users per project
querying against central auth at reporting time allows a 'refresh' of the cohort by adding new projects that cohort members may have become active on in the intervening weeks or months since the cohort was created
see also Analytics/Wikimetrics/Adding_New_Features/CentralAuth_Cohorts

Wikimetrics superuser can view and change other users' datasets

Story: Grantmaking officer has special userright allowing them to perform any action on a Wikimetrics users' cohorts that the user themselves can perform: tag, untag, delete cohort, delete user from cohort, run reports.

Notes

may not need to have a admin interface initially: admin can use spreadsheet report as reference for finding cohorts. See #Wikimetrics admin reporting

View cohort report output in Vital Signs dashboard

Story: WikiMetrics user can run a report and then easily create a Vital Signs dashboard that shows Global Metrics by that cohort across time and across projects.

Notes

is the dashboard available via a persistent URL?
is the dashboard publicly available?
can the dashboard page be a) refreshed when report is re-run, and/or b) set to refresh automatically on a scheduled basis ("b" requires scheduled reporting... a separate story)
how does a user delete a dashboard they created?

View metric definitions on metrics page

Story: As a Wikimetrics user, I want to be able to read about individual standard metrics when setting up my report so I can choose the right ones and interpret my results correctly.

Notes

link to the wiki pages for each metric, rather than hard-coding the definition into the WikiMetrics interface.
the best place for links to the wiki pages is probably the individual metrics pages.

View cohort members

Story: As a Wikimetrics user, I want to view my cohort at some arbitrary point after upload so I can check the user names/ids that are included in that cohort.

Notes

View reports run on a cohort

Story: As a Wikimetrics user, I want to view a list of the reports that I have run on a particular cohort so that I don't run the same report multiple times by mistake.

Notes

in the "My reports" screen, the cohort is listed along with the metric used in the report and the date the report was run. Reports-per-cohort could be displayed in a similar fashion.

------ HOLDING ------

Stories/features that have been scoped, but we decided not to implement yet.

User reports pages edited by cohort

Story: As a Program leader or grant recipient Wikimetrics user, I want to be able to report on the number of pages on a wiki that the members of my cohort edited during a specified timeframe so that I can report on the global metric "number of articles created or improved in Wikimedia projects".

Pages edited option is made available when the user selects metrics to report for their cohort. The report can contain individual and aggregated results, filtered by namespace, just like the pages created metric.

Notes

Needed for January 2015 Grants reporting standards for Global metrics. Specifically to answer the number of articles improved and for analysis of topical content generation efforts re diversity and gaps.
Because of the way Wikimetrics is configured, this feature will require substantial re-development to implement, so we're putting it on hold. Instead, a template query in Quarry can be used: http://quarry.wmflabs.org/query/1053
https://bugzilla.wikimedia.org/show_bug.cgi?id=73072

------ STORY NEEDED ------

Wikimetrics ignore headers in cohort upload files
share/unshare cohorts with other Wikimetrics users (see draft epic Analytics/Epics/Wikimetrics_cohort_sharing)
edit description of cohort
when creating a report , make it easy to use the same configuration (time windows, namespace) for all the parameter to track (edit, bytes, etc..)
see the report result in the webbrowser
with some cohorts, their involvement in a program always stays the same, that is from a specific STARTTIME to ENDTIME. Would be nice to be able to enter the start/end time of the cohort when the cohort is uploaded, Create a report page automatically sets the default Start time/end time for all the metrics to be that of the event, but still adjustable if needed. Would save a lot of time. --EGalvez (WMF) (talk) 01:05, 15 November 2014 (UTC)[reply]
the ability to add the time your time zone and have it automatically figure out the UTC time. --EGalvez (WMF) (talk) 01:05, 15 November 2014 (UTC)[reply]
Add users to cohorts?? --EGalvez (WMF) (talk) 22:41, 9 January 2015 (UTC)[reply]