Wikimedia Technical Documentation Team/Doc metrics/Sources
 | This page is currently a draft.
|
This page provides an overview of the data and research sources TBurmeister (WMF) used to design v0 of the proposed metrics for technical documentation. Where applicable, the page links to the sources and/or to additional information about how these sources were used.
 | We have not yet started the community consultation process for this project; we needed to have a draft set of metrics to consult about! The community discussions section below only refers to the historical content we consulted during the research and design phases, like notes from community events, old Phabricator tasks, etc. |
| Metrics criteria | Doc characteristics and metrics categories | MediaWiki core doc collections | |
| Tech Docs team | ☑ | ☑ | ☑ |
| SME interviews (focused on MediaWiki doc metrics): | ☑ | ☑ | |
| Existing wiki metrics tools | ☑ | ||
| Documentation style guides | ☑ | ||
| Wikimedia metrics projects | ☑ | ☑ | |
| Wikimedia technical documentation projects | ☑ | ☑ | |
| Community discussions in event notes, Phabricator tasks, and Talk pages | ☑ | ☑ | |
| Industry and scholarly literature | ☑ | ☑ |
Wiki metrics tools
[edit]These tools offer data at the page or wiki (aggregate) level. For this project, this data is considered "raw": we can't directly correlate a single piece of data with a meaningful metric, like "accuracy of technical documentation". Some well-known wiki statistics tools and data sources may not appear here because they aren't implemented to include mediawiki.org.
Built-in MediaWiki metrics
[edit]- Page info (example data for Gerrit page)
- Aggregate wiki info: Special:Statistics
- Special:FewestRevisions
- Special:AncientPages
- Special:LongPages
- Special:DeadendPages
- Special:LonelyPages
- Special:MostLinkedPages
- API:Links#Example_1:_Fetch_all_the_links_in_a_page
- Special:MostRevisions
Dashboards
[edit]- https://techdoc-dashboard.toolforge.org/ - this tool is in alpha mode, and is intended to provide aggregate metrics for curated collections of technical documentation on wikis
- XTools (example data for New_Developers page)
- WikiStats (example data for mediawiki.org
- Pageviews tool (example data for mediawiki.org
- Bitergia - has potential to cover more doc-specific dimensions
- PhpMetrics (nothing doc-specific, but this approach to metrics aligns with what could be useful for tech docs)
Extensions
[edit]Style guides and policies
[edit]Documentation style guides
[edit]en-wiki policies
[edit]
Wikimedia metrics projects
[edit]WMF Essential / Standard Metrics
[edit]- Essential_metrics#Essential_metrics_evaluation_dimensions
- Research_and_Decision_Science/Data_glossary
- https://meta.wikimedia.org/wiki/Research:Standard_metrics
- https://meta.wikimedia.org/wiki/Category:Standardized_metric
- https://meta.wikimedia.org/wiki/Research:Page_view
Knowledge Gaps Index
[edit]- meta:Research:Developing_Metrics_for_Content_Gaps_(Knowledge_Gaps_Taxonomy)
- Especially: Choosing_a_Set_of_Metrics and Standard_Quality_Criteria
- meta:Research:Content_gaps_on_Wikipedia
NLP-based tools
[edit]For assessing readability:
- https://meta.wikimedia.org/wiki/Research:Multilingual_Readability_Research
- API endpoint: https://api.wikimedia.org/wiki/Lift_Wing_API/Reference/Get_readability_prediction
- functional user interface on toolforge: https://readability.toolforge.org/
(Also consulted non-wiki-specific readability tools like https://www.expresso-app.org/about, and some tech docs industry discussions of readability metrics.)
Developer surveys
[edit]- Developer Satisfaction Survey 2024
- Developer Satisfaction Survey 2023
- 2020 Cloud Services Annual Survey
- 2021 Cloud Services Annual Survey
- 2022 Cloud Services Annual Survey
Wikimedia technical documentation and docs projects
[edit]MediaWiki technical documentation
[edit]Just a sampling of sources consulted, not a comprehensive index of all MediaWiki tech docs!
- https://doc.wikimedia.org/mediawiki-core/master/php/index.html and https://doc.wikimedia.org/mediawiki-core/master/php/topics.html
- Category:MediaWiki technical documentation
- Project:Categories#Category organization
- Entry point files (Manual:Code, such as Manual:load.php)
- maintenance scripts
- Manual:Hooks
- Manual:Global object variables, Manual:Configuration settings
- API:Main page / https://www.mediawiki.org/w/api.php
Other technical documentation projects
[edit]Sources consulted that are specific to MediaWiki docs:
- Requests_for_comment/Documentation_overhaul
- User:Waldyrious/Docs#Proposed_structure
- User:Zakgreant/MediaWiki Technical Documentation Plan and WMF_Projects/Technical_Documentation(historical)
- Manual:Contents/To do - may be useful to inform workflows / developer tasks for that facet of doc collections
Sources consulted that relate to the larger ecosystem of tech docs, not just to MediaWiki docs.
- Developer Portal/History#Key themes from conversations
- User:TBurmeister (WMF)/Sandbox/Collections
- User:KBach-WMF/Collections/Conclusions
- User:APaskulin (WMF)/Sandbox/Metrics
- User:Pavithraes/Sandbox/Technical documentation prioritization#Prioritizing
Records of past community discussions
[edit]- Wikimedia Technical Conference 2019 Session
- Wikimedia_Hackathon_2017/Lessons_Learned (search for "technical documentation")
- Documentation/Hackathon 2022 docs discussions
- phab:T156301 - "Create a developer documentation special interest group"
- Talk:Requests_for_comment/Documentation_overhaul
- Wikimedia_technical_documentation:_Friends_of_the_Docs and related notes
- https://phabricator.wikimedia.org/T93026
- https://phabricator.wikimedia.org/T113694
- https://phabricator.wikimedia.org/T48526
- https://phabricator.wikimedia.org/T1287
Industry and scholarly literature
[edit]See the references on the Research page.