Requests for comment/Markdown

Tracked in Phabricator
Task T137946

Request for comment (RFC)

Develop strategy regarding Markdown
Component	General
Creation date	15 June 2016
Author(s)	RobLa-WMF
Document status	declined See Phabricator.

This document is an attempt to develop a strategy around the Markdown family of markup languages. Is "Markdown support" an important feature for MediaWiki? How important is interoperability with software that claims to support Markdown? Is Markdown an important alternative to MediaWiki syntax that we should think about?

Background

Wikitext has benefited from years of iterative improvement, guided by years of focus on complicated content (such as the Wikimedia sites). For the past few years, MediaWiki users have benefited from the implementation independent implementations.

Wikitext is a very powerful and complicated markup format. For users that need the power (e.g. Wikipedia), the capabilities are essential.

Markdown is a version of wiki markup that seems to return to the wiki markup's plain text origins. From John Gruber's site (via the Markdown Wikipedia article):

Readability, however, is emphasized above all else. A Markdown-formatted document should be publishable as-is, as plain text, without looking like it’s been marked up with tags or formatting instructions. While Markdown’s syntax has been influenced by several existing text-to-HTML filters — including Setext, atx, Textile, reStructuredText, Grutatext, and EtText — the single biggest source of inspiration for Markdown’s syntax is the format of plain text email

Problem

The power of wikitext and the popularity of Wikipedia has created a difficult problem. Wikis were originally designed to reduce complexity. As Ward Cunningham noted in a 1995 email explaining his early wiki software, he pointed out that his new website editing software "has a forms-based authoring capability that doesn't require familiarity with html".

MediaWiki wikitext became complex due to years of evolution. MediaWiki evolved rapidly to keep pace with the rapid growth of Wikipedia, and the corresponding growth in the use of wiki software. Other wiki software was able to differentiate itself from MediaWiki by providing simpler (but less powerful) alternatives to MediaWiki wikitext.

In 2003, the Wikitext standard page was created. The original author wrote:

A single Wikitext standard based on Wikitext syntax as supported by mediawiki is of great importance for the evolution of the Wikiverse:

A clear standard can be published and supported by other wikis, like MoinMoinWiki (-based) and tikiwiki (see sourceforge), which are very popular, and more extensible than mediawiki, which has serious structural problems

Even if mediawiki can keep up with those other packages, text import/export has to be one of our priorities, given the objectives of this project to become a central GNU FDL repository of trusted text for use all over the place.

The Wikimedia activities shouldn't be in any way trapped or tied to mediawiki, if drastically better software emerges from tikiwiki or the Python world. And it will.

In 2016, MediaWiki wikitext seems to be largely relegated to MediaWiki use (and tools that must interoperate with MediaWiki wikis). The alternative parsers page has a long list of MediaWiki-compatible parsers.

Cutting and pasting

MediaWiki content editors would benefit from having good interchange formats for cutting and pasting rich text into MediaWiki (particularly VisualEditor) than the multitude of "text/html compliant" markup out there.

Cutting and pasting text typically involves one of two formats:

1. plain text (as described in section 1.1 of <http://tools.ietf.org/html/rfc5147>)
2. HTML, which includes the horrifying diaspora of complicated "compliant" sources noted above

Raw HTML is a difficult medium to interchange rich text; web browsers are designed to handle the network source of the format as a consideration for what functionality will be enabled in the format. Email illustrates the difficulty. In an effort to support basic formatting (e.g. bold, italic), many graphical email clients supported the text/enriched media type (RFC 1563). This met with resistance from security-minded email recipients, some of which started the ASCII Ribbon Campaign in order to promote a return to plain text.

Proposal

This proposal is for the MediaWiki development community to take a serious look at the Markdown ecosystem.

Markdown seems to be on track to become a representation for markup that preserves the semantics of rich text without introducing the security risks. It has widespread support, and there is IETF work occuring to better specify the format (<https://tools.ietf.org/html/draft-ietf-appsawg-text-markdown-12>).

What this RFC is not

This proposal is not:

• A proposal to replace MediaWiki wikitext in mainstream MediaWiki use (e.g. Wikipedia)
• An attempt to position Markdown as "better" than MediaWiki wikitext. The two are different tools that have developed different ecosystems to solve the problems they each respectively have.

What this RFC is

An attempt for us to figure out how working with Markdown tools and the wider Markdown community might help us achieve better interoperability with the web ecosystem. Can we evolve our platform to support Markdown as a first class citizen (especially for cutting and pasting)?

Powering ahead with HTML

It will be a while before "text/markdown" has widespread, mainstream use (if it ever does). See Phab:T127329 for strategies that deal with the web of today as we know it.

Markdown fragmentation

The Markdown provides this information about Markdown standardization:

There is no clearly defined Markdown standard, apart from the original writeup and implementation by John Gruber, which some consider abandonware. This has led to fragmentation as different vendors write their own variants of the language to correct flaws or add missing features.

CommonMark has emerged as one particularly credible effort to standardize Markdown. Also, the Markdown article also points out:

A number of other lightweight markup languages extend Markdown by adding features (such as tables, footnotes, definition lists, and Markdown inside HTML blocks) not available in plain Markdown syntax. Among these are w:Markdown Extra, w:MultiMarkdown, and the w:Pandoc Markdown extension.

The object model

The Pandoc Markdown documentation describes many of the interesting complexities of supporting “Markdown”, in particular, what the object model for Markdown needs to be. What the team working on Parsoid has discovered is that implementing an object model is a very important step toward allowing two-way conversion between wiki markup and HTML that can be rendered by a browser.

Existing Markdown support

The following extensions are listed in Category:Markdown extensions

• Extension:AlternateSyntaxParser - status: unmaintained
• Extension:MarkdownExtraGeshiSyntax status: unmaintained
• Extension:MarkdownExtraParser - status: unmaintained. From the page: "allows for hooking MediaWiki's parser to use Michael Fortin's PHP Markdown Extra library. It works additional to the out of box Mediawiki Markup spec beside some conflicts.". Author: Trevor N. Suarez (Rican7)
• Extension:MarkdownSyntax - status: unmaintained

Also on Github:

• https://github.com/jdlrobson/AlternateParsers - status: proof of concept published in October 2015

Questions

• Is "Markdown support" an important feature for MediaWiki?
• How important is interoperability with software that claims to support Markdown?
• Is Markdown an important alternative to MediaWiki syntax that we should think about?
• Are there current support not noted?
• Are there any important MediaWIk/Markdown-related efforts not noted in this document?
• Which MediaWiki alternative parsers are still in active use and/or development?
• Is there wiki software besides MediaWiki that uses MediaWiki wikitext as its primary format?