Content translation/Development Plan/Roadmap/CX01Release

Content Translation 0.01 release

Release Announcement

See below for the detailed development plan for each of these features

Increase language support (Labs->Beta->Prod)

Languages with high-quality support through Apertium
1. Catalan -> Spanish (reverse language pair)
2. Spanish -> Portuguese
3. Maltese -> Arabic (RTL)
Languages well supported by Apertium, but will require better markup alignment support
1. English - Spanish
2. English - Catalan

Feature Set

Entry Points
1. Red Interlanguage Link (implemented; needs integration)
2. New entry point showing in Wikidata.
3. Entry points showing by: editing on other languages, Babel, browser language, Accept-Language
Language tools
1. Editing support (basic text styling)
2. Existing paragraph translation - only design, not coding (improvement)
3. Display disambiguation page links for link adaptation
4. Increase number of tools Eg: MT backends, dictionary providers, Reference
Translation dashboard
1. continue ongoing translations, suggestions and impact (enhancement)
Templates
1. Pass through template support
2. Inline templates adaptation - show placeholders instead of removing
3. Category adaptation (not addition/editing)
Infrastructure improvements (improvement)
1. Improve Machine translation abuse algorithms
2. Latest upstream Apertium requires for supporting more language pairs
3. MT Infrastructure enhancement- Work on leftover technical items on MT backend
4. Service provider registry
5. Performance
6. Test coverage
Basic Analytics Dashboard: (enhancement)
1. Collection
2. Visualization

Production Deployment - Resources & Provisioning

Milestones

Completion Date/Milestones	Features	Sprints
August 18 - August 29 2014		73
September 2014		74-75

Development Plan

Mingle Story Board

Feature	Details
Entry Points	Red interlanguage link
Editor	Layout and Design Design a simple experimental editing interface for CX Floating Header Progress bar Show progress indicator for MT Tools column Editing Allow restoring MT after editing Fix erroneous appearance of cx-segment span in translation Highlight segment pairs on hover Configurable translation guidelines link Reference Adaptation Automatic Translation Publishing bug 66200 Warn users attempting to translate (and publish) translation of existing articles
Link Adaptation	Handle red links in the source column Link adaptation - add link by clicking on source links (for empty translation paragraph, no MT-related) References, inline markup: apply to MT candidate text Show blue highlight on links when clicked and remove when clicked outside
Machine Translation Support (mt)	ContentTranslation tool service providers registry Send simpler HTML to MT API Tornado-based API interface for cxserver Calculate MT abuse according to already translated text
Dictionary Support	Hide the dictionary card till result received Dictionary, take language parameters so that we can show synonyms
Templates Support	Automatic handling for non-adapted templates
Architecture (technical feature)	Refactor tools modules to make self contained
Research and preliminary development

CX Deployment Plan for 0.01 Release September 2014

Deployment date: TBD (September 20-25 2014)

Project: Content Translation Framework

Release: 0.01 (second release after the MVP release)

Long-term project roadmap: Content_translation/Roadmap

Language Pairs to be supported: Catalan-Spanish, Spanish-Portuguese, Maltese-Arabic, English-Spanish, English-Catalan

Release as: Beta Feature

Overall Plan

System Architecture

See: https://www.mediawiki.org/wiki/Content_translation/Technical_Architecture

https://www.mediawiki.org/wiki/Content_translation#Workflow_and_Technical_Architecture

https://www.mediawiki.org/wiki/Content_translation

Caching Architecture

The following diagram includes the caching requirements for the CX framework.

https://www.mediawiki.org/wiki/Content_translation/Server_communications_workflow

https://commons.wikimedia.org/wiki/File:CX_ArchitectureV1.svg

Components to be provisioned for production

CX server installation and configuration: https://git.wikimedia.org/markdown/mediawiki%2Fservices%2Fcxserver.git/HEAD/README.md

See Setup: https://www.mediawiki.org/wiki/Content_translation/Setup for detailed information about component, installation and configuation and instructions.

Node.js

Dictd server (Also see: https://www.mediawiki.org/wiki/Content_translation/Dictionaries)

Apertium

Extension dependencies:
- BetaFeatures
- CLDR
- EventLogging

Backend Services

Varnish:

External APIs called by CX
- Wikidata
- Parsoid API

Configuration Scripts

Upstart and Systemd scripts are at: https://www.mediawiki.org/wiki/Content_translation/Setup

Provisioning Plan

Storage Requirements

To be determined from discussion with ops

Hardware Requirements

To be determined from discussion with ops

Bandwidth Requirements

To be determined from discussion with ops

Performance expectations

- MT TPS (Transactions per second)
- User responsiveness
- MT Round trip
- General guidelines

Monitoring and metrics

EventLogging activity for CX
Number of users enabling the feature
Performance of S:CX, backend calls?
Check for node and varnish? Who to page?
Graph showing requests or timings for the WikiData API(s) we are calling
Graph showing requests or timings for the Parsoid API(s) we are calling

External Signoffs Required

Faidon - Ops
Gabriel - Infrastructure architecture
Ori - Performance
Chris Steipp - Security
Greg G - Release engineering
Mark - Ops
Tim - Platform

LE Team responsibilities

Kartik - Deployment, Engineer
Niklas - Engineer, Code Reviewer
Santhosh - Engineer, Code Reviewer
David - Engineer, Code Reviewer
Joel - Engineer, Code Reviewer
Runa - Team Scrum-Ninja / testing and communications
Pau - Feature UX reviewer, designer
Amir - Feature signoff
Alolita - Engineering coordination, Eng Manager