Wikimedia Release Engineering Team/Checkin archive/20160822

2016-08-22

Vacations/Important dates

How to do it: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Time_off

• Sept 02: Q2 goals draft published
• Sept 05: US Holiday (Labor day) - Greg
• Sept 23: Q2 goals finalized
• Oct 01: Start of Q2
• October 10: US Holiday (Indigenous People's Day)
• October 17-21: Offsite in Washington D.C.
• October 31: Mukunda maybe?
• October 28 - Nov 2 (ish) - Chad
• November 24: US Holiday (Thanksgiving)
• January 9-11: Dev Summit
• January 12-13: All Hands

Team Business

Time spent spreadsheet

• Week 33 - https://docs.google.com/spreadsheets/d/1IrwGPdTDZ6H8x9Mf5dmCYlkK4hZ8sbUSLODEM4cFc4g/edit#gid=734323691

Rotating positions and absences

Maniphest query for deployment blocker tasks: https://phabricator.wikimedia.org/u/blockers

weeks of Aug 8 and Aug 15

• Train: Mukunda
  • wmf.14
  • wmf.15
• SoS: Chad
  • https://phabricator.wikimedia.org/E155/19
  • https://phabricator.wikimedia.org/E155/20
• Out:
  • Zeljko: July 25 - Aug 15
  • Antoine: July 30 - Aug 21

weeks of Aug 22 and Aug 29

• Train: Antoine
  • wmf.16
  • wmf.17
• SoS: Tyler
  • https://phabricator.wikimedia.org/E155/21
  • https://phabricator.wikimedia.org/E155/22
• Out:

weeks of Sep 05 and Sep 12

• Train: Chad
  • wmf.18
  • wmf.19
• SoS: Mukunda
  • https://phabricator.wikimedia.org/E155/23
  • https://phabricator.wikimedia.org/E155/24
• Out:
  • Sept 05 (Monday): US Holiday (Labor day)

Actions from last meeting

• TODO: Chad - lay out ideation on the LongLivedBranches task to then get Timo to review ( https://phabricator.wikimedia.org/T140921 )

Scrum of Scrums

https://phabricator.wikimedia.org/project/board/64/

Blocked on us: https://phabricator.wikimedia.org/maniphest/query/h7YTCBTJsepS/#R

This week

• Blocking
• Blocked
  • Contint networking: https://phabricator.wikimedia.org/T140257#2553490
• Updates
  • Bugfix release of scap

Last week ... Ooops, didn't happen

• Blocking
• Blocked
  • Consolidate, remove, and/or downsize Beta Cluster instances to help with wikitech:Purge_2016 - https://phabricator.wikimedia.org/T142288
    • specifically: -conftool (Ops?/Joe?), -conf3, (Analytics-ops?/Ottomata?), -kafka (Analytics-ops?), OCG (-pdf1 and -pdf2, but only 1 seems to be used?)
• Updates
  • New SWAT window schedule starting Aug 22nd
    • See: https://wikitech.wikimedia.org/wiki/Deployments#Week_of_August_22nd
    • And: https://wikitech.wikimedia.org/wiki/SWAT_deploys#The_team
    • https://phabricator.wikimedia.org/T137970

Other Team Business

Offsite

• Rachel is working on venue options still; some good options so far
• what do you want to talk about? Fill this out/vote on ideas:
  • https://etherpad.wikimedia.org/p/releng-offsite201610-proposedtopics

Q2 (Oct - Dec) Goals

• That time again!
• Out of date: https://www.mediawiki.org/wiki/Wikimedia_Release_Engineering_Team/Goals/201617Q2
  • Differential?
    • fix Jenkins tests, maybe
    • migrate android
    • not a goal
  • MALU
    • pause
  • LLB + MW + Extension deploys to scap3 ?
    • not a goal
  • Python software deployment via scap3 (Zuul + Nodepool)
    • think more on it (Tyler and Antoine), not a goal for now

• CI Tech Debt (meta goal)
  • #1
  • #2
• Experiment/Stretch: MW deploy tech debt:
  • scap swat
  • ability to check something???

Q1 goal/project check-in

Phase out Ubuntu Precise

Replace primary production Continuous Integration host (gallium) - task T95757

• NEXT: https://phabricator.wikimedia.org/T139771 - "Identify metric (or metrics) that gives a useful indication of user-perceived (Wikimedia developer) service of CI"

Upgrade Phabricator database servers to Maria10/Jessie - task T138460

•  Done

Move Gerrit off of ytterbium

•  Done

Reduce Technical Debt

Perform a technical debt analysis of software and services maintained by WMF Release Engineering - task T138225

Hot spots:

• MW 3rd party release, l10nupdate, (and probably swat) tooling
• Nodepool

Next steps?

• ???

Streamline deployments (long-lived branches)

keyresult task:

• Convert our production deployment strategy to use long-lived branches - task T89945

project view: https://phabricator.wikimedia.org/project/view/2117/

Non-Quarterly goal work

SWAT deploy changes

• European SWAT deploys (task T137970
  • NEXT: this week :)
• Future changes?

CI Scaling/Nodepool

• Catch Antoine up on Nodepool state

Browser tests

• https://phabricator.wikimedia.org/T142600 "Various browser tests failing with MediawikiApi::LoginError"
  • Gergo is asking for help figuring out next steps
  • Zeljko is looking into it

Other

DB Inconsistencies

• Figure out how to help Jaime with the DB schema inconsistencies issue:
  • https://phabricator.wikimedia.org/T132416 and https://phabricator.wikimedia.org/T104459 (see also: https://www.mediawiki.org/wiki/Development_policy#Database_patches )
  • What can we do in CI to help prevent, mostly?
  • Question: Does Chad need any help (other than time)?

People status updates

Antoine

Last week

• Vacation

This week

• Catch up on Nodepool incident
• Migrate jobs back to Nodepool instance
• Ideally get quota raised
• Figure out contint1001 network with ops / Tyler
• done: clear out 3 weeks worth of mails
• personal: learn how to play https://www.youtube.com/watch?v=d9i_zXmULyk

Chad

Last week

• DB consistencies ....
• Long lived branches
• CI outage incident report

This week

• MW release today (finally)
• Finally going to do DB consistency script -- per our 1:1 this shouldn't be so hard
• Long lived branches (long may they ilve)

Dan

Last week

• Start poking at MW-Vagrant jessie base image https://phabricator.wikimedia.org/T136429
• Migrate deployment-prep to jessie https://phabricator.wikimedia.org/T138778
• Follow up on mw-selenium/browser tests dependency updates re https://phabricator.wikimedia.org/T129483

This week

• Start poking at MW-Vagrant jessie base image https://phabricator.wikimedia.org/T136429
• Migrate deployment-prep to jessie https://phabricator.wikimedia.org/T138778

Mukunda

Last week

• Mediawiki train: 1.28.0-wmf.15
  • https://phabricator.wikimedia.org/T140971
• LLB
  • Continue working on the `scap swat` tool, more experimentation with gerrit api
    • https://phabricator.wikimedia.org/T142880
    • Made good progress on this: https://phabricator.wikimedia.org/P3854
  • Deploy release-tools repo with scap3 (if time allows)
    • https://phabricator.wikimedia.org/T142588
    • `scap swat` is going to live in mediawiki-config repo, so this is no longer needed.

This week

• Finish the `scap swat` tool which is taking shape nicely.
• Propose Improvements to the scap remote execution api to make it easy to use from scap plugins
  • This could facilitate development of arbitrary scap checks which can be ran separately from deployments
  • Will discuss with Tyler during the deployments meeting and go from there.

Tyler

Last week

• Bugfix scap update
• Try to stay on top of gallium things
• Incident report for CI outage!

This week

• Bugfix scap update
• nodepool things

Željko

Last week

• Catching up

This week

• https://phabricator.wikimedia.org/T142600 Various browser tests failing with MediawikiApi::LoginError (halp plz)
• https://phabricator.wikimedia.org/T139613 Run language screenshots script for VisualEditor in Jenkins
  • https://phabricator.wikimedia.org/T143563 Upload language screenshot files using commons_upload ruby gem (halp plz)
• https://phabricator.wikimedia.org/T139247 Analyze (and share analysis of) the browser testing feedback survey