Jump to content

Readers/Web/Team/Etherpad/Architecture Review Q3 2012-2013

From mediawiki.org

Slides: [1]

Topcis to be discussed:

  • Caching
    • Report current caching metrics
      • 53%
    • Removing need for variance by device (ESI)
  • Settings & session cookie
  • Dynamic sections
  • What does mobile need from ops


Current state of mobile architecture

[edit]

Overview of request flow:

  1. squid checks mobile
  2. if m.* domain then (c-code) sends to varnish

Vary headers (HTTP):

  • accept-Encoding
  • X-Device (20+ values)
  • Cookie (this will bypass varnish)
  • X-Carrier
    • set by Varnish based on carrier ACLs
    • used by zero for banner
    • Candidate for ESI
  • X-Subdomain
    • used by ZeroRatedMobileAccess
    • possibly being phased out
    • good candidate to delete (it's splitting the cache unnecessarily)
  • X-Images
    • used by ZeroRatedMobileAccess

Cookies that bypass varnish:

  • disable (disable images)
  • optin (alpha/neta)
  • session (if login, or if mobile settings form visited)
  • forceHTTPS (after login)
  • Gabriel: can we consolidate a lot of headers into a cookie?
  • Juliusz: does this really make hit rate better? (answer: not really)

TTL:

  • s-maxage (31 days)
  • frontend varish: 5 minutes

Cache flushes:

  • used to be on every deployment, now les frequently (RL usage)
  • do this when page HTML changes
  • Mark notes that during the Vector deployment, we purged the cache over the course of a day

Cache-hit ratios:

  • ~53%

Backend App Server Usage:

  • mobile is currently 5% of overall pageviews
  • look at an app server apache over the last 5 minutes, 45% of all backend requests were from mobile frontend

??

[edit]

Increase hit rate:

  • remove variance on x-device
    • via ESI?
    • redesign JS/CSS to make universal HTML? [most of the new code is device-independent; some base code still varies and should be fixed?]
      • viewport locking discussion (HTML-based, not JS-based) - Asher is hoping we can do as much as possible in Javascript
        • Change-Id: I9e3e387cc3454e05156c2e285982eae36cbb85c7 was what brought this in (certain browsers zoom when you click - Opera. Maybe better way to do this?)
      • need to review this

Target mobile varnish hit rate?

  • 80-90% target?
  • or more modest: 70%?
  • Tim: X-Device is a big win. Beyond that, maybe increasing the cache size. 53% hit rate isn't all that bad, really.
  • Asher: 45% of Apache requests are coming from the m domain. 10% of the time is in the MobileFrontend code
  • Mark: separate pool?
  • Tim: three ways to look at Mobile extension:
    • Mobile skin
    • Mobile API
    • Mobile Frontend
  • X-Subdomain header can be used to help profile.

Other areas:

  • continued improvement in mobile resoruceloading
  • reassess WAP support (.03% total traffic)
    • kill off support?
  • dynamic section loading
    • experimented in the past with this in alpha/beta site

Q4 roadmap?

  • photo upload refinements
  • image galleries on articles
  • adding categories to images
  • voting on images
  • update out-of-date-images on articles
  • article editng
  • nearby (geolocate articles to device)

Concerns:

  • improving/replace CentralAuth
    • cookies in Safari are broken because SSO image cookeis are blocked (cross domain blocking)
  • OSM tile rendering
  • Solr support (Geolocation)
  • other mobile contribs stuff?

Needs from Ops:

  • support and recommendatiosn on perormance improvements
  • OSM tile rendering on cluster
  • support for Solr on cluster
    • There are different engineering teams that are planning to leverage solr
    • this potentially creates fragmentation
  • support in capacity planning as contributory features are added

Problem areas

[edit]

ESI:

  • Arthur: Case against ESI?
  • Asher: ESI would be fine in many cases. X-Device is a problem because there's going to be an internal request for every single request. We'd prefer to do this with Javascript
  • Mark: We don't want to use ESI for everything
  • Tim: is ESI not cacheable?
  • Mark: it is, but it still uses a lot of processor. We haven't done a lot of profiling with this though.
  • Tim: using it for X-Carrier is pretty obvious, right?
  • Mark: yup
  • Tim: we can profile it after we set that up.
  • Max: (missed this - discussion about varying CSS)
  • Asher: it would be nice if CSS came from m domain
  • Tim: a lot better to vary CSS than HTML
  • Brion: need to make sure we don't trip over ResourceLoader

Dynamic sections in mobile section of website Original motivation: 50% of visits didn't need full page or images. The original design involved loading the page via API and displaying via DOM

Gabriel: header overhead may negate performance gains Brion: we're investigating different options Juliusz: perhaps instead of separate request, we can possibly put in Javascript.

Other concerns:

  • Mark: no other concerns. section loading is biggest concern
  • Juliusz: could we have an alias for commons.wikipedia.org?

Chris will be primary contact point in Platform for the CentralAuth stuff

/apple-touch-icon-precomposed.png is currently the most popular cache missed URL, and 404s. Perhaps we should provide :)

Retrieved from "https://www.mediawiki.org/w/index.php?title=Readers/Web/Team/Etherpad/Architecture_Review_Q3_2012-2013&oldid=3045080"