Jump to content

Architecture meetings/RFC review 2014-06-11

From mediawiki.org

2100 UTC Wednesday, 11 June at #wikimedia-office connect.


Requests for Comment to review

[edit]

Lightning round!

  1. Requests for comment/HTML templating library - per Gabriel Wicke's note, does anyone from Mobile have feedback on the prototype? Initial thoughts on HTML content templating?
  2. Reducing image quality for mobile - the patch has been merged into core. Adam Baso said: "The next step we think is to rewrite the <img> tags on a popular page (e.g., http://en.m.wikipedia.org/wiki/Cats) when Wikipedia Zero is in force in order to give this a run in production." Yuri asked: do we use it via JavaScript rewrite or Varnish-based rewrite?
  3. Requests for comment/Debugging at production server - any thoughts on the patch or proposed implementation?
  4. VERY BRIEF thoughts on Composer if we have time.

Summary and logs

[edit]

Meeting summary

[edit]
  • future RfCs (sumanah, 21:56:53)


Meeting ended at 22:00:09 UTC.


Action items

[edit]


Action items, by person

[edit]
  • dr0ptp4kt
    • dr0ptp4kt Yuri Astrakhan + Adam Baso: try the Cats page next week, then the week after that expand to a full language, then the week after that all languages. Do qualitative testing of image quality + measure bandwidth savings
    • dr0ptp4kt to check in with Brandon Black and either have him as Ops liaison for this project or find someone else :-)
  • greg-g
    • greg-g to ask Bryan about how whether structured logging is a way around the private data issue for this RfC
  • jdlrobson
    • jdlrobson to ask Juliusz, Maryana, Arthur, or similar to schedule time to give KnockOff / TAssembly a spin


Full log

[edit]
Meeting logs

21:00:00 <sumanah> #startmeeting RfC lightning round | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours
21:00:00 <wm-labs-meetbot> Meeting started Wed Jun 11 21:00:00 2014 UTC and is due to finish in 60 minutes.  The chair is sumanah. Information about MeetBot at http://wiki.debian.org/MeetBot.
21:00:00 <wm-labs-meetbot> Useful Commands: #action #agreed #help #info #idea #link #topic #startvote.
21:00:00 <wm-labs-meetbot> The meeting name has been set to 'rfc_lightning_round___channel_is_logged_and_publicly_posted__do_not_remove_this_note___https___meta_wikimedia_org_wiki_irc_office_hours'
21:00:06 <sumanah> #chair sumanah brion TimStarling
21:00:06 <wm-labs-meetbot> Current chairs: TimStarling brion sumanah
21:00:08 <dr0ptp4kt> sumanah: i'm here
21:00:12 <sumanah> #link https://www.mediawiki.org/wiki/Architecture_meetings/RFC_review_2014-06-11
21:00:14 <sumanah> great dr0ptp4kt!
21:00:22 <sumanah> Today we're talking about HTML templating, reducing image quality for mobile, and (more briefly) about Debugging at production server and Composer.
21:00:35 <sumanah> First up: HTML templating library. I'll cap this at 20 min
21:00:36 <sumanah> #topic HTML templating library
21:00:44 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/HTML_templating_library
21:00:55 <sumanah> #link http://lists.wikimedia.org/pipermail/wikitech-l/2014-June/076861.html Gabriel Wicke sent an update to wikitech-l recently
21:01:07 <sumanah> dr0ptp4kt especially: does anyone from Mobile have feedback on the prototype?
21:01:48 <sumanah> and
21:01:48 <sumanah> #link https://www.mediawiki.org/wiki/HTML_content_templating - Anyone have initial thoughts on this?
21:02:04 <sumanah> ( hi Scott_WUaS :-))
21:02:10 <Scott_WUaS> :)
21:02:11 <^d> TimStarling: I think there's a whole lot of talking past one another and trying to decide about 3 things at once.
21:02:19 * brion hmms
21:02:34 <dr0ptp4kt> sumanah: on the image quality stuff, http://lists.wikimedia.org/pipermail/mobile-l/2014-June/007334.html and http://lists.wikimedia.org/pipermail/mobile-l/2014-June/007333.html capture the latest details
21:02:47 <gwicke> as I said in the mail, HTML templating is very early days
21:02:53 <gwicke> for content
21:03:09 <brion> i’m actually quite interested in this
21:03:11 <gwicke> mobile has not tested KnockOff / TAssembly yet, but plans to do so soon
21:03:15 <dr0ptp4kt> sumanah and gwicke, are we going to focus on templating first in this discussion? that way we don't get too many things going at once.
21:03:25 <dr0ptp4kt> i just asked jdlrobson to join in on #wikimedia-office here
21:03:27 <brion> may be a better way to construct some tables than the traditional wikitext template output
21:03:28 <jdlrobson> sumanah: we have not scheduled time to give it a spin. Would be worth poking jgonera / maryana / awjr to ensure we schedule some time to do that
21:03:33 <dr0ptp4kt> so that jdlrobson can speak to templating
21:03:36 <sumanah> dr0ptp4kt: yes, the topic for the next 17 min is templating
21:03:46 <dr0ptp4kt> cool
21:03:47 <sumanah> dr0ptp4kt: we'll talk about image quality next.
21:03:51 <jdlrobson> we've been too swamped with tablet redirect
21:03:53 <dr0ptp4kt> sumanah: thx
21:03:54 <gwicke> brion: yes, especially data-driven tables
21:04:15 <gwicke> we have also been discussing the idea of a data-based table render widget in that context
21:04:19 <brion> of course any time we go to HTML we need to be careful to define sanitization rules
21:04:37 <brion> (eg, use the same rules as we use on wikitext+html now or extend it, or ....)
21:04:50 <gwicke> yes
21:05:14 <gwicke> one of the longer-term ideas behind KnockOff / TAssembly is that you can sanitize templates in the compiler
21:05:32 <brion> nice
21:05:36 <gwicke> and get context-sensitive sanitization of user data in the TAssembly runtime
21:06:20 <gwicke> the split between compiler and TAssembly runtime makes it possible to build different compiler front-ends with different sanitization behavior
21:06:27 * dr0ptp4kt stepping away for 5-10 mins
21:06:29 <sumanah> jdlrobson: ok, I'll mark an action item for you
21:06:40 <sumanah> #action jdlrobson  to ask Juliusz, Maryana, Arthur, or similar to schedule time to give KnockOff / TAssembly a spin
21:06:41 <gwicke> one idea we'll look into soonish is compiling existing MediaWiki messages to TAssembly using Parsoid
21:07:27 <brion> this would probably also combine well with per-template/module CSS style ability (which I plan to help jdlrobson on moving forward with ideas soon)
21:07:28 <gwicke> but it'll be low prio until we have other things done
21:07:35 <brion> for separating data and presentation layers a bit
21:07:58 <gwicke> possibly
21:08:22 <brion> gwicke: that (compilation of messages via TAssembly) might be nice for localization that’s shared with the apps too
21:08:56 <gwicke> to set expectations: if we have a prototype before Q3 I'll be positively surprised
21:09:00 <brion> eg, for templatizing of UI elements for edit warnings �and such
21:09:01 <Scott_WUaS> Hi Sumana and Gabriel (gwicke), In looking through this -https://www.mediawiki.org/wiki/HTML_content_templating - I didn't see where inter-lingual template (for Wikipedia's 300 languages) planning is heading.
21:09:12 <brion> yeah no rush on that, just thinking it could converge nicely :D
21:10:02 <sumanah> #link http://lists.wikimedia.org/pipermail/wikitech-l/2014-June/076846.html spagewmf talking about MobileFrontend's & Flow's experiences
21:10:25 * dr0ptp4kt back
21:10:38 <gwicke> brion: will be very helpful to have your input on this along the way
21:10:52 <sumanah> Scott_WUaS: Hi. I'm sorry for the misunderstanding, but I believe that is not what we are talking about right now. You're welcome to ask more questions about that in #mediawiki or #mediawiki-i18n
21:11:04 <Scott_WUaS> ok thanks ...
21:11:21 <sumanah> (there are different kinds of templates, and I believe you are talking about 1 and we're talking about another)
21:11:54 <Scott_WUaS> I was just looking through this - https://www.mediawiki.org/wiki/HTML_content_templating ... do you have a link for related other kinds of templates?
21:12:54 <Scott_WUaS> It might be good to aggregate all the various template pages on such main pages as this one -https://www.mediawiki.org/wiki/HTML_content_templating - and other main ones.
21:12:55 <sumanah> #info <gwicke> to set expectations: if we have a prototype before Q3 I'll be positively surprised
21:13:42 <sumanah> Scott_WUaS: Let's talk about that in #mediawiki right now and not here, ok?
21:14:31 <Scott_WUaS> sounds good
21:15:08 <sumanah> OK, are there any more questions people have for spagewmf or gwicke or others around Knockout, TAssembly, the RfC, or similar?
21:16:29 <gwicke> (crickets)
21:16:54 <sumanah> ok!
21:17:01 <sumanah> #topic Reducing image quality for mobile
21:17:04 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Reducing_image_quality_for_mobile
21:17:11 <sumanah> #info the patchset has been merged into core! yay!
21:17:18 <sumanah> #link http://lists.wikimedia.org/pipermail/mobile-l/2014-May/007176.html Adam Baso said: "The next step we think is to rewrite the <img> tags on a popular page (e.g., http://en.m.wikipedia.org/wiki/Cats) when Wikipedia Zero is in force in order to give this a run in production."
21:17:20 <awight> gwicke: have u figured out how to feed template expansion into another template's parameters?
21:17:22 <sumanah> that's dr0ptp4kt
21:17:49 <sumanah> #link http://lists.wikimedia.org/pipermail/mobile-l/2014-June/007275.html Yuri asked: do we use it via JavaScript rewrite or Varnish-based rewrite?
21:18:09 <brion> so
21:18:12 <dr0ptp4kt> sumanah: thanks. all, please see http://lists.wikimedia.org/pipermail/mobile-l/2014-June/007334.html and http://lists.wikimedia.org/pipermail/mobile-l/2014-June/007333.html
21:18:15 <gwicke> awight: in wikitext templates?
21:18:23 <brion> my recommendation is to do this either varnish-side or php-side with varnish-side cache splitting
21:19:11 <brion> JS replacements are not going to play well with the browser pre-downloading things
21:19:18 <dr0ptp4kt> right now i would say we're all comfortable with the Wikipedia Zero traffic getting the HTML rewritten to use the lower quality thumbs (regular images never get lower quality'd, though)
21:19:36 <awight> gwicke: yes.  are you planning to reuse the html templating you're working on now for that purpose? (and maybe we should hop channels)
21:19:51 <dr0ptp4kt> that is, php-side, keeping the existing varnish fragmentation inherent with Wikipedia Zero
21:19:53 <gwicke> awight: #mediawiki-parsoid?
21:21:10 <sumanah> #info <brion> my recommendation is to do this either varnish-side or php-side with varnish-side cache splitting ... JS replacements are not going to play well with the browser pre-downloading things
21:21:17 <dr0ptp4kt> as for doing thumbnail quality reduction /always/ on mobile web, what do people think about my latest suggestion? that is (1) always reducing thumbnail size, then (2) doing the lazy loading of the higher quality thumbnail as the user nears the thumbnail, at least when the user has a higher-JS capable device
21:21:28 <sumanah> #link http://lists.wikimedia.org/pipermail/mobile-l/2014-June/007334.html
21:21:34 <sumanah> #link http://lists.wikimedia.org/pipermail/mobile-l/2014-June/007333.html on gzipping
21:21:47 <brion> dr0ptp4kt: it’s something to consider, especially for retina screens
21:21:57 <brion> where balancing size and quality weighs a little differently
21:23:03 <brion> probably we should wait for things to shake out on the zero side and then do some qualitative testing
21:23:06 <brion> heh...
21:23:11 <brion> qualitative testing of the quality ;)
21:23:16 <brion> and quantitative testing of the bandwidth savings
21:24:04 <sumanah> dr0ptp4kt: about how long do you think it will take for things to settle down?
21:24:27 <dr0ptp4kt> brion, agreed. sumanah: we can start with the Cat page probably next week.
21:24:38 <dr0ptp4kt> then i think the week after that we could expand to a full language
21:24:41 <sumanah> http://en.m.wikipedia.org/wiki/Cats ?
21:24:42 <dr0ptp4kt> then the week after that all languages
21:25:02 <dr0ptp4kt> yes, and only if the user is on a Wikipedia Zero network
21:25:09 <brion> awwww so many kitties
21:25:18 <sumanah> REDUCED QUALITY kitties
21:25:24 <brion> :)
21:25:42 <dr0ptp4kt> if it causes problems, someone will get the handle ImSadMeow
21:25:49 <sumanah> ok dr0ptp4kt shall we set an action item on this? if so, assign it to whom?
21:26:59 <dr0ptp4kt> sumanah: yes, you can assign it to yuri and me.
21:27:19 <sumanah> cool.
21:27:27 <brion> i’ll make a note on apps backlog to look into it also (after our ios release probably)
21:27:52 <dr0ptp4kt> action plan: 1. en.m.wikipedia.org/wiki/Cats week one on W0. 2. <somelang>.m.wikipedia.org week 2 on W0. 3. All <langs>.m.wikipedia.org on week 3 on W0.
21:28:24 <sumanah> #action dr0ptp4kt Yuri Astrakhan + Adam Baso: try the Cats page next week, then the week after that  expand to a full language, then the week after that all languages. Do qualitative testing of image quality + measure bandwidth savings
21:29:31 <sumanah> dr0ptp4kt: brion - are you ready to say #agreed on the Varnish/JS question?
21:30:16 <brion> sumanah: yep
21:30:25 <sumanah> go ahead
21:30:39 <brion> #agreed PHP-side modification for low-qual images on zero, splitting cache in Varnish. JS not suitable
21:31:08 <dr0ptp4kt> and it's thumbnails, specifically. no mucking with the regular images atm.
21:31:12 <dr0ptp4kt> (if ever)
21:31:37 <dr0ptp4kt> (maybe someday it could be an option for the user if the user wants to save bandwidth there, too, but let's see how thumbs go)
21:32:22 <brion> yep
21:32:25 <sumanah> #idea maybe someday we could also change original/regular images, not just thumbnails, if the user wants to save bandwidth there, too, but let's see how thumbs go first. - Adam
21:32:49 <sumanah> ok, sounds like we are ready to move on but I'll wait 30 more seconds on this topic
21:32:58 <sumanah> yurikR: any unanswered questions?
21:33:30 <sumanah> TimStarling I presume you're good with this
21:33:54 <TimStarling> I guess so -- is there an ops liason for this?
21:34:01 <ori> 'lo
21:34:10 <TimStarling> presumably there will be some extra server load as the thumbnails are regenerated
21:34:19 <sumanah> ori:  logs till now: http://bots.wmflabs.org/~wm-bot/logs/%23wikimedia-office/20140611.txt starting at 21:00
21:34:26 <ori> thanks
21:34:49 <dr0ptp4kt> i just confirmed with yurikR he's comfortable with action plan
21:34:57 <sumanah> dr0ptp4kt: who is the Ops liaison for this?
21:35:09 <sumanah> #info Yuri is comfortable with this action plan
21:35:31 <dr0ptp4kt> sumanah TimStarling - i would recommend we liaise with bblack, as he's our normal point of contact on w0 stuff
21:36:02 <TimStarling> yes, sounds good
21:36:21 <sumanah> #action dr0ptp4kt to check in with Brandon Black and either have him as Ops liaison for this project or find someone else :-)
21:36:49 <sumanah> ok, anything else, or shall we move on to #topic Debugging at production server?
21:37:17 * sumanah waits 20 more seconds or so :)
21:37:45 <sumanah> #topic Debugging at production server
21:38:08 <sumanah> #link https://www.mediawiki.org/wiki/Requests_for_comment/Debugging_at_production_server
21:38:14 <sumanah> #info This RfC is by devunt. I asked devunt to be here but I didn't give him/her enough notice, sorry
21:38:20 <sumanah> #info "Problem: Sometimes we have to debug on production wiki, but don't want to show internal information to normal users... But the current architecture of debugging toolbar is available for everyone, so some internal information, like the server's directory structure, debug logs, and so on, can be leaked."
21:38:35 <sumanah> greg-g: chrismcmahon I'd especially like your opinion on this one.
21:38:40 <sumanah> #info Proposal: change things so that only selected users can use the debugging toolbar, and implement this using user rights.
21:38:56 <sumanah> #link https://gerrit.wikimedia.org/r/#/c/119002/ is the patch
21:39:41 <greg-g> how does one enable/see the toolbar now?
21:39:52 <^d> Enable it in settings for all users.
21:39:55 <^d> Or none (default)
21:40:07 <^d> $wgDebugToolbar I believe is the name.
21:40:09 <brion> hmmmm, yeah being able to restrict that to certain users migh t b e handy
21:40:26 <sumanah> is this something that's most applicable to non-WMF wikis?
21:40:36 <^d> I think we could definitely use it on beta.
21:40:41 <^d> I've wanted it before.
21:40:48 <brion> tell the truth i’d LOVE to be able to turn that on in wmf land too, or near-equivalent
21:40:50 <brion> yeah :)
21:41:00 <TimStarling> I wonder what the most sensitive private data in the debug logs is
21:41:02 <sumanah> #info at least some people are interested in turning this on in WMF land, such as beta
21:41:06 <sumanah> csteipp: ^ thoughts?
21:41:10 <TimStarling> i.e. how well does it need to be protected?
21:41:29 <TimStarling> I mean for the main site
21:41:35 <greg-g> well, just a reminder, we still want to protect user ips/etc in beta cluster :/
21:41:44 <brion> i dunno if something might be exposable in user-to-user communications
21:41:51 <brion> or if background job processing happens
21:41:53 <greg-g> it's probably going to get lumped into the big pile of mess that is NDA signing/tracking
21:42:03 <chrismcmahon> sumanah: I've never used much less heard of the debugging toolbar, I'm here for the background.  having something like this in beta labs would be handy.
21:42:34 <^d> If we made this depend on the structured logging stuff we might be a little more confident about turning it on in WMF land.
21:42:46 <^d> Presumably it'd be easy to mark a log as private at that point.
21:43:01 <greg-g> (continuing my thought) .. unless it doesn't show PII?
21:43:06 <brion> http://www.mediawiki.org/wiki/Debugging_toolbar <- for those not familiar with the feature
21:43:16 <TimStarling> obviously we don't use $wgJobRunRate on WMF
21:43:16 <sumanah> #link http://www.mediawiki.org/wiki/Debugging_toolbar <- for those not familiar with the feature
21:43:28 <^d> TimStarling: Yeah, so jobs aren't a worry.
21:43:29 <TimStarling> do exception backtraces appear in the logs?
21:43:39 <TimStarling> I would worry about things like redis passwords
21:43:51 <sumanah> We could assign devunt to look into how to do this in a way that respects what we're trying to protect https://www.mediawiki.org/wiki/Security_for_developers/Architecture#What_are_we_trying_to_protect.3F
21:44:16 <^d> TimStarling: I'm also a bit leery about something like Special:CheckUser. User's name/IP might end up in debug messages easily.
21:44:27 <^d> Easily checked, but still. We've got a lot of extensions.
21:45:00 <TimStarling> yeah, it would be a medium for data release that developers were not expecting
21:45:16 <TimStarling> so there's potentially a lot of ways for things to go wrong
21:45:29 <^d> So, back to my earlier point. Could we make this depend on structured logging?
21:45:30 <sumanah> (Chris is now reading the backscroll :-) )
21:45:55 <TimStarling> not sure how structured logging helps...
21:46:24 <^d> Maybe if we had a way to flag a log as private or not.
21:46:26 <^d> Just a thought.
21:47:12 <greg-g> doesn't seem to be in the RfC for structured logging
21:47:38 <sumanah> and I think Bryan is unavailable to talk right now
21:47:51 <greg-g> yeah, he's on the beach
21:47:55 <^d> Fair enough, I just wanted to mention it as a possible way around the private data issue.
21:47:55 <csteipp> Yeah, I would worry about things like redis/db passwords showing up... but I'd have to look at it a little more
21:48:17 <TimStarling> well, we have a private flag in wfDebugLog()
21:48:34 <TimStarling> not sure how much it helps
21:48:39 <sumanah> csteipp: in my opinion that can wait - let's give https://www.mediawiki.org/wiki/Security_for_developers/Architecture#What_are_we_trying_to_protect.3F to devunt and ask him/her to draw The Data Flow Diagram or otherwise project-manage this
21:48:53 <^d> today I learned we have a private flag in wfDebugLog().
21:49:05 <sumanah> I don't think you need to proactively look into this and push your other 4 full-time jobs aside ;-) right now
21:49:09 <^d> So to summarize because I think we're all on the same page here.
21:49:30 <TimStarling> yeah, so ask yourself: if someone else had introduced structured logging, and they added a private flag, would you know about that and use it?
21:49:30 <^d> I think we like the idea in theory of making the debug log available to trusted users (especially on beta)
21:49:32 <greg-g> ^d: TimStarling that isn't mentioned in the structured logging RfC, is that needed there?
21:49:35 <^d> But we have to worry about private data.
21:49:54 <^d> TimStarling: Not necessarily, but I hadn't thought about it until today :p
21:50:03 <TimStarling> greg-g: maybe
21:50:18 <greg-g> TimStarling: I'll bug bryan about it when he's back
21:50:54 <sumanah> #action greg-g to ask Bryan about how whether structured logging is a way around the private data issue for this RfC
21:51:26 <sumanah> #info <csteipp> Yeah, I would worry about things like redis/db passwords showing up... but I'd have to look at it a little more
21:51:35 <greg-g> it'd probably only be part of a solution, honestly
21:51:53 <greg-g> even if structured logging was perfect in segmenting out private/non-private
21:52:27 <^d> I think devunt's approach is fine for core, if not sufficient for WMF.
21:53:30 <sumanah> ok, so, this caused more interest than I expected
21:54:06 <sumanah> is there anything else people want to say here? I'll be pasting the summary onto the RfC talk page.
21:54:57 <sumanah> #action devunt to review https://www.mediawiki.org/wiki/Security_for_developers/Architecture#What_are_we_trying_to_protect.3F and draw a data flow diagram https://www.mediawiki.org/wiki/Security_for_developers/Architecture#Threat_Modeling and reach out to Chris Steipp once that's done
21:55:00 <greg-g> I think that covers the preliminary points until we get a better idea on the data flow
21:55:03 <csteipp> +1. Now that I've read the rfc, I do like it in concept for core. I'll think more about it for wmf..
21:55:56 <sumanah> devunt has gotten a lot of "I don't see the use case" in the changeset comments https://gerrit.wikimedia.org/r/119002 in case you want to go counter that
21:56:47 <sumanah> OK, I think we do not have time to continue the epic Composer thread right here, so I suggest we wrap up early. The 1 housekeeping thing I will say is:
21:56:53 <sumanah> #topic future RfCs
21:57:15 <sumanah> Please feel free to push forward on your own RfCs & ones you care about, by bringing them up onlist
21:57:17 <greg-g> marktraceur: ^^ the bot /topic issue is fixed, fyi
21:57:24 <marktraceur> Yaaaaaaaaaay
21:57:43 <sumanah> I think I fixed it by setting the name of the meeting to:
21:57:44 <sumanah> RfC lightning round | Channel is logged and publicly posted (DO NOT REMOVE THIS NOTE). https://meta.wikimedia.org/wiki/IRC_office_hours
21:57:49 <marktraceur> Yeah
21:57:56 <marktraceur> Oh well.
21:57:58 <greg-g> oh, sumanah's just on it ;)
21:58:02 <marktraceur> #soon
21:58:08 <Scott_WUaS> Thanks, Sumana!
21:58:14 <sumanah> Future RfC discussions - I don't necessarily know what you're blocked on or waiting for, so please shout onlist :-)
21:58:31 <sumanah> Anything people particularly want to discuss or get decisions on in the next few weeks?
21:59:04 <sumanah> I particularly have my eye on SOA auth https://www.mediawiki.org/wiki/Requests_for_comment/SOA_Authentication
21:59:23 <^d> I should write an RfC on version numbering.
21:59:57 <sumanah> given that meetings are the opposite of meditation, I find it ironapropos that I now need to close this meeting to go meditate
22:00:05 <sumanah> catch ya later
22:00:09 <sumanah> #endmeeting
Retrieved from "https://www.mediawiki.org/w/index.php?title=Architecture_meetings/RFC_review_2014-06-11&oldid=1033908"