Jump to content

Developer Satisfaction Survey/2023/Beta

From mediawiki.org

πŸ“–Β Developer Satisfaction 2023

The developer satisfaction survey is an annual survey of the Wikimedia developer community.

πŸ§ͺΒ Beta cluster

[edit]
88%

88% of all comments we coded as negative mentioned at least 1 of 3 things (in order of decreasing frequency):

  • Production-likeness
  • Ownership/Sustainability
  • Reliability

πŸ”— Beta cluster's use over the past year

We asked survey takers, β€œIn the past year, have you used Beta Cluster?”

  • The majority (59%) of respondents said they had used Beta Cluster
  • 36% said they had not used Beta Cluster
  • 5% were unsure

πŸ”— Beta cluster satisfaction

πŸ’¬ We rely on beta to catch issues before they go live to production but we can't in turn often rely on beta!

We asked survey takers, β€œHow satisfied are you with Beta Cluster?”

  • 38% said they were satisfied with Beta Cluster
  • 33% said they were dissatisfied
  • 25% said they were neither satisfied nor dissatisfied
  • 4% were unsure

πŸ”— Beta cluster satisfaction (by role)

  • Test engineers had a majority of respondents indicate they were satisfied with Beta Cluster
  • All but two role groups had more respondents indicate satisfaction than dissatisfaction
  • Tool developers and SRE Infrastructure developers had more respondents indicate disatisfaction than satisfaction

πŸ”— Beta cluster features ranked by priority

πŸ’¬ the beta cluster as it currently exists is trying to be too many things at once.

We asked respondents, β€œHow would you rank the following use cases for Beta Cluster? Rank the following from 1 (most important) to 5 (least important).”

Respondents were asked to order each of the following use cases by priority:

  • Exploratory testing/QA of recent patches
  • Demoing/showcasing
  • Test automation of end-to-end tests
  • Test automation of integration tests
  • Test automation of unit tests
  • Other

These were presented in a random order to each respondent.

The chart below shows descending order of average rank. "Exploratory testing" has the highest average rank and received the most #1 rankings. The use cases with the average lowest rankings were "test automation of integration test" and "test automation of unit tests". The graph does not capture "other".

There was no clear consensus on "Other" priorities. People listed a wide range of priorities such as:

  • developing infrastructure changes
  • developing userscripts/gadgets
  • understanding what's going out with the weekly train deployments

πŸ’¬ What needs to be done is identify 1 or 2 useful use cases, implement them in some other way and then burn the thing to the ground.

πŸ”— Activities on the beta cluster

We asked respondents, β€œDo you engage in any of the following activities on Beta Cluster?”

πŸ”— Activities on the beta cluster (by role)