Growth/Personalized first day/Newcomer tasks/Milestone analysis, February 2023
 | This page in a nutshell: This analysis investigated how quickly newcomers make a specific number of Suggested Edits, which are edits tagged with "newcomer task" or variations thereof. We find that this generally happens quickly, and that the limiting factor is the number of edits made rather than the time taken to make them. |
For the Positive reinforcement project, the Growth team is interested in understanding to what extent newcomers are able to complete a certain number of tasks.
In order to learn more about this, we gathered a dataset of 132,597 newcomer task edits made by 27,080 newcomers from five of the Growth team's pilot wikis (Arabic, Bangla, Czech, Spanish, and Vietnamese) as well as seven wikis who got Add a Link during the second round of deployments (Hebrew, Hindi, Korean, Norwegian bokmål, Portuguese, Swedish, and Ukrainian). In this project we were interested in calculating a reasonable estimate of the baselines, rather than seeking to find "true" proportions, and therefore used this limited number of wikis. Newcomers were given 90 days from time of registration to complete edits, and we gathered data from users registered starting the day Suggested Edits was initially deployed on 2019-11-21 until 2022-10-01. Newcomers were required to register on the given wiki, and known test accounts and bots were excluded.
From Figure 1 shown above, we can see that more than 15% of those who make one suggested edit go on to complete five such edits. There is then a rather large drop for those who reach ten edits, and at 15 edits we are almost down to a proportion of 5%. The exact numbers for each threshold are shown below in Table 1.
| Threshold | Number of editors | Proportion |
|---|---|---|
| 5 | 4,405 | 16.3% |
| 10 | 2,255 | 8.3% |
| 15 | 1,461 | 5.4% |
| 20 | 1,105 | 4.1% |
| 25 | 881 | 3.3% |
| 30 | 707 | 2.6% |
| 35 | 587 | 2.2% |
| 40 | 502 | 1.9% |
| 45 | 433 | 1.6% |
| 50 | 369 | 1.4% |
| 55 | 314 | 1.2% |
| 60 | 287 | 1.1% |
| 65 | 260 | 1.0% |
| 70 | 236 | 0.9% |
| 75 | 215 | 0.8% |
We have also investigated this specifically for a variety of task types (e.g. Add a Link, Add an Image, and the unstructured link task). What we have found is that the proportion decays more slowly for Add a Link, and that task dominates the overall statistics because of the large volume of edits generated by that task. For other tasks the decay is faster, which means that we likely cannot require newcomers to make a large number of such tasks as that would mean only a small proportion of newcomers will reach the threshold.
We also investigated how quickly newcomers reached a given threshold, varying the threshold from 2 to 20 edits. A visual representation of that for 2 to 10 edits is shown in Figure 2. As we can see, a newcomer who will reach a given threshold typically does so very quickly as the X-axis in Figure 2 only covers 50 hours.