User:OrenBochman/Search/Test Plan

Search NG Project Todo List Operational Plan Test Plan Risk Assessment NG Search Spec Search NG Analytics NLP Tools Search Tools Search Labs Configuration Lucene-search Spec Old Code Review Q&A

Overview

Functional Test

• Testing Indexing
  • Indexing of MediaWiki source
  • Indexing of hypertext source
  • Indexing of external links
• Testing Update Mechanism
  • Requires a robot that uploads content into the test Wikipedia installation.
  • Test new pages added
  • Test existing pages are updated
  • Test Indexing propagated from indexer to searcher
  • Test Indexing propagated from searcher to replication.
• Integration tests of component.
  • Tests OI Repository is online and working.
  • Tests Search UI is online and working.
    • Offer search results from Sister Projects
  • Tests Search suggestions are online and working.
  • Tests Search extension is healthy.
    • last update in special pages
  • Test Index is not corrupted.

Relevance Test

• Checks to what degree ranking is altered overtime (by various configuration).
  • Select N=100 queries.
  • Store to 10 results for 100 queries.
  • Associate these with a ranking profile. (Query cooking + Analyzer + Linguistics etc).
  • Order of results is consistent across multiple hosts

Benchmarks

• Benchmark indexing.
• Benchmark testing.

Regression Tests

tests that bugs from the past do not return

Testing Q&A

• Converting Did you know to triavia question for a open relevence format question
• User engagement trivia game.
• why - today the full questions return nothing - this is very bad.

1. w:Stacey Porter
   1. Which Olympic silver bronze medalist was the first Aboriginal member of Australian women's national team to represent softball Olympics.
   2. Which aboriginal Olympic silver bronze medalist Australian national team softball

1. w:Tinirau
   1. Which Devonian stem tetrapod, a transitional species between fish and land vertebrates, was named after the half-human half-fish character in Polynesian legend?
   2. Devonian stem tetrapod Polynesian

1. w:Soft & Cuddly
   1. In which game must the protagonist must reassemble his mother's dismembered body
   2. game mother's dismembered.

1. w:World Naked Gardening Day
   1. What gardening day in May was first observed in 2005?

1. w:Umberto Barulli
   1. Who was the Communist Party leader appointed head of state in San Marino in 1988?
   2. Communist Party leader head of state in San Marino in 1988

1. w:Dynasphere
   1. It was criticized for its poor braking and steering capabilities, as well as its gerbiling tendencies?
   2. w:Special:Search/criticism OR braking OR steering OR gerbiling OR tendencies OR capabilities OR poor OR well)
   3. w:Special:Search/~criticism OR ~braking OR ~steering OR ~gerbiling OR ~tendencies OR ~capabilities OR ~poor OR ~well)
   4. w:Special:Search/braking OR gerbiling
   5. w:Special:Search/(criticism OR capabilities) AND (braking OR steering) AND (gerbiling OR tendencies) works
   6. return monowheel not Dynasphere

Notes

Patterns - delete NE Patterns keep only NE

antitest

crowdsource user feedback directly into results instead of findinf bugs.

Moving Target

• lots of arros - lots of test.
• precise hits - degree of closensss.
• Cps pf missing - special tests.
• how does it move - monitoring.
• unexpectred movement - spike.
• precision + accuracy - non determinism.

• oracles.
• dictionary.